2024-09-11T06:59:52.691 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-11T06:59:52.705 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-11T06:59:52.792 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/gabrioux-2024-09-11_06:53:43-orch:cephadm-wip-guits-quincy-2024-09-10-2008-quincy-distro-default-smithi/7901202 branch: wip-guits-quincy-2024-09-10-2008-quincy description: orch:cephadm/with-work/{0-distro/centos_9.stream fixed-2 mode/root mon_election/connectivity msgr/async-v1only start tasks/rados_python} email: gabrioux@ibm.com first_in_suite: false job_id: '7901202' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi name: gabrioux-2024-09-11_06:53:43-orch:cephadm-wip-guits-quincy-2024-09-10-2008-quincy-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 10 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-guits-quincy-2024-09-10-2008-quincy ceph: conf: global: mon election default strategy: 3 ms bind msgr2: false ms type: async mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: debug ms: 1 debug osd: 20 osd shutdown pgref assert: true flavor: default log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - but it is still running - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(PG_ - \(OSD_ - \(OBJECT_ - \(POOL_APP_NOT_ENABLED\) mon_bind_msgr2: false sha1: d0e6828a2016d48cf25ad84064e50742bb1c39b9 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} cephadm: cephadm_mode: root install: ceph: extra_system_packages: deb: - python3-pytest rpm: - python3-pytest flavor: default sha1: d0e6828a2016d48cf25ad84064e50742bb1c39b9 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 workunit: branch: wip-guits-quincy-2024-09-10-2008-quincy sha1: d0e6828a2016d48cf25ad84064e50742bb1c39b9 owner: scheduled_gabrioux@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - ceph.rgw.foo.a - node-exporter.a - alertmanager.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - grafana.a - node-exporter.b - ceph.iscsi.iscsi.a seed: 2282 sha1: d0e6828a2016d48cf25ad84064e50742bb1c39b9 sleep_before_teardown: 0 subset: 1/8 suite: orch:cephadm suite_branch: wip-guits-quincy-2024-09-10-2008-quincy suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_d0e6828a2016d48cf25ad84064e50742bb1c39b9/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: d0e6828a2016d48cf25ad84064e50742bb1c39b9 targets: smithi071.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKBhfIl1x21CJ0m55NC935ycZ7HKzmEE396cnqPDECnXWQAZM0UinMD00x01na78icNc4hSpIwrVCHGLV4ucqzg= smithi196.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDLnvsybwPhPVLHSKioVWYLxwN9ZM1jnuuCBH/6SaEFWgcM+UBr5rPz56ojAyDrz/Ft/iiMbDiqAwjvpG9kOdro= tasks: - pexec: all: - sudo dnf remove nvme-cli -y - sudo dnf install nvmetcli nvme-cli -y - install: null - cephadm: conf: mgr: debug mgr: 20 debug ms: 1 - workunit: clients: client.0: - rados/test_python.sh teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-11_06:53:43 tube: smithi user: gabrioux verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-11T06:59:52.792 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_d0e6828a2016d48cf25ad84064e50742bb1c39b9/qa; will attempt to use it 2024-09-11T06:59:52.793 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_d0e6828a2016d48cf25ad84064e50742bb1c39b9/qa/tasks 2024-09-11T06:59:52.793 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-11T06:59:52.794 INFO:teuthology.task.internal:Checking packages... 2024-09-11T06:59:52.814 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'd0e6828a2016d48cf25ad84064e50742bb1c39b9' 2024-09-11T06:59:52.814 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-11T06:59:52.815 INFO:teuthology.packaging:ref: None 2024-09-11T06:59:52.815 INFO:teuthology.packaging:tag: None 2024-09-11T06:59:52.815 INFO:teuthology.packaging:branch: wip-guits-quincy-2024-09-10-2008-quincy 2024-09-11T06:59:52.815 INFO:teuthology.packaging:sha1: d0e6828a2016d48cf25ad84064e50742bb1c39b9 2024-09-11T06:59:52.815 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-guits-quincy-2024-09-10-2008-quincy 2024-09-11T06:59:52.958 INFO:teuthology.task.internal:Found packages for ceph version 17.2.7-1623.gd0e6828a 2024-09-11T06:59:52.960 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-11T06:59:52.966 INFO:teuthology.task.internal:no buildpackages task found 2024-09-11T06:59:52.966 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-11T06:59:53.001 INFO:teuthology.task.internal:Saving configuration 2024-09-11T06:59:53.012 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-11T06:59:53.027 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-11T06:59:53.054 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi071.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/gabrioux-2024-09-11_06:53:43-orch:cephadm-wip-guits-quincy-2024-09-10-2008-quincy-distro-default-smithi/7901202', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-11 06:54:57.975064', 'locked_by': 'scheduled_gabrioux@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKBhfIl1x21CJ0m55NC935ycZ7HKzmEE396cnqPDECnXWQAZM0UinMD00x01na78icNc4hSpIwrVCHGLV4ucqzg='} 2024-09-11T06:59:53.079 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi196.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/gabrioux-2024-09-11_06:53:43-orch:cephadm-wip-guits-quincy-2024-09-10-2008-quincy-distro-default-smithi/7901202', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-11 06:54:57.973235', 'locked_by': 'scheduled_gabrioux@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDLnvsybwPhPVLHSKioVWYLxwN9ZM1jnuuCBH/6SaEFWgcM+UBr5rPz56ojAyDrz/Ft/iiMbDiqAwjvpG9kOdro='} 2024-09-11T06:59:53.079 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-11T06:59:53.085 INFO:teuthology.task.internal:roles: ubuntu@smithi071.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'ceph.rgw.foo.a', 'node-exporter.a', 'alertmanager.a'] 2024-09-11T06:59:53.085 INFO:teuthology.task.internal:roles: ubuntu@smithi196.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'grafana.a', 'node-exporter.b', 'ceph.iscsi.iscsi.a'] 2024-09-11T06:59:53.086 INFO:teuthology.run_tasks:Running task console_log... 2024-09-11T06:59:53.166 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fa2e5eabac0>, signals=[15]) 2024-09-11T06:59:53.166 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-11T06:59:53.172 INFO:teuthology.task.internal:Opening connections... 2024-09-11T06:59:53.172 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi071.front.sepia.ceph.com 2024-09-11T06:59:53.174 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi071.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T06:59:53.261 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi196.front.sepia.ceph.com 2024-09-11T06:59:53.262 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T06:59:53.338 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-11T06:59:53.345 DEBUG:teuthology.orchestra.run.smithi071:> uname -m 2024-09-11T06:59:53.363 INFO:teuthology.orchestra.run.smithi071.stdout:x86_64 2024-09-11T06:59:53.363 DEBUG:teuthology.orchestra.run.smithi071:> cat /etc/os-release 2024-09-11T06:59:53.422 INFO:teuthology.orchestra.run.smithi071.stdout:NAME="CentOS Stream" 2024-09-11T06:59:53.422 INFO:teuthology.orchestra.run.smithi071.stdout:VERSION="9" 2024-09-11T06:59:53.422 INFO:teuthology.orchestra.run.smithi071.stdout:ID="centos" 2024-09-11T06:59:53.422 INFO:teuthology.orchestra.run.smithi071.stdout:ID_LIKE="rhel fedora" 2024-09-11T06:59:53.422 INFO:teuthology.orchestra.run.smithi071.stdout:VERSION_ID="9" 2024-09-11T06:59:53.422 INFO:teuthology.orchestra.run.smithi071.stdout:PLATFORM_ID="platform:el9" 2024-09-11T06:59:53.422 INFO:teuthology.orchestra.run.smithi071.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-11T06:59:53.422 INFO:teuthology.orchestra.run.smithi071.stdout:ANSI_COLOR="0;31" 2024-09-11T06:59:53.423 INFO:teuthology.orchestra.run.smithi071.stdout:LOGO="fedora-logo-icon" 2024-09-11T06:59:53.423 INFO:teuthology.orchestra.run.smithi071.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-11T06:59:53.423 INFO:teuthology.orchestra.run.smithi071.stdout:HOME_URL="https://centos.org/" 2024-09-11T06:59:53.423 INFO:teuthology.orchestra.run.smithi071.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-11T06:59:53.423 INFO:teuthology.orchestra.run.smithi071.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-11T06:59:53.423 INFO:teuthology.orchestra.run.smithi071.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-11T06:59:53.424 INFO:teuthology.lock.ops:Updating smithi071.front.sepia.ceph.com on lock server 2024-09-11T06:59:53.446 DEBUG:teuthology.orchestra.run.smithi196:> uname -m 2024-09-11T06:59:53.464 INFO:teuthology.orchestra.run.smithi196.stdout:x86_64 2024-09-11T06:59:53.465 DEBUG:teuthology.orchestra.run.smithi196:> cat /etc/os-release 2024-09-11T06:59:53.523 INFO:teuthology.orchestra.run.smithi196.stdout:NAME="CentOS Stream" 2024-09-11T06:59:53.523 INFO:teuthology.orchestra.run.smithi196.stdout:VERSION="9" 2024-09-11T06:59:53.523 INFO:teuthology.orchestra.run.smithi196.stdout:ID="centos" 2024-09-11T06:59:53.523 INFO:teuthology.orchestra.run.smithi196.stdout:ID_LIKE="rhel fedora" 2024-09-11T06:59:53.523 INFO:teuthology.orchestra.run.smithi196.stdout:VERSION_ID="9" 2024-09-11T06:59:53.524 INFO:teuthology.orchestra.run.smithi196.stdout:PLATFORM_ID="platform:el9" 2024-09-11T06:59:53.524 INFO:teuthology.orchestra.run.smithi196.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-11T06:59:53.524 INFO:teuthology.orchestra.run.smithi196.stdout:ANSI_COLOR="0;31" 2024-09-11T06:59:53.524 INFO:teuthology.orchestra.run.smithi196.stdout:LOGO="fedora-logo-icon" 2024-09-11T06:59:53.524 INFO:teuthology.orchestra.run.smithi196.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-11T06:59:53.524 INFO:teuthology.orchestra.run.smithi196.stdout:HOME_URL="https://centos.org/" 2024-09-11T06:59:53.524 INFO:teuthology.orchestra.run.smithi196.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-11T06:59:53.524 INFO:teuthology.orchestra.run.smithi196.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-11T06:59:53.524 INFO:teuthology.orchestra.run.smithi196.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-11T06:59:53.525 INFO:teuthology.lock.ops:Updating smithi196.front.sepia.ceph.com on lock server 2024-09-11T06:59:53.546 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-11T06:59:53.555 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-11T06:59:53.576 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-11T06:59:53.576 DEBUG:teuthology.orchestra.run.smithi071:> test '!' -e /home/ubuntu/cephtest 2024-09-11T06:59:53.579 DEBUG:teuthology.orchestra.run.smithi196:> test '!' -e /home/ubuntu/cephtest 2024-09-11T06:59:53.595 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-11T06:59:53.600 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-11T06:59:53.600 DEBUG:teuthology.orchestra.run.smithi071:> test -z $(ls -A /var/lib/ceph) 2024-09-11T06:59:53.637 DEBUG:teuthology.orchestra.run.smithi196:> test -z $(ls -A /var/lib/ceph) 2024-09-11T06:59:53.748 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-11T06:59:53.802 INFO:teuthology.run_tasks:Running task kernel... 2024-09-11T06:59:53.817 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-11T06:59:53.817 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-11T06:59:53.817 DEBUG:teuthology.orchestra.run.smithi071:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-11T06:59:53.818 DEBUG:teuthology.orchestra.run.smithi196:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-11T06:59:53.834 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T06:59:53.834 DEBUG:teuthology.orchestra.run.smithi196:> uname -r 2024-09-11T06:59:53.835 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T06:59:53.835 DEBUG:teuthology.orchestra.run.smithi071:> uname -r 2024-09-11T06:59:53.892 INFO:teuthology.orchestra.run.smithi196.stdout:5.14.0-503.el9.x86_64 2024-09-11T06:59:53.893 INFO:teuthology.task.kernel:Running kernel on smithi196: 5.14.0-503.el9.x86_64 2024-09-11T06:59:53.893 DEBUG:teuthology.orchestra.run.smithi196:> sudo yum install -y kernel 2024-09-11T06:59:53.894 INFO:teuthology.orchestra.run.smithi071.stdout:5.14.0-503.el9.x86_64 2024-09-11T06:59:53.894 INFO:teuthology.task.kernel:Running kernel on smithi071: 5.14.0-503.el9.x86_64 2024-09-11T06:59:53.894 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum install -y kernel 2024-09-11T06:59:56.254 INFO:teuthology.orchestra.run.smithi071.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.2 MB 00:00 2024-09-11T06:59:56.510 INFO:teuthology.orchestra.run.smithi196.stdout:CentOS Stream 9 - BaseOS 9.4 MB/s | 8.2 MB 00:00 2024-09-11T06:59:58.106 INFO:teuthology.orchestra.run.smithi071.stdout:CentOS Stream 9 - AppStream 28 MB/s | 20 MB 00:00 2024-09-11T06:59:58.536 INFO:teuthology.orchestra.run.smithi196.stdout:CentOS Stream 9 - AppStream 22 MB/s | 20 MB 00:00 2024-09-11T07:00:02.970 INFO:teuthology.orchestra.run.smithi196.stdout:CentOS Stream 9 - CRB 10 MB/s | 6.5 MB 00:00 2024-09-11T07:00:04.420 INFO:teuthology.orchestra.run.smithi196.stdout:CentOS Stream 9 - Extras packages 73 kB/s | 19 kB 00:00 2024-09-11T07:00:05.183 INFO:teuthology.orchestra.run.smithi196.stdout:Extra Packages for Enterprise Linux 35 MB/s | 23 MB 00:00 2024-09-11T07:00:05.481 INFO:teuthology.orchestra.run.smithi071.stdout:CentOS Stream 9 - CRB 1.8 MB/s | 6.5 MB 00:03 2024-09-11T07:00:07.215 INFO:teuthology.orchestra.run.smithi071.stdout:CentOS Stream 9 - Extras packages 35 kB/s | 19 kB 00:00 2024-09-11T07:00:08.419 INFO:teuthology.orchestra.run.smithi071.stdout:Extra Packages for Enterprise Linux 21 MB/s | 23 MB 00:01 2024-09-11T07:00:10.530 INFO:teuthology.orchestra.run.smithi196.stdout:lab-extras 25 kB/s | 1.7 kB 00:00 2024-09-11T07:00:12.511 INFO:teuthology.orchestra.run.smithi196.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-11T07:00:12.512 INFO:teuthology.orchestra.run.smithi196.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-11T07:00:12.601 INFO:teuthology.orchestra.run.smithi196.stdout:Dependencies resolved. 2024-09-11T07:00:12.607 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-09-11T07:00:12.607 INFO:teuthology.orchestra.run.smithi196.stdout: Package Architecture Version Repository Size 2024-09-11T07:00:12.607 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-09-11T07:00:12.607 INFO:teuthology.orchestra.run.smithi196.stdout:Installing: 2024-09-11T07:00:12.607 INFO:teuthology.orchestra.run.smithi196.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-11T07:00:12.607 INFO:teuthology.orchestra.run.smithi196.stdout:Installing dependencies: 2024-09-11T07:00:12.607 INFO:teuthology.orchestra.run.smithi196.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-11T07:00:12.607 INFO:teuthology.orchestra.run.smithi196.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-11T07:00:12.607 INFO:teuthology.orchestra.run.smithi196.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-11T07:00:12.607 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-11T07:00:12.607 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction Summary 2024-09-11T07:00:12.607 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-09-11T07:00:12.608 INFO:teuthology.orchestra.run.smithi196.stdout:Install 4 Packages 2024-09-11T07:00:12.608 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-11T07:00:12.608 INFO:teuthology.orchestra.run.smithi196.stdout:Total download size: 79 M 2024-09-11T07:00:12.608 INFO:teuthology.orchestra.run.smithi196.stdout:Installed size: 126 M 2024-09-11T07:00:12.608 INFO:teuthology.orchestra.run.smithi196.stdout:Downloading Packages: 2024-09-11T07:00:12.931 INFO:teuthology.orchestra.run.smithi196.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 76 kB/s | 23 kB 00:00 2024-09-11T07:00:13.825 INFO:teuthology.orchestra.run.smithi071.stdout:lab-extras 21 kB/s | 1.7 kB 00:00 2024-09-11T07:00:15.865 INFO:teuthology.orchestra.run.smithi071.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-11T07:00:15.865 INFO:teuthology.orchestra.run.smithi071.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-11T07:00:15.962 INFO:teuthology.orchestra.run.smithi071.stdout:Dependencies resolved. 2024-09-11T07:00:15.967 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-09-11T07:00:15.967 INFO:teuthology.orchestra.run.smithi071.stdout: Package Architecture Version Repository Size 2024-09-11T07:00:15.967 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-09-11T07:00:15.967 INFO:teuthology.orchestra.run.smithi071.stdout:Installing: 2024-09-11T07:00:15.967 INFO:teuthology.orchestra.run.smithi071.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-11T07:00:15.968 INFO:teuthology.orchestra.run.smithi071.stdout:Installing dependencies: 2024-09-11T07:00:15.968 INFO:teuthology.orchestra.run.smithi071.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-11T07:00:15.968 INFO:teuthology.orchestra.run.smithi071.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-11T07:00:15.968 INFO:teuthology.orchestra.run.smithi071.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-11T07:00:15.968 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T07:00:15.968 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction Summary 2024-09-11T07:00:15.968 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-09-11T07:00:15.968 INFO:teuthology.orchestra.run.smithi071.stdout:Install 4 Packages 2024-09-11T07:00:15.968 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T07:00:15.968 INFO:teuthology.orchestra.run.smithi071.stdout:Total download size: 79 M 2024-09-11T07:00:15.968 INFO:teuthology.orchestra.run.smithi071.stdout:Installed size: 126 M 2024-09-11T07:00:15.968 INFO:teuthology.orchestra.run.smithi071.stdout:Downloading Packages: 2024-09-11T07:00:16.593 INFO:teuthology.orchestra.run.smithi071.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 41 kB/s | 23 kB 00:00 2024-09-11T07:00:16.747 INFO:teuthology.orchestra.run.smithi196.stdout:(2/4): kernel-modules-core-5.14.0-505.el9.x86_6 7.5 MB/s | 29 MB 00:03 2024-09-11T07:00:17.081 INFO:teuthology.orchestra.run.smithi196.stdout:(3/4): kernel-core-5.14.0-505.el9.x86_64.rpm 3.5 MB/s | 16 MB 00:04 2024-09-11T07:00:17.435 INFO:teuthology.orchestra.run.smithi071.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 11 MB/s | 16 MB 00:01 2024-09-11T07:00:17.940 INFO:teuthology.orchestra.run.smithi196.stdout:(4/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 6.5 MB/s | 35 MB 00:05 2024-09-11T07:00:17.940 INFO:teuthology.orchestra.run.smithi196.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:00:17.940 INFO:teuthology.orchestra.run.smithi196.stdout:Total 15 MB/s | 79 MB 00:05 2024-09-11T07:00:18.127 INFO:teuthology.orchestra.run.smithi071.stdout:(3/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 16 MB/s | 35 MB 00:02 2024-09-11T07:00:18.620 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction check 2024-09-11T07:00:18.818 INFO:teuthology.orchestra.run.smithi071.stdout:(4/4): kernel-modules-core-5.14.0-505.el9.x86_6 13 MB/s | 29 MB 00:02 2024-09-11T07:00:18.819 INFO:teuthology.orchestra.run.smithi071.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:00:18.819 INFO:teuthology.orchestra.run.smithi071.stdout:Total 28 MB/s | 79 MB 00:02 2024-09-11T07:00:18.957 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction check succeeded. 2024-09-11T07:00:18.957 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction test 2024-09-11T07:00:19.500 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction check 2024-09-11T07:00:19.870 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction check succeeded. 2024-09-11T07:00:19.871 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction test 2024-09-11T07:00:19.963 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction test succeeded. 2024-09-11T07:00:19.963 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction 2024-09-11T07:00:20.812 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction test succeeded. 2024-09-11T07:00:20.813 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction 2024-09-11T07:00:22.421 INFO:teuthology.orchestra.run.smithi196.stdout: Preparing : 1/1 2024-09-11T07:00:23.140 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-11T07:00:23.225 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-11T07:00:23.262 INFO:teuthology.orchestra.run.smithi071.stdout: Preparing : 1/1 2024-09-11T07:00:23.970 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-11T07:00:24.054 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-11T07:00:26.009 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-11T07:00:26.134 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-11T07:00:26.864 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-11T07:00:27.017 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-11T07:00:30.976 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-11T07:00:31.318 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-11T07:00:31.838 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-11T07:00:31.958 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-11T07:00:36.178 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-11T07:00:36.802 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-11T07:01:14.410 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-11T07:01:14.450 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-11T07:01:14.817 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-11T07:01:14.817 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-11T07:01:14.818 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-11T07:01:14.818 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-11T07:01:15.174 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-11T07:01:15.223 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-11T07:01:15.614 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-11T07:01:15.614 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-11T07:01:15.614 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-11T07:01:15.614 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-11T07:01:15.620 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-11T07:01:15.620 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-11T07:01:15.620 INFO:teuthology.orchestra.run.smithi196.stdout:Installed: 2024-09-11T07:01:15.620 INFO:teuthology.orchestra.run.smithi196.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-11T07:01:15.620 INFO:teuthology.orchestra.run.smithi196.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-11T07:01:15.620 INFO:teuthology.orchestra.run.smithi196.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-11T07:01:15.620 INFO:teuthology.orchestra.run.smithi196.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-11T07:01:15.620 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-11T07:01:15.620 INFO:teuthology.orchestra.run.smithi196.stdout:Complete! 2024-09-11T07:01:15.986 DEBUG:teuthology.orchestra.run.smithi196:> rpm -q kernel | sort -rV | head -n 1 2024-09-11T07:01:16.079 INFO:teuthology.orchestra.run.smithi196.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-11T07:01:16.079 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-11T07:01:16.079 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-11T07:01:16.079 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-11T07:01:16.079 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-11T07:01:16.080 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-09-11T07:01:16.080 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi196.front.sepia.ceph.com, path=None, version=distro) 2024-09-11T07:01:16.080 DEBUG:teuthology.orchestra.run.smithi196:> rpm -q kernel | sort -rV | head -n 1 2024-09-11T07:01:16.137 INFO:teuthology.orchestra.run.smithi196.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-11T07:01:16.138 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-11T07:01:16.138 DEBUG:teuthology.orchestra.run.smithi196:> sudo rpm -qi grub2-tools 2024-09-11T07:01:16.180 INFO:teuthology.orchestra.run.smithi196.stdout:Name : grub2-tools 2024-09-11T07:01:16.180 INFO:teuthology.orchestra.run.smithi196.stdout:Epoch : 1 2024-09-11T07:01:16.180 INFO:teuthology.orchestra.run.smithi196.stdout:Version : 2.06 2024-09-11T07:01:16.180 INFO:teuthology.orchestra.run.smithi196.stdout:Release : 82.el9 2024-09-11T07:01:16.181 INFO:teuthology.orchestra.run.smithi196.stdout:Architecture: x86_64 2024-09-11T07:01:16.181 INFO:teuthology.orchestra.run.smithi196.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-11T07:01:16.181 INFO:teuthology.orchestra.run.smithi196.stdout:Group : Unspecified 2024-09-11T07:01:16.181 INFO:teuthology.orchestra.run.smithi196.stdout:Size : 8274670 2024-09-11T07:01:16.181 INFO:teuthology.orchestra.run.smithi196.stdout:License : GPLv3+ 2024-09-11T07:01:16.181 INFO:teuthology.orchestra.run.smithi196.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-11T07:01:16.181 INFO:teuthology.orchestra.run.smithi196.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-11T07:01:16.181 INFO:teuthology.orchestra.run.smithi196.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-11T07:01:16.181 INFO:teuthology.orchestra.run.smithi196.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-11T07:01:16.181 INFO:teuthology.orchestra.run.smithi196.stdout:Packager : builder@centos.org 2024-09-11T07:01:16.181 INFO:teuthology.orchestra.run.smithi196.stdout:Vendor : CentOS 2024-09-11T07:01:16.182 INFO:teuthology.orchestra.run.smithi196.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-11T07:01:16.182 INFO:teuthology.orchestra.run.smithi196.stdout:Summary : Support tools for GRUB. 2024-09-11T07:01:16.182 INFO:teuthology.orchestra.run.smithi196.stdout:Description : 2024-09-11T07:01:16.182 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-11T07:01:16.182 INFO:teuthology.orchestra.run.smithi196.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-11T07:01:16.182 INFO:teuthology.orchestra.run.smithi196.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-11T07:01:16.182 INFO:teuthology.orchestra.run.smithi196.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-11T07:01:16.182 INFO:teuthology.orchestra.run.smithi196.stdout:hardware devices. 2024-09-11T07:01:16.182 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-11T07:01:16.182 INFO:teuthology.orchestra.run.smithi196.stdout:This subpackage provides tools for support of all platforms. 2024-09-11T07:01:16.183 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-11T07:01:16.183 INFO:teuthology.task.kernel:Updating grub on smithi196 to boot 5.14.0-505.el9.x86_64 2024-09-11T07:01:16.183 DEBUG:teuthology.orchestra.run.smithi196:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-11T07:01:16.475 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-11T07:01:16.475 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T07:01:16.476 INFO:teuthology.orchestra.run.smithi071.stdout:Installed: 2024-09-11T07:01:16.476 INFO:teuthology.orchestra.run.smithi071.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-11T07:01:16.476 INFO:teuthology.orchestra.run.smithi071.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-11T07:01:16.476 INFO:teuthology.orchestra.run.smithi071.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-11T07:01:16.476 INFO:teuthology.orchestra.run.smithi071.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-11T07:01:16.476 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T07:01:16.476 INFO:teuthology.orchestra.run.smithi071.stdout:Complete! 2024-09-11T07:01:16.850 DEBUG:teuthology.orchestra.run.smithi071:> rpm -q kernel | sort -rV | head -n 1 2024-09-11T07:01:16.861 INFO:teuthology.orchestra.run.smithi196.stderr:Generating grub configuration file ... 2024-09-11T07:01:16.938 INFO:teuthology.orchestra.run.smithi071.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-11T07:01:16.938 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-11T07:01:16.938 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-11T07:01:16.938 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-11T07:01:16.938 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-11T07:01:16.938 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-11T07:01:16.938 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi071.front.sepia.ceph.com, path=None, version=distro) 2024-09-11T07:01:16.938 DEBUG:teuthology.orchestra.run.smithi071:> rpm -q kernel | sort -rV | head -n 1 2024-09-11T07:01:16.997 INFO:teuthology.orchestra.run.smithi071.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-11T07:01:16.997 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-11T07:01:16.997 DEBUG:teuthology.orchestra.run.smithi071:> sudo rpm -qi grub2-tools 2024-09-11T07:01:17.039 INFO:teuthology.orchestra.run.smithi071.stdout:Name : grub2-tools 2024-09-11T07:01:17.039 INFO:teuthology.orchestra.run.smithi071.stdout:Epoch : 1 2024-09-11T07:01:17.039 INFO:teuthology.orchestra.run.smithi071.stdout:Version : 2.06 2024-09-11T07:01:17.039 INFO:teuthology.orchestra.run.smithi071.stdout:Release : 82.el9 2024-09-11T07:01:17.039 INFO:teuthology.orchestra.run.smithi071.stdout:Architecture: x86_64 2024-09-11T07:01:17.039 INFO:teuthology.orchestra.run.smithi071.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-11T07:01:17.039 INFO:teuthology.orchestra.run.smithi071.stdout:Group : Unspecified 2024-09-11T07:01:17.039 INFO:teuthology.orchestra.run.smithi071.stdout:Size : 8274670 2024-09-11T07:01:17.039 INFO:teuthology.orchestra.run.smithi071.stdout:License : GPLv3+ 2024-09-11T07:01:17.039 INFO:teuthology.orchestra.run.smithi071.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-11T07:01:17.040 INFO:teuthology.orchestra.run.smithi071.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-11T07:01:17.040 INFO:teuthology.orchestra.run.smithi071.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-11T07:01:17.040 INFO:teuthology.orchestra.run.smithi071.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-11T07:01:17.040 INFO:teuthology.orchestra.run.smithi071.stdout:Packager : builder@centos.org 2024-09-11T07:01:17.040 INFO:teuthology.orchestra.run.smithi071.stdout:Vendor : CentOS 2024-09-11T07:01:17.040 INFO:teuthology.orchestra.run.smithi071.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-11T07:01:17.040 INFO:teuthology.orchestra.run.smithi071.stdout:Summary : Support tools for GRUB. 2024-09-11T07:01:17.040 INFO:teuthology.orchestra.run.smithi071.stdout:Description : 2024-09-11T07:01:17.040 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T07:01:17.040 INFO:teuthology.orchestra.run.smithi071.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-11T07:01:17.040 INFO:teuthology.orchestra.run.smithi071.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-11T07:01:17.040 INFO:teuthology.orchestra.run.smithi071.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-11T07:01:17.041 INFO:teuthology.orchestra.run.smithi071.stdout:hardware devices. 2024-09-11T07:01:17.041 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T07:01:17.041 INFO:teuthology.orchestra.run.smithi071.stdout:This subpackage provides tools for support of all platforms. 2024-09-11T07:01:17.041 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-11T07:01:17.042 INFO:teuthology.task.kernel:Updating grub on smithi071 to boot 5.14.0-505.el9.x86_64 2024-09-11T07:01:17.042 DEBUG:teuthology.orchestra.run.smithi071:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-11T07:01:17.703 INFO:teuthology.orchestra.run.smithi071.stderr:Generating grub configuration file ... 2024-09-11T07:01:18.166 INFO:teuthology.orchestra.run.smithi196.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-11T07:01:18.202 INFO:teuthology.orchestra.run.smithi196.stderr:done 2024-09-11T07:01:18.205 DEBUG:teuthology.orchestra.run.smithi196:> mktemp 2024-09-11T07:01:18.220 INFO:teuthology.orchestra.run.smithi196.stdout:/tmp/tmp.sPA6pG7E1B 2024-09-11T07:01:18.220 DEBUG:teuthology.orchestra.run.smithi196:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.sPA6pG7E1B 2024-09-11T07:01:18.286 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod 0666 /tmp/tmp.sPA6pG7E1B 2024-09-11T07:01:18.422 DEBUG:teuthology.orchestra.remote:smithi196:/tmp/tmp.sPA6pG7E1B is 6KB 2024-09-11T07:01:18.471 DEBUG:teuthology.orchestra.run.smithi196:> rm -fr /tmp/tmp.sPA6pG7E1B 2024-09-11T07:01:18.486 DEBUG:teuthology.orchestra.run.smithi196:> sudo /bin/ls /boot/loader/entries || true 2024-09-11T07:01:18.556 INFO:teuthology.orchestra.run.smithi196.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-11T07:01:18.556 INFO:teuthology.orchestra.run.smithi196.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-11T07:01:18.556 INFO:teuthology.orchestra.run.smithi196.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-11T07:01:18.556 INFO:teuthology.orchestra.run.smithi196.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-11T07:01:18.557 DEBUG:teuthology.orchestra.run.smithi196:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-11T07:01:18.725 DEBUG:teuthology.orchestra.run.smithi196:> sudo shutdown -r now 2024-09-11T07:01:19.047 INFO:teuthology.orchestra.run.smithi071.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-11T07:01:19.084 INFO:teuthology.orchestra.run.smithi071.stderr:done 2024-09-11T07:01:19.086 DEBUG:teuthology.orchestra.run.smithi071:> mktemp 2024-09-11T07:01:19.102 INFO:teuthology.orchestra.run.smithi071.stdout:/tmp/tmp.gMjcl1xfZU 2024-09-11T07:01:19.102 DEBUG:teuthology.orchestra.run.smithi071:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.gMjcl1xfZU 2024-09-11T07:01:19.171 DEBUG:teuthology.orchestra.run.smithi071:> sudo chmod 0666 /tmp/tmp.gMjcl1xfZU 2024-09-11T07:01:19.310 DEBUG:teuthology.orchestra.remote:smithi071:/tmp/tmp.gMjcl1xfZU is 6KB 2024-09-11T07:01:19.359 DEBUG:teuthology.orchestra.run.smithi071:> rm -fr /tmp/tmp.gMjcl1xfZU 2024-09-11T07:01:19.374 DEBUG:teuthology.orchestra.run.smithi071:> sudo /bin/ls /boot/loader/entries || true 2024-09-11T07:01:19.437 INFO:teuthology.orchestra.run.smithi071.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-11T07:01:19.437 INFO:teuthology.orchestra.run.smithi071.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-11T07:01:19.437 INFO:teuthology.orchestra.run.smithi071.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-11T07:01:19.437 INFO:teuthology.orchestra.run.smithi071.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-11T07:01:19.438 DEBUG:teuthology.orchestra.run.smithi071:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-11T07:01:19.597 DEBUG:teuthology.orchestra.run.smithi071:> sudo shutdown -r now 2024-09-11T07:01:48.740 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-11T07:01:48.741 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi196.front.sepia.ceph.com' 2024-09-11T07:01:48.742 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T07:01:49.600 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-11T07:01:49.601 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi071.front.sepia.ceph.com' 2024-09-11T07:01:49.601 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi071.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T07:02:07.165 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.196 2024-09-11T07:02:07.933 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.71 2024-09-11T07:02:16.174 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi196.front.sepia.ceph.com' 2024-09-11T07:02:16.174 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T07:02:16.935 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi071.front.sepia.ceph.com' 2024-09-11T07:02:16.935 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi071.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T07:02:19.229 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.196 2024-09-11T07:02:19.997 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.71 2024-09-11T07:02:31.238 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi196.front.sepia.ceph.com' 2024-09-11T07:02:31.239 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T07:02:31.998 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi071.front.sepia.ceph.com' 2024-09-11T07:02:31.999 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi071.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T07:02:49.661 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.196 2024-09-11T07:02:50.429 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.71 2024-09-11T07:03:04.673 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi196.front.sepia.ceph.com' 2024-09-11T07:03:04.674 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T07:03:05.191 DEBUG:teuthology.orchestra.run.smithi196:> true 2024-09-11T07:03:05.430 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi071.front.sepia.ceph.com' 2024-09-11T07:03:05.430 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi071.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T07:03:05.624 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi196.front.sepia.ceph.com' 2024-09-11T07:03:05.624 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-505.el9.x86_64"... 2024-09-11T07:03:05.625 DEBUG:teuthology.orchestra.run.smithi196:> uname -r 2024-09-11T07:03:05.639 INFO:teuthology.orchestra.run.smithi196.stdout:5.14.0-505.el9.x86_64 2024-09-11T07:03:05.639 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-11T07:03:05.639 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-11T07:03:05.639 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-11T07:03:05.756 DEBUG:teuthology.orchestra.run.smithi071:> true 2024-09-11T07:03:06.122 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi071.front.sepia.ceph.com' 2024-09-11T07:03:06.123 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-505.el9.x86_64"... 2024-09-11T07:03:06.123 DEBUG:teuthology.orchestra.run.smithi071:> uname -r 2024-09-11T07:03:06.137 INFO:teuthology.orchestra.run.smithi071.stdout:5.14.0-505.el9.x86_64 2024-09-11T07:03:06.137 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-11T07:03:06.138 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-11T07:03:06.138 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-11T07:03:06.640 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-11T07:03:06.641 DEBUG:teuthology.orchestra.run.smithi196:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-11T07:03:06.768 INFO:teuthology.orchestra.run.smithi196.stdout:ttyS1 2024-09-11T07:03:06.789 DEBUG:teuthology.parallel:result is None 2024-09-11T07:03:07.139 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-11T07:03:07.139 DEBUG:teuthology.orchestra.run.smithi071:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-11T07:03:07.233 INFO:teuthology.orchestra.run.smithi071.stdout:ttyS1 2024-09-11T07:03:07.258 DEBUG:teuthology.parallel:result is None 2024-09-11T07:03:07.258 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-11T07:03:07.267 INFO:teuthology.task.internal:Creating test directory... 2024-09-11T07:03:07.267 DEBUG:teuthology.orchestra.run.smithi071:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-11T07:03:07.269 DEBUG:teuthology.orchestra.run.smithi196:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-11T07:03:07.284 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-11T07:03:07.317 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-11T07:03:07.353 INFO:teuthology.task.internal:Creating archive directory... 2024-09-11T07:03:07.353 DEBUG:teuthology.orchestra.run.smithi071:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-11T07:03:07.355 DEBUG:teuthology.orchestra.run.smithi196:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-11T07:03:07.457 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-11T07:03:07.467 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-11T07:03:07.468 DEBUG:teuthology.orchestra.run.smithi071:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-11T07:03:07.470 DEBUG:teuthology.orchestra.run.smithi196:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-11T07:03:07.492 INFO:teuthology.orchestra.run.smithi071.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-11T07:03:07.504 INFO:teuthology.orchestra.run.smithi071.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-11T07:03:07.649 INFO:teuthology.orchestra.run.smithi196.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-11T07:03:07.660 INFO:teuthology.orchestra.run.smithi196.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-11T07:03:07.662 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-11T07:03:07.673 INFO:teuthology.task.internal:Configuring sudo... 2024-09-11T07:03:07.673 DEBUG:teuthology.orchestra.run.smithi071:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-11T07:03:07.675 DEBUG:teuthology.orchestra.run.smithi196:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-11T07:03:07.701 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-11T07:03:07.714 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-11T07:03:07.714 DEBUG:teuthology.orchestra.run.smithi071:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-11T07:03:07.742 DEBUG:teuthology.orchestra.run.smithi196:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-11T07:03:07.757 DEBUG:teuthology.orchestra.run.smithi071:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-11T07:03:07.840 DEBUG:teuthology.orchestra.run.smithi071:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-11T07:03:07.893 DEBUG:teuthology.orchestra.run.smithi071:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-11T07:03:07.976 DEBUG:teuthology.orchestra.run.smithi071:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-11T07:03:08.013 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-11T07:03:08.013 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-11T07:03:08.078 DEBUG:teuthology.orchestra.run.smithi196:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-11T07:03:08.152 DEBUG:teuthology.orchestra.run.smithi196:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-11T07:03:08.219 DEBUG:teuthology.orchestra.run.smithi196:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-11T07:03:08.287 DEBUG:teuthology.orchestra.run.smithi196:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-11T07:03:08.359 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-11T07:03:08.360 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-11T07:03:08.420 DEBUG:teuthology.orchestra.run.smithi071:> sudo service rsyslog restart 2024-09-11T07:03:08.422 DEBUG:teuthology.orchestra.run.smithi196:> sudo service rsyslog restart 2024-09-11T07:03:08.472 INFO:teuthology.orchestra.run.smithi071.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-11T07:03:08.536 INFO:teuthology.orchestra.run.smithi196.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-11T07:03:08.991 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-11T07:03:09.000 INFO:teuthology.task.internal:Starting timer... 2024-09-11T07:03:09.000 INFO:teuthology.run_tasks:Running task pcp... 2024-09-11T07:03:09.031 INFO:teuthology.run_tasks:Running task selinux... 2024-09-11T07:03:09.063 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-09-11T07:03:09.063 DEBUG:teuthology.orchestra.run.smithi071:> sudo service auditd rotate 2024-09-11T07:03:09.169 INFO:teuthology.orchestra.run.smithi071.stdout:Rotating logs: 2024-09-11T07:03:09.171 DEBUG:teuthology.orchestra.run.smithi196:> sudo service auditd rotate 2024-09-11T07:03:09.437 INFO:teuthology.orchestra.run.smithi196.stdout:Rotating logs: 2024-09-11T07:03:09.440 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-11T07:03:09.440 DEBUG:teuthology.orchestra.run.smithi071:> /usr/sbin/getenforce 2024-09-11T07:03:09.492 INFO:teuthology.orchestra.run.smithi071.stdout:Permissive 2024-09-11T07:03:09.492 DEBUG:teuthology.orchestra.run.smithi196:> /usr/sbin/getenforce 2024-09-11T07:03:09.570 INFO:teuthology.orchestra.run.smithi196.stdout:Permissive 2024-09-11T07:03:09.570 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi071.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi196.front.sepia.ceph.com': 'permissive'} 2024-09-11T07:03:09.571 DEBUG:teuthology.orchestra.run.smithi071:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-11T07:03:09.597 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:03:09.597 DEBUG:teuthology.orchestra.run.smithi196:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-11T07:03:09.626 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:03:09.626 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-11T07:03:09.627 DEBUG:teuthology.orchestra.run.smithi071:> sudo /usr/sbin/setenforce permissive 2024-09-11T07:03:09.670 DEBUG:teuthology.orchestra.run.smithi196:> sudo /usr/sbin/setenforce permissive 2024-09-11T07:03:09.711 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-11T07:03:09.721 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-11T07:03:09.733 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-11T07:03:09.734 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi071.front.sepia.ceph.com,smithi196.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-11T07:06:50.037 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi071.front.sepia.ceph.com'), Remote(name='ubuntu@smithi196.front.sepia.ceph.com')] 2024-09-11T07:06:50.039 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi071.front.sepia.ceph.com' 2024-09-11T07:06:50.040 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi071.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T07:06:50.157 DEBUG:teuthology.orchestra.run.smithi071:> true 2024-09-11T07:06:50.236 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi071.front.sepia.ceph.com' 2024-09-11T07:06:50.237 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi196.front.sepia.ceph.com' 2024-09-11T07:06:50.237 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T07:06:50.309 DEBUG:teuthology.orchestra.run.smithi196:> true 2024-09-11T07:06:50.384 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi196.front.sepia.ceph.com' 2024-09-11T07:06:50.384 INFO:teuthology.run_tasks:Running task clock... 2024-09-11T07:06:50.397 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-11T07:06:50.397 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-11T07:06:50.398 DEBUG:teuthology.orchestra.run.smithi071:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-11T07:06:50.400 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-11T07:06:50.401 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-11T07:06:50.428 INFO:teuthology.orchestra.run.smithi071.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-11T07:06:50.442 INFO:teuthology.orchestra.run.smithi071.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-11T07:06:50.454 INFO:teuthology.orchestra.run.smithi196.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-11T07:06:50.467 INFO:teuthology.orchestra.run.smithi071.stderr:sudo: ntpd: command not found 2024-09-11T07:06:50.469 INFO:teuthology.orchestra.run.smithi196.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-11T07:06:50.478 INFO:teuthology.orchestra.run.smithi071.stdout:506 Cannot talk to daemon 2024-09-11T07:06:50.492 INFO:teuthology.orchestra.run.smithi071.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-11T07:06:50.493 INFO:teuthology.orchestra.run.smithi196.stderr:sudo: ntpd: command not found 2024-09-11T07:06:50.504 INFO:teuthology.orchestra.run.smithi196.stdout:506 Cannot talk to daemon 2024-09-11T07:06:50.506 INFO:teuthology.orchestra.run.smithi071.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-11T07:06:50.518 INFO:teuthology.orchestra.run.smithi196.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-11T07:06:50.533 INFO:teuthology.orchestra.run.smithi196.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-11T07:06:50.559 INFO:teuthology.orchestra.run.smithi071.stderr:bash: line 1: ntpq: command not found 2024-09-11T07:06:50.562 INFO:teuthology.orchestra.run.smithi071.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-11T07:06:50.563 INFO:teuthology.orchestra.run.smithi071.stdout:=============================================================================== 2024-09-11T07:06:50.563 INFO:teuthology.orchestra.run.smithi071.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T07:06:50.563 INFO:teuthology.orchestra.run.smithi071.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T07:06:50.563 INFO:teuthology.orchestra.run.smithi071.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T07:06:50.563 INFO:teuthology.orchestra.run.smithi071.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T07:06:50.586 INFO:teuthology.orchestra.run.smithi196.stderr:bash: line 1: ntpq: command not found 2024-09-11T07:06:50.589 INFO:teuthology.orchestra.run.smithi196.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-11T07:06:50.590 INFO:teuthology.orchestra.run.smithi196.stdout:=============================================================================== 2024-09-11T07:06:50.590 INFO:teuthology.orchestra.run.smithi196.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T07:06:50.590 INFO:teuthology.orchestra.run.smithi196.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T07:06:50.590 INFO:teuthology.orchestra.run.smithi196.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T07:06:50.590 INFO:teuthology.orchestra.run.smithi196.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T07:06:50.590 INFO:teuthology.run_tasks:Running task pexec... 2024-09-11T07:06:50.599 INFO:teuthology.task.pexec:Executing custom commands... 2024-09-11T07:06:50.600 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi071.front.sepia.ceph.com 2024-09-11T07:06:50.600 DEBUG:teuthology.orchestra.run.smithi071:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-09-11T07:06:50.600 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi196.front.sepia.ceph.com 2024-09-11T07:06:50.600 DEBUG:teuthology.orchestra.run.smithi196:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-09-11T07:06:50.952 INFO:teuthology.orchestra.run.smithi071.stdout:Dependencies resolved. 2024-09-11T07:06:50.964 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-09-11T07:06:50.964 INFO:teuthology.orchestra.run.smithi071.stdout: Package Architecture Version Repository Size 2024-09-11T07:06:50.964 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-09-11T07:06:50.965 INFO:teuthology.orchestra.run.smithi071.stdout:Removing: 2024-09-11T07:06:50.965 INFO:teuthology.orchestra.run.smithi071.stdout: nvme-cli x86_64 2.9.1-6.el9 @baseos 5.3 M 2024-09-11T07:06:50.965 INFO:teuthology.orchestra.run.smithi071.stdout:Removing unused dependencies: 2024-09-11T07:06:50.965 INFO:teuthology.orchestra.run.smithi071.stdout: libnvme x86_64 1.9-3.el9 @baseos 273 k 2024-09-11T07:06:50.965 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T07:06:50.965 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction Summary 2024-09-11T07:06:50.965 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-09-11T07:06:50.965 INFO:teuthology.orchestra.run.smithi071.stdout:Remove 2 Packages 2024-09-11T07:06:50.965 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T07:06:50.965 INFO:teuthology.orchestra.run.smithi071.stdout:Freed space: 5.5 M 2024-09-11T07:06:50.965 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction check 2024-09-11T07:06:50.969 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction check succeeded. 2024-09-11T07:06:50.969 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction test 2024-09-11T07:06:50.982 INFO:teuthology.orchestra.run.smithi196.stdout:Dependencies resolved. 2024-09-11T07:06:50.998 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-09-11T07:06:50.999 INFO:teuthology.orchestra.run.smithi196.stdout: Package Architecture Version Repository Size 2024-09-11T07:06:51.000 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-09-11T07:06:51.000 INFO:teuthology.orchestra.run.smithi196.stdout:Removing: 2024-09-11T07:06:51.000 INFO:teuthology.orchestra.run.smithi196.stdout: nvme-cli x86_64 2.9.1-6.el9 @baseos 5.3 M 2024-09-11T07:06:51.000 INFO:teuthology.orchestra.run.smithi196.stdout:Removing unused dependencies: 2024-09-11T07:06:51.000 INFO:teuthology.orchestra.run.smithi196.stdout: libnvme x86_64 1.9-3.el9 @baseos 273 k 2024-09-11T07:06:51.000 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-11T07:06:51.000 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction Summary 2024-09-11T07:06:51.000 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-09-11T07:06:51.000 INFO:teuthology.orchestra.run.smithi196.stdout:Remove 2 Packages 2024-09-11T07:06:51.000 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-11T07:06:51.000 INFO:teuthology.orchestra.run.smithi196.stdout:Freed space: 5.5 M 2024-09-11T07:06:51.000 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction check 2024-09-11T07:06:51.003 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction check succeeded. 2024-09-11T07:06:51.003 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction test 2024-09-11T07:06:51.030 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction test succeeded. 2024-09-11T07:06:51.030 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction 2024-09-11T07:06:51.064 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction test succeeded. 2024-09-11T07:06:51.064 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction 2024-09-11T07:06:51.174 INFO:teuthology.orchestra.run.smithi071.stdout: Preparing : 1/1 2024-09-11T07:06:51.207 INFO:teuthology.orchestra.run.smithi196.stdout: Preparing : 1/1 2024-09-11T07:06:51.259 INFO:teuthology.orchestra.run.smithi071.stdout: Erasing : nvme-cli-2.9.1-6.el9.x86_64 1/2 2024-09-11T07:06:51.270 INFO:teuthology.orchestra.run.smithi196.stdout: Erasing : nvme-cli-2.9.1-6.el9.x86_64 1/2 2024-09-11T07:06:51.314 INFO:teuthology.orchestra.run.smithi071.stdout: Erasing : libnvme-1.9-3.el9.x86_64 2/2 2024-09-11T07:06:51.326 INFO:teuthology.orchestra.run.smithi196.stdout: Erasing : libnvme-1.9-3.el9.x86_64 2/2 2024-09-11T07:06:51.818 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: libnvme-1.9-3.el9.x86_64 2/2 2024-09-11T07:06:51.818 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/2 2024-09-11T07:06:51.819 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: libnvme-1.9-3.el9.x86_64 2/2 2024-09-11T07:06:51.819 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/2 2024-09-11T07:06:52.069 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/2 2024-09-11T07:06:52.069 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T07:06:52.070 INFO:teuthology.orchestra.run.smithi071.stdout:Removed: 2024-09-11T07:06:52.070 INFO:teuthology.orchestra.run.smithi071.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-11T07:06:52.070 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T07:06:52.070 INFO:teuthology.orchestra.run.smithi071.stdout:Complete! 2024-09-11T07:06:52.078 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/2 2024-09-11T07:06:52.078 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-11T07:06:52.078 INFO:teuthology.orchestra.run.smithi196.stdout:Removed: 2024-09-11T07:06:52.078 INFO:teuthology.orchestra.run.smithi196.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-11T07:06:52.078 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-11T07:06:52.078 INFO:teuthology.orchestra.run.smithi196.stdout:Complete! 2024-09-11T07:06:52.885 INFO:teuthology.orchestra.run.smithi196.stdout:Last metadata expiration check: 0:00:46 ago on Wed 11 Sep 2024 07:06:06 AM UTC. 2024-09-11T07:06:52.917 INFO:teuthology.orchestra.run.smithi071.stdout:Last metadata expiration check: 0:00:47 ago on Wed 11 Sep 2024 07:06:05 AM UTC. 2024-09-11T07:06:53.022 INFO:teuthology.orchestra.run.smithi196.stdout:Dependencies resolved. 2024-09-11T07:06:53.023 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-09-11T07:06:53.023 INFO:teuthology.orchestra.run.smithi196.stdout: Package Architecture Version Repository Size 2024-09-11T07:06:53.023 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-09-11T07:06:53.023 INFO:teuthology.orchestra.run.smithi196.stdout:Installing: 2024-09-11T07:06:53.023 INFO:teuthology.orchestra.run.smithi196.stdout: nvme-cli x86_64 2.9.1-6.el9 baseos 906 k 2024-09-11T07:06:53.023 INFO:teuthology.orchestra.run.smithi196.stdout: nvmetcli noarch 0.7-3.el9 baseos 44 k 2024-09-11T07:06:53.023 INFO:teuthology.orchestra.run.smithi196.stdout:Installing dependencies: 2024-09-11T07:06:53.023 INFO:teuthology.orchestra.run.smithi196.stdout: libnvme x86_64 1.9-3.el9 baseos 106 k 2024-09-11T07:06:53.023 INFO:teuthology.orchestra.run.smithi196.stdout: python3-configshell noarch 1:1.1.30-1.el9 baseos 72 k 2024-09-11T07:06:53.024 INFO:teuthology.orchestra.run.smithi196.stdout: python3-kmod x86_64 0.9-32.el9 baseos 84 k 2024-09-11T07:06:53.024 INFO:teuthology.orchestra.run.smithi196.stdout: python3-urwid x86_64 2.1.2-4.el9 baseos 837 k 2024-09-11T07:06:53.024 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-11T07:06:53.024 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction Summary 2024-09-11T07:06:53.024 INFO:teuthology.orchestra.run.smithi196.stdout:================================================================================ 2024-09-11T07:06:53.024 INFO:teuthology.orchestra.run.smithi196.stdout:Install 6 Packages 2024-09-11T07:06:53.024 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-11T07:06:53.027 INFO:teuthology.orchestra.run.smithi196.stdout:Total download size: 2.0 M 2024-09-11T07:06:53.028 INFO:teuthology.orchestra.run.smithi196.stdout:Installed size: 9.2 M 2024-09-11T07:06:53.028 INFO:teuthology.orchestra.run.smithi196.stdout:Downloading Packages: 2024-09-11T07:06:53.053 INFO:teuthology.orchestra.run.smithi071.stdout:Dependencies resolved. 2024-09-11T07:06:53.054 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-09-11T07:06:53.054 INFO:teuthology.orchestra.run.smithi071.stdout: Package Architecture Version Repository Size 2024-09-11T07:06:53.054 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-09-11T07:06:53.055 INFO:teuthology.orchestra.run.smithi071.stdout:Installing: 2024-09-11T07:06:53.055 INFO:teuthology.orchestra.run.smithi071.stdout: nvme-cli x86_64 2.9.1-6.el9 baseos 906 k 2024-09-11T07:06:53.055 INFO:teuthology.orchestra.run.smithi071.stdout: nvmetcli noarch 0.7-3.el9 baseos 44 k 2024-09-11T07:06:53.055 INFO:teuthology.orchestra.run.smithi071.stdout:Installing dependencies: 2024-09-11T07:06:53.055 INFO:teuthology.orchestra.run.smithi071.stdout: libnvme x86_64 1.9-3.el9 baseos 106 k 2024-09-11T07:06:53.055 INFO:teuthology.orchestra.run.smithi071.stdout: python3-configshell noarch 1:1.1.30-1.el9 baseos 72 k 2024-09-11T07:06:53.055 INFO:teuthology.orchestra.run.smithi071.stdout: python3-kmod x86_64 0.9-32.el9 baseos 84 k 2024-09-11T07:06:53.055 INFO:teuthology.orchestra.run.smithi071.stdout: python3-urwid x86_64 2.1.2-4.el9 baseos 837 k 2024-09-11T07:06:53.055 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T07:06:53.055 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction Summary 2024-09-11T07:06:53.055 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-09-11T07:06:53.055 INFO:teuthology.orchestra.run.smithi071.stdout:Install 6 Packages 2024-09-11T07:06:53.055 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T07:06:53.066 INFO:teuthology.orchestra.run.smithi071.stdout:Total download size: 2.0 M 2024-09-11T07:06:53.066 INFO:teuthology.orchestra.run.smithi071.stdout:Installed size: 9.2 M 2024-09-11T07:06:53.067 INFO:teuthology.orchestra.run.smithi071.stdout:Downloading Packages: 2024-09-11T07:06:53.204 INFO:teuthology.orchestra.run.smithi071.stdout:(1/6): nvmetcli-0.7-3.el9.noarch.rpm 391 kB/s | 44 kB 00:00 2024-09-11T07:06:53.246 INFO:teuthology.orchestra.run.smithi071.stdout:(2/6): nvme-cli-2.9.1-6.el9.x86_64.rpm 5.7 MB/s | 906 kB 00:00 2024-09-11T07:06:53.271 INFO:teuthology.orchestra.run.smithi071.stdout:(3/6): libnvme-1.9-3.el9.x86_64.rpm 586 kB/s | 106 kB 00:00 2024-09-11T07:06:53.296 INFO:teuthology.orchestra.run.smithi071.stdout:(4/6): python3-configshell-1.1.30-1.el9.noarch. 777 kB/s | 72 kB 00:00 2024-09-11T07:06:53.321 INFO:teuthology.orchestra.run.smithi071.stdout:(5/6): python3-kmod-0.9-32.el9.x86_64.rpm 1.1 MB/s | 84 kB 00:00 2024-09-11T07:06:53.388 INFO:teuthology.orchestra.run.smithi071.stdout:(6/6): python3-urwid-2.1.2-4.el9.x86_64.rpm 7.0 MB/s | 837 kB 00:00 2024-09-11T07:06:53.389 INFO:teuthology.orchestra.run.smithi071.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:06:53.389 INFO:teuthology.orchestra.run.smithi071.stdout:Total 6.2 MB/s | 2.0 MB 00:00 2024-09-11T07:06:53.461 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction check 2024-09-11T07:06:53.474 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction check succeeded. 2024-09-11T07:06:53.474 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction test 2024-09-11T07:06:53.674 INFO:teuthology.orchestra.run.smithi196.stdout:(1/6): nvmetcli-0.7-3.el9.noarch.rpm 74 kB/s | 44 kB 00:00 2024-09-11T07:06:53.741 INFO:teuthology.orchestra.run.smithi196.stdout:(2/6): libnvme-1.9-3.el9.x86_64.rpm 160 kB/s | 106 kB 00:00 2024-09-11T07:06:53.769 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction test succeeded. 2024-09-11T07:06:53.769 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction 2024-09-11T07:06:53.975 INFO:teuthology.orchestra.run.smithi196.stdout:(3/6): nvme-cli-2.9.1-6.el9.x86_64.rpm 1.0 MB/s | 906 kB 00:00 2024-09-11T07:06:54.206 INFO:teuthology.orchestra.run.smithi071.stdout: Preparing : 1/1 2024-09-11T07:06:54.299 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-urwid-2.1.2-4.el9.x86_64 1/6 2024-09-11T07:06:54.317 INFO:teuthology.orchestra.run.smithi196.stdout:(4/6): python3-configshell-1.1.30-1.el9.noarch. 112 kB/s | 72 kB 00:00 2024-09-11T07:06:54.336 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-configshell-1:1.1.30-1.el9.noarch 2/6 2024-09-11T07:06:54.364 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-kmod-0.9-32.el9.x86_64 3/6 2024-09-11T07:06:54.408 INFO:teuthology.orchestra.run.smithi196.stdout:(5/6): python3-kmod-0.9-32.el9.x86_64.rpm 126 kB/s | 84 kB 00:00 2024-09-11T07:06:54.572 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libnvme-1.9-3.el9.x86_64 4/6 2024-09-11T07:06:54.612 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : nvme-cli-2.9.1-6.el9.x86_64 5/6 2024-09-11T07:06:54.867 INFO:teuthology.orchestra.run.smithi196.stdout:(6/6): python3-urwid-2.1.2-4.el9.x86_64.rpm 948 kB/s | 837 kB 00:00 2024-09-11T07:06:54.867 INFO:teuthology.orchestra.run.smithi196.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:06:54.868 INFO:teuthology.orchestra.run.smithi196.stdout:Total 1.1 MB/s | 2.0 MB 00:01 2024-09-11T07:06:54.949 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction check 2024-09-11T07:06:54.964 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction check succeeded. 2024-09-11T07:06:54.964 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction test 2024-09-11T07:06:55.058 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: nvme-cli-2.9.1-6.el9.x86_64 5/6 2024-09-11T07:06:55.089 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : nvmetcli-0.7-3.el9.noarch 6/6 2024-09-11T07:06:55.238 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction test succeeded. 2024-09-11T07:06:55.238 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction 2024-09-11T07:06:55.646 INFO:teuthology.orchestra.run.smithi196.stdout: Preparing : 1/1 2024-09-11T07:06:55.736 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-urwid-2.1.2-4.el9.x86_64 1/6 2024-09-11T07:06:55.770 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-configshell-1:1.1.30-1.el9.noarch 2/6 2024-09-11T07:06:55.799 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-kmod-0.9-32.el9.x86_64 3/6 2024-09-11T07:06:56.008 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libnvme-1.9-3.el9.x86_64 4/6 2024-09-11T07:06:56.056 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : nvme-cli-2.9.1-6.el9.x86_64 5/6 2024-09-11T07:06:56.070 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: nvmetcli-0.7-3.el9.noarch 6/6 2024-09-11T07:06:56.070 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/6 2024-09-11T07:06:56.070 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/6 2024-09-11T07:06:56.070 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : nvmetcli-0.7-3.el9.noarch 3/6 2024-09-11T07:06:56.070 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-configshell-1:1.1.30-1.el9.noarch 4/6 2024-09-11T07:06:56.070 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-kmod-0.9-32.el9.x86_64 5/6 2024-09-11T07:06:56.446 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-urwid-2.1.2-4.el9.x86_64 6/6 2024-09-11T07:06:56.447 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T07:06:56.447 INFO:teuthology.orchestra.run.smithi071.stdout:Installed: 2024-09-11T07:06:56.447 INFO:teuthology.orchestra.run.smithi071.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-11T07:06:56.447 INFO:teuthology.orchestra.run.smithi071.stdout: nvmetcli-0.7-3.el9.noarch python3-configshell-1:1.1.30-1.el9.noarch 2024-09-11T07:06:56.447 INFO:teuthology.orchestra.run.smithi071.stdout: python3-kmod-0.9-32.el9.x86_64 python3-urwid-2.1.2-4.el9.x86_64 2024-09-11T07:06:56.447 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T07:06:56.447 INFO:teuthology.orchestra.run.smithi071.stdout:Complete! 2024-09-11T07:06:56.505 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: nvme-cli-2.9.1-6.el9.x86_64 5/6 2024-09-11T07:06:56.533 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : nvmetcli-0.7-3.el9.noarch 6/6 2024-09-11T07:06:56.804 DEBUG:teuthology.parallel:result is None 2024-09-11T07:06:57.564 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: nvmetcli-0.7-3.el9.noarch 6/6 2024-09-11T07:06:57.564 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/6 2024-09-11T07:06:57.564 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/6 2024-09-11T07:06:57.564 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : nvmetcli-0.7-3.el9.noarch 3/6 2024-09-11T07:06:57.564 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-configshell-1:1.1.30-1.el9.noarch 4/6 2024-09-11T07:06:57.564 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-kmod-0.9-32.el9.x86_64 5/6 2024-09-11T07:06:57.935 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-urwid-2.1.2-4.el9.x86_64 6/6 2024-09-11T07:06:57.935 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-11T07:06:57.935 INFO:teuthology.orchestra.run.smithi196.stdout:Installed: 2024-09-11T07:06:57.935 INFO:teuthology.orchestra.run.smithi196.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-11T07:06:57.935 INFO:teuthology.orchestra.run.smithi196.stdout: nvmetcli-0.7-3.el9.noarch python3-configshell-1:1.1.30-1.el9.noarch 2024-09-11T07:06:57.935 INFO:teuthology.orchestra.run.smithi196.stdout: python3-kmod-0.9-32.el9.x86_64 python3-urwid-2.1.2-4.el9.x86_64 2024-09-11T07:06:57.935 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-11T07:06:57.935 INFO:teuthology.orchestra.run.smithi196.stdout:Complete! 2024-09-11T07:06:58.272 DEBUG:teuthology.parallel:result is None 2024-09-11T07:06:58.272 INFO:teuthology.run_tasks:Running task install... 2024-09-11T07:06:58.280 DEBUG:teuthology.task.install:project ceph 2024-09-11T07:06:58.280 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'extra_system_packages': {'deb': ['python3-pytest'], 'rpm': ['python3-pytest']}, 'flavor': 'default', 'sha1': 'd0e6828a2016d48cf25ad84064e50742bb1c39b9'}} 2024-09-11T07:06:58.281 DEBUG:teuthology.task.install:config {'extra_system_packages': {'deb': ['python3-pytest'], 'rpm': ['python3-pytest']}, 'flavor': 'default', 'sha1': 'd0e6828a2016d48cf25ad84064e50742bb1c39b9'} 2024-09-11T07:06:58.281 INFO:teuthology.task.install:Using flavor: default 2024-09-11T07:06:58.286 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-11T07:06:58.286 INFO:teuthology.task.install:extra packages: [] 2024-09-11T07:06:58.286 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': {'deb': ['python3-pytest'], 'rpm': ['python3-pytest']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd0e6828a2016d48cf25ad84064e50742bb1c39b9', 'tag': None, 'wait_for_package': False} 2024-09-11T07:06:58.287 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d0e6828a2016d48cf25ad84064e50742bb1c39b9 2024-09-11T07:06:58.289 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': {'deb': ['python3-pytest'], 'rpm': ['python3-pytest']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd0e6828a2016d48cf25ad84064e50742bb1c39b9', 'tag': None, 'wait_for_package': False} 2024-09-11T07:06:58.289 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d0e6828a2016d48cf25ad84064e50742bb1c39b9 2024-09-11T07:06:58.437 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-10-2008-quincy/d0e6828a2016d48cf25ad84064e50742bb1c39b9/centos/9/flavors/default/ 2024-09-11T07:06:58.437 INFO:teuthology.task.install.rpm:Package version is 17.2.7-1623.gd0e6828a 2024-09-11T07:06:58.441 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-10-2008-quincy/d0e6828a2016d48cf25ad84064e50742bb1c39b9/centos/9/flavors/default/ 2024-09-11T07:06:58.441 INFO:teuthology.task.install.rpm:Package version is 17.2.7-1623.gd0e6828a 2024-09-11T07:06:58.549 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-10-2008-quincy/d0e6828a2016d48cf25ad84064e50742bb1c39b9/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-10-2008-quincy/d0e6828a2016d48cf25ad84064e50742bb1c39b9/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-10-2008-quincy/d0e6828a2016d48cf25ad84064e50742bb1c39b9/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-11T07:06:58.549 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-11T07:06:58.549 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-11T07:06:58.557 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-10-2008-quincy/d0e6828a2016d48cf25ad84064e50742bb1c39b9/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-10-2008-quincy/d0e6828a2016d48cf25ad84064e50742bb1c39b9/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-10-2008-quincy/d0e6828a2016d48cf25ad84064e50742bb1c39b9/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-11T07:06:58.557 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-11T07:06:58.558 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-11T07:06:58.577 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-pytest, python3-pytest, python3-pytest on remote rpm x86_64 2024-09-11T07:06:58.577 DEBUG:teuthology.orchestra.run.smithi196:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d0e6828a2016d48cf25ad84064e50742bb1c39b9/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-11T07:06:58.590 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-pytest, python3-pytest, python3-pytest, python3-pytest on remote rpm x86_64 2024-09-11T07:06:58.591 DEBUG:teuthology.orchestra.run.smithi071:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d0e6828a2016d48cf25ad84064e50742bb1c39b9/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-11T07:06:58.642 DEBUG:teuthology.orchestra.run.smithi196:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-11T07:06:58.660 DEBUG:teuthology.orchestra.run.smithi071:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-11T07:06:58.718 DEBUG:teuthology.orchestra.run.smithi196:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-11T07:06:58.741 DEBUG:teuthology.orchestra.run.smithi071:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-11T07:06:58.783 INFO:teuthology.orchestra.run.smithi196.stdout:check_obsoletes = 1 2024-09-11T07:06:58.785 DEBUG:teuthology.orchestra.run.smithi196:> sudo yum clean all 2024-09-11T07:06:58.808 INFO:teuthology.orchestra.run.smithi071.stdout:check_obsoletes = 1 2024-09-11T07:06:58.810 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum clean all 2024-09-11T07:06:59.006 INFO:teuthology.orchestra.run.smithi196.stdout:45 files removed 2024-09-11T07:06:59.030 INFO:teuthology.orchestra.run.smithi071.stdout:45 files removed 2024-09-11T07:06:59.032 DEBUG:teuthology.orchestra.run.smithi196:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-pytest python3-pytest python3-pytest python3-pytest 2024-09-11T07:06:59.057 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-pytest python3-pytest python3-pytest python3-pytest 2024-09-11T07:06:59.615 INFO:teuthology.orchestra.run.smithi196.stdout:ceph packages for x86_64 261 kB/s | 77 kB 00:00 2024-09-11T07:06:59.620 INFO:teuthology.orchestra.run.smithi071.stdout:ceph packages for x86_64 278 kB/s | 77 kB 00:00 2024-09-11T07:06:59.899 INFO:teuthology.orchestra.run.smithi196.stdout:ceph noarch packages 45 kB/s | 11 kB 00:00 2024-09-11T07:06:59.904 INFO:teuthology.orchestra.run.smithi071.stdout:ceph noarch packages 45 kB/s | 11 kB 00:00 2024-09-11T07:07:00.174 INFO:teuthology.orchestra.run.smithi196.stdout:ceph source packages 7.5 kB/s | 1.8 kB 00:00 2024-09-11T07:07:00.179 INFO:teuthology.orchestra.run.smithi071.stdout:ceph source packages 7.4 kB/s | 1.8 kB 00:00 2024-09-11T07:07:00.946 INFO:teuthology.orchestra.run.smithi071.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.2 MB 00:00 2024-09-11T07:07:01.924 INFO:teuthology.orchestra.run.smithi196.stdout:CentOS Stream 9 - BaseOS 4.8 MB/s | 8.2 MB 00:01 2024-09-11T07:07:03.809 INFO:teuthology.orchestra.run.smithi196.stdout:CentOS Stream 9 - AppStream 26 MB/s | 20 MB 00:00 2024-09-11T07:07:08.076 INFO:teuthology.orchestra.run.smithi196.stdout:CentOS Stream 9 - CRB 15 MB/s | 6.5 MB 00:00 2024-09-11T07:07:10.310 INFO:teuthology.orchestra.run.smithi196.stdout:CentOS Stream 9 - Extras packages 19 kB/s | 19 kB 00:00 2024-09-11T07:07:10.787 INFO:teuthology.orchestra.run.smithi071.stdout:CentOS Stream 9 - AppStream 2.3 MB/s | 20 MB 00:08 2024-09-11T07:07:11.256 INFO:teuthology.orchestra.run.smithi196.stdout:Extra Packages for Enterprise Linux 28 MB/s | 23 MB 00:00 2024-09-11T07:07:16.346 INFO:teuthology.orchestra.run.smithi071.stdout:CentOS Stream 9 - CRB 3.8 MB/s | 6.5 MB 00:01 2024-09-11T07:07:16.745 INFO:teuthology.orchestra.run.smithi196.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-09-11T07:07:18.022 INFO:teuthology.orchestra.run.smithi071.stdout:CentOS Stream 9 - Extras packages 63 kB/s | 19 kB 00:00 2024-09-11T07:07:18.951 INFO:teuthology.orchestra.run.smithi071.stdout:Extra Packages for Enterprise Linux 28 MB/s | 23 MB 00:00 2024-09-11T07:07:19.148 INFO:teuthology.orchestra.run.smithi196.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-11T07:07:19.148 INFO:teuthology.orchestra.run.smithi196.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-11T07:07:19.211 INFO:teuthology.orchestra.run.smithi196.stdout:Dependencies resolved. 2024-09-11T07:07:19.217 INFO:teuthology.orchestra.run.smithi196.stdout:======================================================================================= 2024-09-11T07:07:19.217 INFO:teuthology.orchestra.run.smithi196.stdout: Package Arch Version Repository Size 2024-09-11T07:07:19.217 INFO:teuthology.orchestra.run.smithi196.stdout:======================================================================================= 2024-09-11T07:07:19.217 INFO:teuthology.orchestra.run.smithi196.stdout:Installing: 2024-09-11T07:07:19.218 INFO:teuthology.orchestra.run.smithi196.stdout: ceph x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 6.5 k 2024-09-11T07:07:19.218 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-base x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 5.0 M 2024-09-11T07:07:19.218 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-fuse x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 813 k 2024-09-11T07:07:19.218 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-immutable-object-cache x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 137 k 2024-09-11T07:07:19.218 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 1.4 M 2024-09-11T07:07:19.218 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-cephadm noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 124 k 2024-09-11T07:07:19.218 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-dashboard noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 3.4 M 2024-09-11T07:07:19.218 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 7.4 M 2024-09-11T07:07:19.218 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-rook noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 47 k 2024-09-11T07:07:19.218 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-radosgw x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 14 M 2024-09-11T07:07:19.218 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-test x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 48 M 2024-09-11T07:07:19.218 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-volume noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 270 k 2024-09-11T07:07:19.219 INFO:teuthology.orchestra.run.smithi196.stdout: cephadm noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 83 k 2024-09-11T07:07:19.219 INFO:teuthology.orchestra.run.smithi196.stdout: libcephfs-devel x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 25 k 2024-09-11T07:07:19.219 INFO:teuthology.orchestra.run.smithi196.stdout: libcephfs2 x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 636 k 2024-09-11T07:07:19.219 INFO:teuthology.orchestra.run.smithi196.stdout: librados-devel x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 126 k 2024-09-11T07:07:19.219 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cephfs x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 156 k 2024-09-11T07:07:19.219 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pytest noarch 6.2.2-7.el9 appstream 519 k 2024-09-11T07:07:19.219 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rados x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 320 k 2024-09-11T07:07:19.219 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rbd x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 294 k 2024-09-11T07:07:19.219 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rgw x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 99 k 2024-09-11T07:07:19.219 INFO:teuthology.orchestra.run.smithi196.stdout: rbd-fuse x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 82 k 2024-09-11T07:07:19.219 INFO:teuthology.orchestra.run.smithi196.stdout: rbd-mirror x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 2.9 M 2024-09-11T07:07:19.219 INFO:teuthology.orchestra.run.smithi196.stdout: rbd-nbd x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 165 k 2024-09-11T07:07:19.219 INFO:teuthology.orchestra.run.smithi196.stdout:Upgrading: 2024-09-11T07:07:19.220 INFO:teuthology.orchestra.run.smithi196.stdout: librados2 x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 3.1 M 2024-09-11T07:07:19.220 INFO:teuthology.orchestra.run.smithi196.stdout: librbd1 x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 2.9 M 2024-09-11T07:07:19.220 INFO:teuthology.orchestra.run.smithi196.stdout:Installing dependencies: 2024-09-11T07:07:19.220 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-common x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 23 M 2024-09-11T07:07:19.220 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-grafana-dashboards noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 24 k 2024-09-11T07:07:19.220 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mds x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 2.1 M 2024-09-11T07:07:19.220 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-modules-core noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 240 k 2024-09-11T07:07:19.220 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mon x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 3.9 M 2024-09-11T07:07:19.220 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-osd x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 15 M 2024-09-11T07:07:19.220 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-prometheus-alerts noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 15 k 2024-09-11T07:07:19.220 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-selinux x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 25 k 2024-09-11T07:07:19.220 INFO:teuthology.orchestra.run.smithi196.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-11T07:07:19.220 INFO:teuthology.orchestra.run.smithi196.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-11T07:07:19.220 INFO:teuthology.orchestra.run.smithi196.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-11T07:07:19.221 INFO:teuthology.orchestra.run.smithi196.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-11T07:07:19.221 INFO:teuthology.orchestra.run.smithi196.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-11T07:07:19.221 INFO:teuthology.orchestra.run.smithi196.stdout: libcephsqlite x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 163 k 2024-09-11T07:07:19.221 INFO:teuthology.orchestra.run.smithi196.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-11T07:07:19.221 INFO:teuthology.orchestra.run.smithi196.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-11T07:07:19.221 INFO:teuthology.orchestra.run.smithi196.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-11T07:07:19.221 INFO:teuthology.orchestra.run.smithi196.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-11T07:07:19.221 INFO:teuthology.orchestra.run.smithi196.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-11T07:07:19.221 INFO:teuthology.orchestra.run.smithi196.stdout: libradosstriper1 x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 457 k 2024-09-11T07:07:19.221 INFO:teuthology.orchestra.run.smithi196.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-11T07:07:19.221 INFO:teuthology.orchestra.run.smithi196.stdout: librgw2 x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 4.9 M 2024-09-11T07:07:19.221 INFO:teuthology.orchestra.run.smithi196.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-11T07:07:19.222 INFO:teuthology.orchestra.run.smithi196.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-11T07:07:19.222 INFO:teuthology.orchestra.run.smithi196.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-11T07:07:19.222 INFO:teuthology.orchestra.run.smithi196.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-11T07:07:19.222 INFO:teuthology.orchestra.run.smithi196.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-11T07:07:19.222 INFO:teuthology.orchestra.run.smithi196.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-11T07:07:19.222 INFO:teuthology.orchestra.run.smithi196.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-11T07:07:19.222 INFO:teuthology.orchestra.run.smithi196.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-11T07:07:19.222 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-11T07:07:19.222 INFO:teuthology.orchestra.run.smithi196.stdout: python3-ceph-argparse x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 45 k 2024-09-11T07:07:19.222 INFO:teuthology.orchestra.run.smithi196.stdout: python3-ceph-common x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 120 k 2024-09-11T07:07:19.222 INFO:teuthology.orchestra.run.smithi196.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-11T07:07:19.222 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-11T07:07:19.223 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-11T07:07:19.223 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-11T07:07:19.223 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-11T07:07:19.223 INFO:teuthology.orchestra.run.smithi196.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-11T07:07:19.223 INFO:teuthology.orchestra.run.smithi196.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-11T07:07:19.223 INFO:teuthology.orchestra.run.smithi196.stdout: python3-iniconfig noarch 1.1.1-7.el9 appstream 17 k 2024-09-11T07:07:19.223 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-11T07:07:19.223 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-11T07:07:19.223 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-11T07:07:19.223 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-11T07:07:19.223 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-11T07:07:19.224 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-11T07:07:19.224 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-11T07:07:19.224 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-11T07:07:19.224 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-09-11T07:07:19.224 INFO:teuthology.orchestra.run.smithi196.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-11T07:07:19.224 INFO:teuthology.orchestra.run.smithi196.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-11T07:07:19.224 INFO:teuthology.orchestra.run.smithi196.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-11T07:07:19.224 INFO:teuthology.orchestra.run.smithi196.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-11T07:07:19.224 INFO:teuthology.orchestra.run.smithi196.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-11T07:07:19.224 INFO:teuthology.orchestra.run.smithi196.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-11T07:07:19.224 INFO:teuthology.orchestra.run.smithi196.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-11T07:07:19.224 INFO:teuthology.orchestra.run.smithi196.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-11T07:07:19.224 INFO:teuthology.orchestra.run.smithi196.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-11T07:07:19.224 INFO:teuthology.orchestra.run.smithi196.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-11T07:07:19.225 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-11T07:07:19.225 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pluggy noarch 0.13.1-7.el9 appstream 41 k 2024-09-11T07:07:19.225 INFO:teuthology.orchestra.run.smithi196.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-11T07:07:19.225 INFO:teuthology.orchestra.run.smithi196.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-11T07:07:19.225 INFO:teuthology.orchestra.run.smithi196.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-11T07:07:19.225 INFO:teuthology.orchestra.run.smithi196.stdout: python3-py noarch 1.10.0-6.el9 appstream 477 k 2024-09-11T07:07:19.225 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-11T07:07:19.225 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-11T07:07:19.225 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-11T07:07:19.225 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-11T07:07:19.225 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-11T07:07:19.225 INFO:teuthology.orchestra.run.smithi196.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-11T07:07:19.225 INFO:teuthology.orchestra.run.smithi196.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-11T07:07:19.225 INFO:teuthology.orchestra.run.smithi196.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-11T07:07:19.225 INFO:teuthology.orchestra.run.smithi196.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-11T07:07:19.226 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-11T07:07:19.226 INFO:teuthology.orchestra.run.smithi196.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-11T07:07:19.226 INFO:teuthology.orchestra.run.smithi196.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-11T07:07:19.226 INFO:teuthology.orchestra.run.smithi196.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-11T07:07:19.226 INFO:teuthology.orchestra.run.smithi196.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-11T07:07:19.226 INFO:teuthology.orchestra.run.smithi196.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-11T07:07:19.226 INFO:teuthology.orchestra.run.smithi196.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-11T07:07:19.226 INFO:teuthology.orchestra.run.smithi196.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-11T07:07:19.226 INFO:teuthology.orchestra.run.smithi196.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-11T07:07:19.226 INFO:teuthology.orchestra.run.smithi196.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-11T07:07:19.226 INFO:teuthology.orchestra.run.smithi196.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-11T07:07:19.226 INFO:teuthology.orchestra.run.smithi196.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-11T07:07:19.226 INFO:teuthology.orchestra.run.smithi196.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-11T07:07:19.226 INFO:teuthology.orchestra.run.smithi196.stdout:Installing weak dependencies: 2024-09-11T07:07:19.227 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-09-11T07:07:19.227 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-11T07:07:19.227 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction Summary 2024-09-11T07:07:19.227 INFO:teuthology.orchestra.run.smithi196.stdout:======================================================================================= 2024-09-11T07:07:19.227 INFO:teuthology.orchestra.run.smithi196.stdout:Install 113 Packages 2024-09-11T07:07:19.227 INFO:teuthology.orchestra.run.smithi196.stdout:Upgrade 2 Packages 2024-09-11T07:07:19.227 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-11T07:07:19.227 INFO:teuthology.orchestra.run.smithi196.stdout:Total download size: 191 M 2024-09-11T07:07:19.227 INFO:teuthology.orchestra.run.smithi196.stdout:Downloading Packages: 2024-09-11T07:07:19.472 INFO:teuthology.orchestra.run.smithi196.stdout:(1/115): ceph-17.2.7-1623.gd0e6828a.el9.x86_64. 55 kB/s | 6.5 kB 00:00 2024-09-11T07:07:19.697 INFO:teuthology.orchestra.run.smithi196.stdout:(2/115): ceph-fuse-17.2.7-1623.gd0e6828a.el9.x8 3.5 MB/s | 813 kB 00:00 2024-09-11T07:07:19.805 INFO:teuthology.orchestra.run.smithi196.stdout:(3/115): ceph-base-17.2.7-1623.gd0e6828a.el9.x8 11 MB/s | 5.0 MB 00:00 2024-09-11T07:07:19.831 INFO:teuthology.orchestra.run.smithi196.stdout:(4/115): ceph-immutable-object-cache-17.2.7-162 1.0 MB/s | 137 kB 00:00 2024-09-11T07:07:19.914 INFO:teuthology.orchestra.run.smithi196.stdout:(5/115): ceph-mds-17.2.7-1623.gd0e6828a.el9.x86 19 MB/s | 2.1 MB 00:00 2024-09-11T07:07:20.065 INFO:teuthology.orchestra.run.smithi196.stdout:(6/115): ceph-mgr-17.2.7-1623.gd0e6828a.el9.x86 6.0 MB/s | 1.4 MB 00:00 2024-09-11T07:07:20.131 INFO:teuthology.orchestra.run.smithi196.stdout:(7/115): ceph-mon-17.2.7-1623.gd0e6828a.el9.x86 18 MB/s | 3.9 MB 00:00 2024-09-11T07:07:20.665 INFO:teuthology.orchestra.run.smithi196.stdout:(8/115): ceph-common-17.2.7-1623.gd0e6828a.el9. 17 MB/s | 23 MB 00:01 2024-09-11T07:07:20.715 INFO:teuthology.orchestra.run.smithi196.stdout:(9/115): ceph-selinux-17.2.7-1623.gd0e6828a.el9 506 kB/s | 25 kB 00:00 2024-09-11T07:07:20.866 INFO:teuthology.orchestra.run.smithi196.stdout:(10/115): ceph-radosgw-17.2.7-1623.gd0e6828a.el 20 MB/s | 14 MB 00:00 2024-09-11T07:07:20.933 INFO:teuthology.orchestra.run.smithi196.stdout:(11/115): libcephfs-devel-17.2.7-1623.gd0e6828a 375 kB/s | 25 kB 00:00 2024-09-11T07:07:21.016 INFO:teuthology.orchestra.run.smithi196.stdout:(12/115): libcephfs2-17.2.7-1623.gd0e6828a.el9. 7.5 MB/s | 636 kB 00:00 2024-09-11T07:07:21.083 INFO:teuthology.orchestra.run.smithi196.stdout:(13/115): libcephsqlite-17.2.7-1623.gd0e6828a.e 2.4 MB/s | 163 kB 00:00 2024-09-11T07:07:21.150 INFO:teuthology.orchestra.run.smithi196.stdout:(14/115): librados-devel-17.2.7-1623.gd0e6828a. 1.9 MB/s | 126 kB 00:00 2024-09-11T07:07:21.317 INFO:teuthology.orchestra.run.smithi196.stdout:(15/115): ceph-osd-17.2.7-1623.gd0e6828a.el9.x8 12 MB/s | 15 MB 00:01 2024-09-11T07:07:21.351 INFO:teuthology.orchestra.run.smithi196.stdout:(16/115): libradosstriper1-17.2.7-1623.gd0e6828 2.2 MB/s | 457 kB 00:00 2024-09-11T07:07:21.409 INFO:teuthology.orchestra.run.smithi196.stdout:(17/115): python3-ceph-argparse-17.2.7-1623.gd0 774 kB/s | 45 kB 00:00 2024-09-11T07:07:21.485 INFO:teuthology.orchestra.run.smithi196.stdout:(18/115): python3-ceph-common-17.2.7-1623.gd0e6 1.6 MB/s | 120 kB 00:00 2024-09-11T07:07:21.552 INFO:teuthology.orchestra.run.smithi196.stdout:(19/115): python3-cephfs-17.2.7-1623.gd0e6828a. 2.3 MB/s | 156 kB 00:00 2024-09-11T07:07:21.660 INFO:teuthology.orchestra.run.smithi196.stdout:(20/115): librgw2-17.2.7-1623.gd0e6828a.el9.x86 14 MB/s | 4.9 MB 00:00 2024-09-11T07:07:21.686 INFO:teuthology.orchestra.run.smithi196.stdout:(21/115): python3-rados-17.2.7-1623.gd0e6828a.e 2.3 MB/s | 320 kB 00:00 2024-09-11T07:07:21.719 INFO:teuthology.orchestra.run.smithi196.stdout:(22/115): python3-rbd-17.2.7-1623.gd0e6828a.el9 4.9 MB/s | 294 kB 00:00 2024-09-11T07:07:21.744 INFO:teuthology.orchestra.run.smithi196.stdout:(23/115): python3-rgw-17.2.7-1623.gd0e6828a.el9 1.7 MB/s | 99 kB 00:00 2024-09-11T07:07:21.769 INFO:teuthology.orchestra.run.smithi196.stdout:(24/115): rbd-fuse-17.2.7-1623.gd0e6828a.el9.x8 1.6 MB/s | 82 kB 00:00 2024-09-11T07:07:21.837 INFO:teuthology.orchestra.run.smithi196.stdout:(25/115): rbd-nbd-17.2.7-1623.gd0e6828a.el9.x86 2.4 MB/s | 165 kB 00:00 2024-09-11T07:07:21.887 INFO:teuthology.orchestra.run.smithi196.stdout:(26/115): rbd-mirror-17.2.7-1623.gd0e6828a.el9. 21 MB/s | 2.9 MB 00:00 2024-09-11T07:07:21.912 INFO:teuthology.orchestra.run.smithi196.stdout:(27/115): ceph-grafana-dashboards-17.2.7-1623.g 325 kB/s | 24 kB 00:00 2024-09-11T07:07:21.938 INFO:teuthology.orchestra.run.smithi196.stdout:(28/115): ceph-mgr-cephadm-17.2.7-1623.gd0e6828 2.4 MB/s | 124 kB 00:00 2024-09-11T07:07:22.280 INFO:teuthology.orchestra.run.smithi196.stdout:(29/115): ceph-mgr-diskprediction-local-17.2.7- 22 MB/s | 7.4 MB 00:00 2024-09-11T07:07:22.330 INFO:teuthology.orchestra.run.smithi196.stdout:(30/115): ceph-mgr-dashboard-17.2.7-1623.gd0e68 8.0 MB/s | 3.4 MB 00:00 2024-09-11T07:07:22.355 INFO:teuthology.orchestra.run.smithi196.stdout:(31/115): ceph-mgr-modules-core-17.2.7-1623.gd0 3.1 MB/s | 240 kB 00:00 2024-09-11T07:07:22.380 INFO:teuthology.orchestra.run.smithi196.stdout:(32/115): ceph-mgr-rook-17.2.7-1623.gd0e6828a.e 948 kB/s | 47 kB 00:00 2024-09-11T07:07:22.406 INFO:teuthology.orchestra.run.smithi196.stdout:(33/115): ceph-prometheus-alerts-17.2.7-1623.gd 289 kB/s | 15 kB 00:00 2024-09-11T07:07:22.439 INFO:teuthology.orchestra.run.smithi196.stdout:(34/115): ceph-volume-17.2.7-1623.gd0e6828a.el9 4.5 MB/s | 270 kB 00:00 2024-09-11T07:07:22.473 INFO:teuthology.orchestra.run.smithi196.stdout:(35/115): cephadm-17.2.7-1623.gd0e6828a.el9.noa 1.2 MB/s | 83 kB 00:00 2024-09-11T07:07:23.032 INFO:teuthology.orchestra.run.smithi196.stdout:(36/115): ceph-test-17.2.7-1623.gd0e6828a.el9.x 21 MB/s | 48 MB 00:02 2024-09-11T07:07:23.490 INFO:teuthology.orchestra.run.smithi196.stdout:(37/115): fuse-2.9.9-16.el9.x86_64.rpm 76 kB/s | 80 kB 00:01 2024-09-11T07:07:23.757 INFO:teuthology.orchestra.run.smithi196.stdout:(38/115): libgfortran-11.5.0-2.el9.x86_64.rpm 624 kB/s | 801 kB 00:01 2024-09-11T07:07:23.849 INFO:teuthology.orchestra.run.smithi196.stdout:(39/115): libquadmath-11.5.0-2.el9.x86_64.rpm 238 kB/s | 191 kB 00:00 2024-09-11T07:07:24.315 INFO:teuthology.orchestra.run.smithi071.stdout:lab-extras 48 kB/s | 1.7 kB 00:00 2024-09-11T07:07:24.316 INFO:teuthology.orchestra.run.smithi196.stdout:(40/115): python3-cffi-1.14.5-5.el9.x86_64.rpm 307 kB/s | 253 kB 00:00 2024-09-11T07:07:24.541 INFO:teuthology.orchestra.run.smithi196.stdout:(41/115): python3-ply-3.11-14.el9.noarch.rpm 154 kB/s | 106 kB 00:00 2024-09-11T07:07:24.733 INFO:teuthology.orchestra.run.smithi196.stdout:(42/115): python3-cryptography-36.0.1-4.el9.x86 1.3 MB/s | 1.2 MB 00:00 2024-09-11T07:07:25.042 INFO:teuthology.orchestra.run.smithi196.stdout:(43/115): python3-pycparser-2.20-6.el9.noarch.r 186 kB/s | 135 kB 00:00 2024-09-11T07:07:25.109 INFO:teuthology.orchestra.run.smithi196.stdout:(44/115): flexiblas-3.0.4-8.el9.x86_64.rpm 499 kB/s | 33 kB 00:00 2024-09-11T07:07:25.242 INFO:teuthology.orchestra.run.smithi196.stdout:(45/115): python3-requests-2.25.1-8.el9.noarch. 180 kB/s | 125 kB 00:00 2024-09-11T07:07:25.443 INFO:teuthology.orchestra.run.smithi196.stdout:(46/115): flexiblas-openblas-openmp-3.0.4-8.el9 91 kB/s | 18 kB 00:00 2024-09-11T07:07:25.493 INFO:teuthology.orchestra.run.smithi196.stdout:(47/115): python3-urllib3-1.26.5-6.el9.noarch.r 283 kB/s | 215 kB 00:00 2024-09-11T07:07:25.552 INFO:teuthology.orchestra.run.smithi196.stdout:(48/115): flexiblas-netlib-3.0.4-8.el9.x86_64.r 6.8 MB/s | 3.0 MB 00:00 2024-09-11T07:07:25.602 INFO:teuthology.orchestra.run.smithi196.stdout:(49/115): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.0 MB/s | 160 kB 00:00 2024-09-11T07:07:25.619 INFO:teuthology.orchestra.run.smithi196.stdout:(50/115): librabbitmq-0.11.0-7.el9.x86_64.rpm 361 kB/s | 45 kB 00:00 2024-09-11T07:07:25.669 INFO:teuthology.orchestra.run.smithi196.stdout:(51/115): openblas-0.3.26-2.el9.x86_64.rpm 772 kB/s | 38 kB 00:00 2024-09-11T07:07:25.703 INFO:teuthology.orchestra.run.smithi196.stdout:(52/115): librdkafka-1.6.1-102.el9.x86_64.rpm 4.3 MB/s | 662 kB 00:00 2024-09-11T07:07:25.728 INFO:teuthology.orchestra.run.smithi196.stdout:(53/115): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.3 MB/s | 292 kB 00:00 2024-09-11T07:07:25.787 INFO:teuthology.orchestra.run.smithi196.stdout:(54/115): python3-attrs-20.3.0-7.el9.noarch.rpm 1.0 MB/s | 87 kB 00:00 2024-09-11T07:07:25.870 INFO:teuthology.orchestra.run.smithi196.stdout:(55/115): python3-devel-3.9.19-8.el9.x86_64.rpm 2.9 MB/s | 245 kB 00:00 2024-09-11T07:07:25.937 INFO:teuthology.orchestra.run.smithi196.stdout:(56/115): python3-iniconfig-1.1.1-7.el9.noarch. 263 kB/s | 17 kB 00:00 2024-09-11T07:07:26.013 INFO:teuthology.orchestra.run.smithi196.stdout:(57/115): python3-babel-2.9.1-2.el9.noarch.rpm 21 MB/s | 6.0 MB 00:00 2024-09-11T07:07:26.071 INFO:teuthology.orchestra.run.smithi196.stdout:(58/115): python3-jsonpatch-1.21-16.el9.noarch. 440 kB/s | 26 kB 00:00 2024-09-11T07:07:26.096 INFO:teuthology.orchestra.run.smithi196.stdout:(59/115): python3-jinja2-2.11.3-6.el9.noarch.rp 1.5 MB/s | 248 kB 00:00 2024-09-11T07:07:26.147 INFO:teuthology.orchestra.run.smithi196.stdout:(60/115): python3-jsonpointer-2.0-4.el9.noarch. 255 kB/s | 19 kB 00:00 2024-09-11T07:07:26.189 INFO:teuthology.orchestra.run.smithi196.stdout:(61/115): python3-mako-1.1.4-6.el9.noarch.rpm 1.8 MB/s | 172 kB 00:00 2024-09-11T07:07:26.214 INFO:teuthology.orchestra.run.smithi196.stdout:(62/115): python3-markupsafe-1.1.1-12.el9.x86_6 519 kB/s | 35 kB 00:00 2024-09-11T07:07:26.298 INFO:teuthology.orchestra.run.smithi196.stdout:(63/115): openblas-openmp-0.3.26-2.el9.x86_64.r 7.9 MB/s | 4.9 MB 00:00 2024-09-11T07:07:26.373 INFO:teuthology.orchestra.run.smithi196.stdout:(64/115): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.5 MB/s | 250 kB 00:00 2024-09-11T07:07:26.398 INFO:teuthology.orchestra.run.smithi196.stdout:(65/115): python3-oauthlib-3.1.1-5.el9.noarch.r 2.2 MB/s | 222 kB 00:00 2024-09-11T07:07:26.457 INFO:teuthology.orchestra.run.smithi196.stdout:(66/115): python3-packaging-20.9-5.el9.noarch.r 926 kB/s | 77 kB 00:00 2024-09-11T07:07:26.482 INFO:teuthology.orchestra.run.smithi196.stdout:(67/115): python3-pluggy-0.13.1-7.el9.noarch.rp 496 kB/s | 41 kB 00:00 2024-09-11T07:07:26.532 INFO:teuthology.orchestra.run.smithi196.stdout:(68/115): python3-prettytable-0.7.2-27.el9.noar 561 kB/s | 42 kB 00:00 2024-09-11T07:07:26.616 INFO:teuthology.orchestra.run.smithi196.stdout:(69/115): python3-py-1.10.0-6.el9.noarch.rpm 3.5 MB/s | 477 kB 00:00 2024-09-11T07:07:26.641 INFO:teuthology.orchestra.run.smithi196.stdout:(70/115): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.4 MB/s | 159 kB 00:00 2024-09-11T07:07:26.721 INFO:teuthology.orchestra.run.smithi071.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-11T07:07:26.721 INFO:teuthology.orchestra.run.smithi071.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-11T07:07:26.733 INFO:teuthology.orchestra.run.smithi196.stdout:(71/115): python3-pyasn1-modules-0.4.8-6.el9.no 2.3 MB/s | 279 kB 00:00 2024-09-11T07:07:26.788 INFO:teuthology.orchestra.run.smithi071.stdout:Dependencies resolved. 2024-09-11T07:07:26.795 INFO:teuthology.orchestra.run.smithi071.stdout:======================================================================================= 2024-09-11T07:07:26.795 INFO:teuthology.orchestra.run.smithi071.stdout: Package Arch Version Repository Size 2024-09-11T07:07:26.795 INFO:teuthology.orchestra.run.smithi071.stdout:======================================================================================= 2024-09-11T07:07:26.795 INFO:teuthology.orchestra.run.smithi071.stdout:Installing: 2024-09-11T07:07:26.795 INFO:teuthology.orchestra.run.smithi071.stdout: ceph x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 6.5 k 2024-09-11T07:07:26.795 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-base x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 5.0 M 2024-09-11T07:07:26.795 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-fuse x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 813 k 2024-09-11T07:07:26.795 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-immutable-object-cache x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 137 k 2024-09-11T07:07:26.795 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 1.4 M 2024-09-11T07:07:26.795 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-cephadm noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 124 k 2024-09-11T07:07:26.795 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-dashboard noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 3.4 M 2024-09-11T07:07:26.795 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 7.4 M 2024-09-11T07:07:26.796 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-rook noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 47 k 2024-09-11T07:07:26.796 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-radosgw x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 14 M 2024-09-11T07:07:26.796 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-test x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 48 M 2024-09-11T07:07:26.796 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-volume noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 270 k 2024-09-11T07:07:26.796 INFO:teuthology.orchestra.run.smithi071.stdout: cephadm noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 83 k 2024-09-11T07:07:26.796 INFO:teuthology.orchestra.run.smithi071.stdout: libcephfs-devel x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 25 k 2024-09-11T07:07:26.796 INFO:teuthology.orchestra.run.smithi071.stdout: libcephfs2 x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 636 k 2024-09-11T07:07:26.796 INFO:teuthology.orchestra.run.smithi071.stdout: librados-devel x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 126 k 2024-09-11T07:07:26.796 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cephfs x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 156 k 2024-09-11T07:07:26.796 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pytest noarch 6.2.2-7.el9 appstream 519 k 2024-09-11T07:07:26.796 INFO:teuthology.orchestra.run.smithi071.stdout: python3-rados x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 320 k 2024-09-11T07:07:26.796 INFO:teuthology.orchestra.run.smithi071.stdout: python3-rbd x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 294 k 2024-09-11T07:07:26.797 INFO:teuthology.orchestra.run.smithi071.stdout: python3-rgw x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 99 k 2024-09-11T07:07:26.797 INFO:teuthology.orchestra.run.smithi071.stdout: rbd-fuse x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 82 k 2024-09-11T07:07:26.797 INFO:teuthology.orchestra.run.smithi071.stdout: rbd-mirror x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 2.9 M 2024-09-11T07:07:26.797 INFO:teuthology.orchestra.run.smithi071.stdout: rbd-nbd x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 165 k 2024-09-11T07:07:26.797 INFO:teuthology.orchestra.run.smithi071.stdout:Upgrading: 2024-09-11T07:07:26.797 INFO:teuthology.orchestra.run.smithi071.stdout: librados2 x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 3.1 M 2024-09-11T07:07:26.797 INFO:teuthology.orchestra.run.smithi071.stdout: librbd1 x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 2.9 M 2024-09-11T07:07:26.797 INFO:teuthology.orchestra.run.smithi071.stdout:Installing dependencies: 2024-09-11T07:07:26.797 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-common x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 23 M 2024-09-11T07:07:26.797 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-grafana-dashboards noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 24 k 2024-09-11T07:07:26.797 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mds x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 2.1 M 2024-09-11T07:07:26.797 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-modules-core noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 240 k 2024-09-11T07:07:26.797 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mon x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 3.9 M 2024-09-11T07:07:26.798 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-osd x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 15 M 2024-09-11T07:07:26.798 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-prometheus-alerts noarch 2:17.2.7-1623.gd0e6828a.el9 ceph-noarch 15 k 2024-09-11T07:07:26.798 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-selinux x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 25 k 2024-09-11T07:07:26.798 INFO:teuthology.orchestra.run.smithi071.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-11T07:07:26.798 INFO:teuthology.orchestra.run.smithi071.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-11T07:07:26.798 INFO:teuthology.orchestra.run.smithi071.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-11T07:07:26.798 INFO:teuthology.orchestra.run.smithi071.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-11T07:07:26.798 INFO:teuthology.orchestra.run.smithi071.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-11T07:07:26.798 INFO:teuthology.orchestra.run.smithi071.stdout: libcephsqlite x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 163 k 2024-09-11T07:07:26.798 INFO:teuthology.orchestra.run.smithi071.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-11T07:07:26.798 INFO:teuthology.orchestra.run.smithi071.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-11T07:07:26.798 INFO:teuthology.orchestra.run.smithi071.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-11T07:07:26.798 INFO:teuthology.orchestra.run.smithi071.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-11T07:07:26.799 INFO:teuthology.orchestra.run.smithi071.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-11T07:07:26.799 INFO:teuthology.orchestra.run.smithi071.stdout: libradosstriper1 x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 457 k 2024-09-11T07:07:26.799 INFO:teuthology.orchestra.run.smithi071.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-11T07:07:26.799 INFO:teuthology.orchestra.run.smithi071.stdout: librgw2 x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 4.9 M 2024-09-11T07:07:26.799 INFO:teuthology.orchestra.run.smithi071.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-11T07:07:26.799 INFO:teuthology.orchestra.run.smithi071.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-11T07:07:26.799 INFO:teuthology.orchestra.run.smithi071.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-11T07:07:26.799 INFO:teuthology.orchestra.run.smithi071.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-11T07:07:26.799 INFO:teuthology.orchestra.run.smithi071.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-11T07:07:26.799 INFO:teuthology.orchestra.run.smithi071.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-11T07:07:26.799 INFO:teuthology.orchestra.run.smithi071.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-11T07:07:26.799 INFO:teuthology.orchestra.run.smithi071.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-11T07:07:26.800 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-11T07:07:26.800 INFO:teuthology.orchestra.run.smithi071.stdout: python3-ceph-argparse x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 45 k 2024-09-11T07:07:26.800 INFO:teuthology.orchestra.run.smithi071.stdout: python3-ceph-common x86_64 2:17.2.7-1623.gd0e6828a.el9 ceph 120 k 2024-09-11T07:07:26.800 INFO:teuthology.orchestra.run.smithi071.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-11T07:07:26.800 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-11T07:07:26.800 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-11T07:07:26.800 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-11T07:07:26.800 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-11T07:07:26.800 INFO:teuthology.orchestra.run.smithi071.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-11T07:07:26.800 INFO:teuthology.orchestra.run.smithi071.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-11T07:07:26.800 INFO:teuthology.orchestra.run.smithi071.stdout: python3-iniconfig noarch 1.1.1-7.el9 appstream 17 k 2024-09-11T07:07:26.800 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-11T07:07:26.800 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-11T07:07:26.801 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-11T07:07:26.801 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-11T07:07:26.801 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-11T07:07:26.801 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-11T07:07:26.801 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-11T07:07:26.801 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-11T07:07:26.801 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-09-11T07:07:26.801 INFO:teuthology.orchestra.run.smithi071.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-11T07:07:26.801 INFO:teuthology.orchestra.run.smithi071.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-11T07:07:26.801 INFO:teuthology.orchestra.run.smithi071.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-11T07:07:26.801 INFO:teuthology.orchestra.run.smithi071.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-11T07:07:26.801 INFO:teuthology.orchestra.run.smithi071.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-11T07:07:26.801 INFO:teuthology.orchestra.run.smithi071.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-11T07:07:26.802 INFO:teuthology.orchestra.run.smithi071.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-11T07:07:26.802 INFO:teuthology.orchestra.run.smithi071.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-11T07:07:26.802 INFO:teuthology.orchestra.run.smithi071.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-11T07:07:26.802 INFO:teuthology.orchestra.run.smithi071.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-11T07:07:26.802 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-11T07:07:26.802 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pluggy noarch 0.13.1-7.el9 appstream 41 k 2024-09-11T07:07:26.802 INFO:teuthology.orchestra.run.smithi071.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-11T07:07:26.802 INFO:teuthology.orchestra.run.smithi071.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-11T07:07:26.802 INFO:teuthology.orchestra.run.smithi071.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-11T07:07:26.802 INFO:teuthology.orchestra.run.smithi071.stdout: python3-py noarch 1.10.0-6.el9 appstream 477 k 2024-09-11T07:07:26.802 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-11T07:07:26.802 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-11T07:07:26.802 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-11T07:07:26.803 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-11T07:07:26.803 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-11T07:07:26.803 INFO:teuthology.orchestra.run.smithi071.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-11T07:07:26.803 INFO:teuthology.orchestra.run.smithi071.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-11T07:07:26.803 INFO:teuthology.orchestra.run.smithi071.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-11T07:07:26.803 INFO:teuthology.orchestra.run.smithi071.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-11T07:07:26.803 INFO:teuthology.orchestra.run.smithi071.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-11T07:07:26.803 INFO:teuthology.orchestra.run.smithi071.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-11T07:07:26.803 INFO:teuthology.orchestra.run.smithi071.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-11T07:07:26.803 INFO:teuthology.orchestra.run.smithi071.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-11T07:07:26.803 INFO:teuthology.orchestra.run.smithi071.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-11T07:07:26.803 INFO:teuthology.orchestra.run.smithi071.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-11T07:07:26.803 INFO:teuthology.orchestra.run.smithi071.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-11T07:07:26.804 INFO:teuthology.orchestra.run.smithi071.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-11T07:07:26.804 INFO:teuthology.orchestra.run.smithi071.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-11T07:07:26.804 INFO:teuthology.orchestra.run.smithi071.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-11T07:07:26.804 INFO:teuthology.orchestra.run.smithi071.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-11T07:07:26.804 INFO:teuthology.orchestra.run.smithi071.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-11T07:07:26.804 INFO:teuthology.orchestra.run.smithi071.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-11T07:07:26.804 INFO:teuthology.orchestra.run.smithi071.stdout:Installing weak dependencies: 2024-09-11T07:07:26.804 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-09-11T07:07:26.804 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T07:07:26.804 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction Summary 2024-09-11T07:07:26.804 INFO:teuthology.orchestra.run.smithi071.stdout:======================================================================================= 2024-09-11T07:07:26.804 INFO:teuthology.orchestra.run.smithi071.stdout:Install 113 Packages 2024-09-11T07:07:26.805 INFO:teuthology.orchestra.run.smithi071.stdout:Upgrade 2 Packages 2024-09-11T07:07:26.805 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T07:07:26.805 INFO:teuthology.orchestra.run.smithi196.stdout:(72/115): python3-pytest-6.2.2-7.el9.noarch.rpm 3.2 MB/s | 519 kB 00:00 2024-09-11T07:07:26.806 INFO:teuthology.orchestra.run.smithi071.stdout:Total download size: 191 M 2024-09-11T07:07:26.806 INFO:teuthology.orchestra.run.smithi071.stdout:Downloading Packages: 2024-09-11T07:07:26.825 INFO:teuthology.orchestra.run.smithi196.stdout:(73/115): python3-pytz-2021.1-5.el9.noarch.rpm 553 kB/s | 51 kB 00:00 2024-09-11T07:07:26.884 INFO:teuthology.orchestra.run.smithi196.stdout:(74/115): python3-requests-oauthlib-1.3.0-12.el 644 kB/s | 54 kB 00:00 2024-09-11T07:07:26.945 INFO:teuthology.orchestra.run.smithi196.stdout:(75/115): python3-toml-0.10.2-6.el9.noarch.rpm 691 kB/s | 42 kB 00:00 2024-09-11T07:07:27.053 INFO:teuthology.orchestra.run.smithi196.stdout:(76/115): socat-1.7.4.1-6.el9.x86_64.rpm 2.7 MB/s | 304 kB 00:00 2024-09-11T07:07:27.137 INFO:teuthology.orchestra.run.smithi196.stdout:(77/115): xmlstarlet-1.6.1-20.el9.x86_64.rpm 765 kB/s | 64 kB 00:00 2024-09-11T07:07:27.204 INFO:teuthology.orchestra.run.smithi196.stdout:(78/115): python3-numpy-1.20.1-5.el9.x86_64.rpm 5.0 MB/s | 5.1 MB 00:01 2024-09-11T07:07:27.254 INFO:teuthology.orchestra.run.smithi196.stdout:(79/115): gperftools-libs-2.9.1-2.el9.x86_64.rp 2.6 MB/s | 309 kB 00:00 2024-09-11T07:07:27.271 INFO:teuthology.orchestra.run.smithi196.stdout:(80/115): liboath-2.6.7-2.el9.x86_64.rpm 728 kB/s | 49 kB 00:00 2024-09-11T07:07:27.296 INFO:teuthology.orchestra.run.smithi196.stdout:(81/115): libunwind-1.6.2-1.el9.x86_64.rpm 1.6 MB/s | 67 kB 00:00 2024-09-11T07:07:27.313 INFO:teuthology.orchestra.run.smithi196.stdout:(82/115): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 2.6 MB/s | 43 kB 00:00 2024-09-11T07:07:27.330 INFO:teuthology.orchestra.run.smithi196.stdout:(83/115): python3-cachetools-4.2.4-1.el9.noarch 1.9 MB/s | 32 kB 00:00 2024-09-11T07:07:27.347 INFO:teuthology.orchestra.run.smithi196.stdout:(84/115): python3-certifi-2023.05.07-4.el9.noar 851 kB/s | 14 kB 00:00 2024-09-11T07:07:27.372 INFO:teuthology.orchestra.run.smithi196.stdout:(85/115): python3-cheroot-8.6.0-4.el9.noarch.rp 6.8 MB/s | 172 kB 00:00 2024-09-11T07:07:27.398 INFO:teuthology.orchestra.run.smithi196.stdout:(86/115): python3-asyncssh-2.13.2-5.el9.noarch. 4.2 MB/s | 548 kB 00:00 2024-09-11T07:07:27.431 INFO:teuthology.orchestra.run.smithi196.stdout:(87/115): python3-cherrypy-18.6.1-2.el9.noarch. 6.0 MB/s | 358 kB 00:00 2024-09-11T07:07:27.457 INFO:teuthology.orchestra.run.smithi196.stdout:(88/115): python3-jaraco-8.2.1-3.el9.noarch.rpm 431 kB/s | 11 kB 00:00 2024-09-11T07:07:27.482 INFO:teuthology.orchestra.run.smithi196.stdout:(89/115): python3-google-auth-2.34.0-1.el9.noar 2.6 MB/s | 220 kB 00:00 2024-09-11T07:07:27.507 INFO:teuthology.orchestra.run.smithi196.stdout:(90/115): python3-jaraco-classes-3.2.1-5.el9.no 352 kB/s | 18 kB 00:00 2024-09-11T07:07:27.533 INFO:teuthology.orchestra.run.smithi196.stdout:(91/115): python3-jaraco-collections-3.0.0-8.el 461 kB/s | 23 kB 00:00 2024-09-11T07:07:27.558 INFO:teuthology.orchestra.run.smithi196.stdout:(92/115): python3-jaraco-functools-3.5.0-2.el9. 387 kB/s | 19 kB 00:00 2024-09-11T07:07:27.583 INFO:teuthology.orchestra.run.smithi196.stdout:(93/115): python3-jaraco-text-3.2.0-6.el9.noarc 392 kB/s | 20 kB 00:00 2024-09-11T07:07:27.608 INFO:teuthology.orchestra.run.smithi196.stdout:(94/115): python3-jwt+crypto-2.4.0-1.el9.noarch 180 kB/s | 9.0 kB 00:00 2024-09-11T07:07:27.633 INFO:teuthology.orchestra.run.smithi196.stdout:(95/115): python3-jwt-2.4.0-1.el9.noarch.rpm 816 kB/s | 41 kB 00:00 2024-09-11T07:07:27.659 INFO:teuthology.orchestra.run.smithi196.stdout:(96/115): python3-logutils-0.3.5-21.el9.noarch. 1.8 MB/s | 46 kB 00:00 2024-09-11T07:07:27.682 INFO:teuthology.orchestra.run.smithi071.stdout:(1/115): ceph-17.2.7-1623.gd0e6828a.el9.x86_64. 48 kB/s | 6.5 kB 00:00 2024-09-11T07:07:27.692 INFO:teuthology.orchestra.run.smithi196.stdout:(97/115): python3-more-itertools-8.12.0-2.el9.n 2.3 MB/s | 79 kB 00:00 2024-09-11T07:07:27.718 INFO:teuthology.orchestra.run.smithi196.stdout:(98/115): python3-natsort-7.1.1-5.el9.noarch.rp 2.2 MB/s | 58 kB 00:00 2024-09-11T07:07:27.752 INFO:teuthology.orchestra.run.smithi196.stdout:(99/115): python3-kubernetes-26.1.0-2.el9.noarc 7.1 MB/s | 1.0 MB 00:00 2024-09-11T07:07:27.777 INFO:teuthology.orchestra.run.smithi196.stdout:(100/115): python3-portend-3.1.0-2.el9.noarch.r 662 kB/s | 16 kB 00:00 2024-09-11T07:07:27.802 INFO:teuthology.orchestra.run.smithi196.stdout:(101/115): python3-pecan-1.4.2-3.el9.noarch.rpm 3.2 MB/s | 272 kB 00:00 2024-09-11T07:07:27.836 INFO:teuthology.orchestra.run.smithi196.stdout:(102/115): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.5 MB/s | 90 kB 00:00 2024-09-11T07:07:27.861 INFO:teuthology.orchestra.run.smithi196.stdout:(103/115): python3-repoze-lru-0.7-16.el9.noarch 526 kB/s | 31 kB 00:00 2024-09-11T07:07:27.886 INFO:teuthology.orchestra.run.smithi196.stdout:(104/115): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-11T07:07:27.907 INFO:teuthology.orchestra.run.smithi071.stdout:(2/115): ceph-fuse-17.2.7-1623.gd0e6828a.el9.x8 3.5 MB/s | 813 kB 00:00 2024-09-11T07:07:27.912 INFO:teuthology.orchestra.run.smithi196.stdout:(105/115): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-09-11T07:07:27.937 INFO:teuthology.orchestra.run.smithi196.stdout:(106/115): python3-tempora-5.0.0-2.el9.noarch.r 711 kB/s | 36 kB 00:00 2024-09-11T07:07:27.963 INFO:teuthology.orchestra.run.smithi196.stdout:(107/115): python3-typing-extensions-4.12.2-2.e 1.4 MB/s | 74 kB 00:00 2024-09-11T07:07:27.988 INFO:teuthology.orchestra.run.smithi196.stdout:(108/115): python3-websocket-client-1.2.3-2.el9 3.5 MB/s | 90 kB 00:00 2024-09-11T07:07:28.013 INFO:teuthology.orchestra.run.smithi196.stdout:(109/115): python3-webob-1.8.8-2.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-09-11T07:07:28.047 INFO:teuthology.orchestra.run.smithi196.stdout:(110/115): python3-zc-lockfile-2.0-10.el9.noarc 602 kB/s | 20 kB 00:00 2024-09-11T07:07:28.049 INFO:teuthology.orchestra.run.smithi071.stdout:(3/115): ceph-base-17.2.7-1623.gd0e6828a.el9.x8 9.9 MB/s | 5.0 MB 00:00 2024-09-11T07:07:28.074 INFO:teuthology.orchestra.run.smithi071.stdout:(4/115): ceph-immutable-object-cache-17.2.7-162 824 kB/s | 137 kB 00:00 2024-09-11T07:07:28.080 INFO:teuthology.orchestra.run.smithi196.stdout:(111/115): python3-werkzeug-2.0.3-3.el9.1.noarc 4.5 MB/s | 427 kB 00:00 2024-09-11T07:07:28.139 INFO:teuthology.orchestra.run.smithi196.stdout:(112/115): thrift-0.15.0-2.el9.x86_64.rpm 17 MB/s | 1.6 MB 00:00 2024-09-11T07:07:28.150 INFO:teuthology.orchestra.run.smithi071.stdout:(5/115): ceph-mds-17.2.7-1623.gd0e6828a.el9.x86 21 MB/s | 2.1 MB 00:00 2024-09-11T07:07:28.367 INFO:teuthology.orchestra.run.smithi071.stdout:(6/115): ceph-mgr-17.2.7-1623.gd0e6828a.el9.x86 4.8 MB/s | 1.4 MB 00:00 2024-09-11T07:07:28.373 INFO:teuthology.orchestra.run.smithi196.stdout:(113/115): python3-scipy-1.6.2-8.el9.x86_64.rpm 11 MB/s | 16 MB 00:01 2024-09-11T07:07:28.442 INFO:teuthology.orchestra.run.smithi071.stdout:(7/115): ceph-mon-17.2.7-1623.gd0e6828a.el9.x86 13 MB/s | 3.9 MB 00:00 2024-09-11T07:07:28.631 INFO:teuthology.orchestra.run.smithi196.stdout:(114/115): librbd1-17.2.7-1623.gd0e6828a.el9.x8 6.0 MB/s | 2.9 MB 00:00 2024-09-11T07:07:28.676 INFO:teuthology.orchestra.run.smithi071.stdout:(8/115): ceph-common-17.2.7-1623.gd0e6828a.el9. 20 MB/s | 23 MB 00:01 2024-09-11T07:07:28.690 INFO:teuthology.orchestra.run.smithi196.stdout:(115/115): librados2-17.2.7-1623.gd0e6828a.el9. 5.1 MB/s | 3.1 MB 00:00 2024-09-11T07:07:28.695 INFO:teuthology.orchestra.run.smithi196.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:07:28.695 INFO:teuthology.orchestra.run.smithi196.stdout:Total 20 MB/s | 191 MB 00:09 2024-09-11T07:07:28.726 INFO:teuthology.orchestra.run.smithi071.stdout:(9/115): ceph-selinux-17.2.7-1623.gd0e6828a.el9 507 kB/s | 25 kB 00:00 2024-09-11T07:07:29.318 INFO:teuthology.orchestra.run.smithi071.stdout:(10/115): ceph-radosgw-17.2.7-1623.gd0e6828a.el 17 MB/s | 14 MB 00:00 2024-09-11T07:07:29.376 INFO:teuthology.orchestra.run.smithi071.stdout:(11/115): libcephfs-devel-17.2.7-1623.gd0e6828a 430 kB/s | 25 kB 00:00 2024-09-11T07:07:29.452 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction check 2024-09-11T07:07:29.493 INFO:teuthology.orchestra.run.smithi071.stdout:(12/115): libcephfs2-17.2.7-1623.gd0e6828a.el9. 5.3 MB/s | 636 kB 00:00 2024-09-11T07:07:29.548 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction check succeeded. 2024-09-11T07:07:29.548 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction test 2024-09-11T07:07:29.669 INFO:teuthology.orchestra.run.smithi071.stdout:(13/115): ceph-osd-17.2.7-1623.gd0e6828a.el9.x8 12 MB/s | 15 MB 00:01 2024-09-11T07:07:29.694 INFO:teuthology.orchestra.run.smithi071.stdout:(14/115): libcephsqlite-17.2.7-1623.gd0e6828a.e 813 kB/s | 163 kB 00:00 2024-09-11T07:07:29.719 INFO:teuthology.orchestra.run.smithi071.stdout:(15/115): librados-devel-17.2.7-1623.gd0e6828a. 2.5 MB/s | 126 kB 00:00 2024-09-11T07:07:29.745 INFO:teuthology.orchestra.run.smithi071.stdout:(16/115): libradosstriper1-17.2.7-1623.gd0e6828 8.9 MB/s | 457 kB 00:00 2024-09-11T07:07:29.795 INFO:teuthology.orchestra.run.smithi071.stdout:(17/115): python3-ceph-argparse-17.2.7-1623.gd0 907 kB/s | 45 kB 00:00 2024-09-11T07:07:29.879 INFO:teuthology.orchestra.run.smithi071.stdout:(18/115): python3-ceph-common-17.2.7-1623.gd0e6 1.4 MB/s | 120 kB 00:00 2024-09-11T07:07:29.970 INFO:teuthology.orchestra.run.smithi071.stdout:(19/115): librgw2-17.2.7-1623.gd0e6828a.el9.x86 20 MB/s | 4.9 MB 00:00 2024-09-11T07:07:29.995 INFO:teuthology.orchestra.run.smithi071.stdout:(20/115): python3-cephfs-17.2.7-1623.gd0e6828a. 1.3 MB/s | 156 kB 00:00 2024-09-11T07:07:30.021 INFO:teuthology.orchestra.run.smithi071.stdout:(21/115): python3-rados-17.2.7-1623.gd0e6828a.e 6.2 MB/s | 320 kB 00:00 2024-09-11T07:07:30.054 INFO:teuthology.orchestra.run.smithi071.stdout:(22/115): python3-rbd-17.2.7-1623.gd0e6828a.el9 4.9 MB/s | 294 kB 00:00 2024-09-11T07:07:30.079 INFO:teuthology.orchestra.run.smithi071.stdout:(23/115): python3-rgw-17.2.7-1623.gd0e6828a.el9 1.7 MB/s | 99 kB 00:00 2024-09-11T07:07:30.105 INFO:teuthology.orchestra.run.smithi071.stdout:(24/115): rbd-fuse-17.2.7-1623.gd0e6828a.el9.x8 1.6 MB/s | 82 kB 00:00 2024-09-11T07:07:30.164 INFO:teuthology.orchestra.run.smithi071.stdout:(25/115): rbd-nbd-17.2.7-1623.gd0e6828a.el9.x86 2.8 MB/s | 165 kB 00:00 2024-09-11T07:07:30.232 INFO:teuthology.orchestra.run.smithi071.stdout:(26/115): rbd-mirror-17.2.7-1623.gd0e6828a.el9. 20 MB/s | 2.9 MB 00:00 2024-09-11T07:07:30.256 INFO:teuthology.orchestra.run.smithi071.stdout:(27/115): ceph-grafana-dashboards-17.2.7-1623.g 266 kB/s | 24 kB 00:00 2024-09-11T07:07:30.281 INFO:teuthology.orchestra.run.smithi071.stdout:(28/115): ceph-mgr-cephadm-17.2.7-1623.gd0e6828 2.4 MB/s | 124 kB 00:00 2024-09-11T07:07:30.507 INFO:teuthology.orchestra.run.smithi071.stdout:(29/115): ceph-mgr-dashboard-17.2.7-1623.gd0e68 13 MB/s | 3.4 MB 00:00 2024-09-11T07:07:30.582 INFO:teuthology.orchestra.run.smithi071.stdout:(30/115): ceph-mgr-modules-core-17.2.7-1623.gd0 3.1 MB/s | 240 kB 00:00 2024-09-11T07:07:30.674 INFO:teuthology.orchestra.run.smithi071.stdout:(31/115): ceph-mgr-diskprediction-local-17.2.7- 19 MB/s | 7.4 MB 00:00 2024-09-11T07:07:30.699 INFO:teuthology.orchestra.run.smithi071.stdout:(32/115): ceph-mgr-rook-17.2.7-1623.gd0e6828a.e 407 kB/s | 47 kB 00:00 2024-09-11T07:07:30.724 INFO:teuthology.orchestra.run.smithi071.stdout:(33/115): ceph-prometheus-alerts-17.2.7-1623.gd 291 kB/s | 15 kB 00:00 2024-09-11T07:07:30.758 INFO:teuthology.orchestra.run.smithi071.stdout:(34/115): ceph-volume-17.2.7-1623.gd0e6828a.el9 4.5 MB/s | 270 kB 00:00 2024-09-11T07:07:30.783 INFO:teuthology.orchestra.run.smithi071.stdout:(35/115): cephadm-17.2.7-1623.gd0e6828a.el9.noa 1.4 MB/s | 83 kB 00:00 2024-09-11T07:07:31.208 INFO:teuthology.orchestra.run.smithi071.stdout:(36/115): ceph-test-17.2.7-1623.gd0e6828a.el9.x 19 MB/s | 48 MB 00:02 2024-09-11T07:07:31.217 INFO:teuthology.orchestra.run.smithi196.stdout:Transaction test succeeded. 2024-09-11T07:07:31.218 INFO:teuthology.orchestra.run.smithi196.stdout:Running transaction 2024-09-11T07:07:31.317 INFO:teuthology.orchestra.run.smithi071.stdout:(37/115): fuse-2.9.9-16.el9.x86_64.rpm 143 kB/s | 80 kB 00:00 2024-09-11T07:07:31.342 INFO:teuthology.orchestra.run.smithi071.stdout:(38/115): libquadmath-11.5.0-2.el9.x86_64.rpm 1.4 MB/s | 191 kB 00:00 2024-09-11T07:07:31.376 INFO:teuthology.orchestra.run.smithi071.stdout:(39/115): libgfortran-11.5.0-2.el9.x86_64.rpm 1.3 MB/s | 801 kB 00:00 2024-09-11T07:07:31.426 INFO:teuthology.orchestra.run.smithi071.stdout:(40/115): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.3 MB/s | 253 kB 00:00 2024-09-11T07:07:31.451 INFO:teuthology.orchestra.run.smithi071.stdout:(41/115): python3-ply-3.11-14.el9.noarch.rpm 1.4 MB/s | 106 kB 00:00 2024-09-11T07:07:31.493 INFO:teuthology.orchestra.run.smithi071.stdout:(42/115): python3-cryptography-36.0.1-4.el9.x86 8.1 MB/s | 1.2 MB 00:00 2024-09-11T07:07:31.527 INFO:teuthology.orchestra.run.smithi071.stdout:(43/115): python3-requests-2.25.1-8.el9.noarch. 1.6 MB/s | 125 kB 00:00 2024-09-11T07:07:31.553 INFO:teuthology.orchestra.run.smithi071.stdout:(44/115): python3-pycparser-2.20-6.el9.noarch.r 1.0 MB/s | 135 kB 00:00 2024-09-11T07:07:31.578 INFO:teuthology.orchestra.run.smithi071.stdout:(45/115): python3-urllib3-1.26.5-6.el9.noarch.r 2.5 MB/s | 215 kB 00:00 2024-09-11T07:07:32.095 INFO:teuthology.orchestra.run.smithi071.stdout:(46/115): flexiblas-openblas-openmp-3.0.4-8.el9 35 kB/s | 18 kB 00:00 2024-09-11T07:07:32.120 INFO:teuthology.orchestra.run.smithi071.stdout:(47/115): flexiblas-3.0.4-8.el9.x86_64.rpm 55 kB/s | 33 kB 00:00 2024-09-11T07:07:32.687 INFO:teuthology.orchestra.run.smithi071.stdout:(48/115): flexiblas-netlib-3.0.4-8.el9.x86_64.r 2.6 MB/s | 3.0 MB 00:01 2024-09-11T07:07:32.712 INFO:teuthology.orchestra.run.smithi071.stdout:(49/115): librabbitmq-0.11.0-7.el9.x86_64.rpm 76 kB/s | 45 kB 00:00 2024-09-11T07:07:32.804 INFO:teuthology.orchestra.run.smithi071.stdout:(50/115): libpmemobj-1.12.1-1.el9.x86_64.rpm 226 kB/s | 160 kB 00:00 2024-09-11T07:07:33.379 INFO:teuthology.orchestra.run.smithi071.stdout:(51/115): openblas-0.3.26-2.el9.x86_64.rpm 67 kB/s | 38 kB 00:00 2024-09-11T07:07:33.554 INFO:teuthology.orchestra.run.smithi071.stdout:(52/115): librdkafka-1.6.1-102.el9.x86_64.rpm 764 kB/s | 662 kB 00:00 2024-09-11T07:07:33.579 INFO:teuthology.orchestra.run.smithi071.stdout:(53/115): lttng-ust-2.12.0-6.el9.x86_64.rpm 337 kB/s | 292 kB 00:00 2024-09-11T07:07:34.263 INFO:teuthology.orchestra.run.smithi071.stdout:(54/115): python3-attrs-20.3.0-7.el9.noarch.rpm 123 kB/s | 87 kB 00:00 2024-09-11T07:07:34.538 INFO:teuthology.orchestra.run.smithi071.stdout:(55/115): openblas-openmp-0.3.26-2.el9.x86_64.r 4.3 MB/s | 4.9 MB 00:01 2024-09-11T07:07:34.797 INFO:teuthology.orchestra.run.smithi071.stdout:(56/115): python3-babel-2.9.1-2.el9.noarch.rpm 4.9 MB/s | 6.0 MB 00:01 2024-09-11T07:07:35.072 INFO:teuthology.orchestra.run.smithi071.stdout:(57/115): python3-iniconfig-1.1.1-7.el9.noarch. 33 kB/s | 17 kB 00:00 2024-09-11T07:07:35.114 INFO:teuthology.orchestra.run.smithi071.stdout:(58/115): python3-devel-3.9.19-8.el9.x86_64.rpm 288 kB/s | 245 kB 00:00 2024-09-11T07:07:35.581 INFO:teuthology.orchestra.run.smithi071.stdout:(59/115): python3-jinja2-2.11.3-6.el9.noarch.rp 320 kB/s | 248 kB 00:00 2024-09-11T07:07:35.598 INFO:teuthology.orchestra.run.smithi071.stdout:(60/115): python3-jsonpatch-1.21-16.el9.noarch. 49 kB/s | 26 kB 00:00 2024-09-11T07:07:35.614 INFO:teuthology.orchestra.run.smithi071.stdout:(61/115): python3-jsonpointer-2.0-4.el9.noarch. 38 kB/s | 19 kB 00:00 2024-09-11T07:07:35.902 INFO:teuthology.orchestra.run.smithi196.stdout: Preparing : 1/1 2024-09-11T07:07:35.959 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/117 2024-09-11T07:07:35.984 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/117 2024-09-11T07:07:36.044 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/117 2024-09-11T07:07:36.156 INFO:teuthology.orchestra.run.smithi071.stdout:(62/115): python3-markupsafe-1.1.1-12.el9.x86_6 62 kB/s | 35 kB 00:00 2024-09-11T07:07:36.298 INFO:teuthology.orchestra.run.smithi071.stdout:(63/115): python3-mako-1.1.4-6.el9.noarch.rpm 241 kB/s | 172 kB 00:00 2024-09-11T07:07:36.373 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 4/117 2024-09-11T07:07:36.477 INFO:teuthology.orchestra.run.smithi196.stdout: Upgrading : librados2-2:17.2.7-1623.gd0e6828a.el9.x86_64 5/117 2024-09-11T07:07:36.588 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: librados2-2:17.2.7-1623.gd0e6828a.el9.x86_64 5/117 2024-09-11T07:07:36.619 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libcephfs2-2:17.2.7-1623.gd0e6828a.el9.x86_64 6/117 2024-09-11T07:07:36.710 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: libcephfs2-2:17.2.7-1623.gd0e6828a.el9.x86_64 6/117 2024-09-11T07:07:36.748 INFO:teuthology.orchestra.run.smithi071.stdout:(64/115): python3-numpy-1.20.1-5.el9.x86_64.rpm 4.5 MB/s | 5.1 MB 00:01 2024-09-11T07:07:36.761 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-rados-2:17.2.7-1623.gd0e6828a.el9.x86_64 7/117 2024-09-11T07:07:36.791 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/117 2024-09-11T07:07:36.965 INFO:teuthology.orchestra.run.smithi071.stdout:(65/115): python3-numpy-f2py-1.20.1-5.el9.x86_6 310 kB/s | 250 kB 00:00 2024-09-11T07:07:37.165 INFO:teuthology.orchestra.run.smithi071.stdout:(66/115): python3-oauthlib-3.1.1-5.el9.noarch.r 256 kB/s | 222 kB 00:00 2024-09-11T07:07:37.230 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/117 2024-09-11T07:07:37.263 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : librgw2-2:17.2.7-1623.gd0e6828a.el9.x86_64 10/117 2024-09-11T07:07:37.335 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: librgw2-2:17.2.7-1623.gd0e6828a.el9.x86_64 10/117 2024-09-11T07:07:37.379 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-packaging-20.9-5.el9.noarch 11/117 2024-09-11T07:07:37.390 INFO:teuthology.orchestra.run.smithi071.stdout:(67/115): python3-packaging-20.9-5.el9.noarch.r 120 kB/s | 77 kB 00:00 2024-09-11T07:07:37.415 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libcephsqlite-2:17.2.7-1623.gd0e6828a.el9.x86_64 12/117 2024-09-11T07:07:37.515 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: libcephsqlite-2:17.2.7-1623.gd0e6828a.el9.x86_64 12/117 2024-09-11T07:07:37.549 INFO:teuthology.orchestra.run.smithi071.stdout:(68/115): python3-pluggy-0.13.1-7.el9.noarch.rp 72 kB/s | 41 kB 00:00 2024-09-11T07:07:37.550 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libradosstriper1-2:17.2.7-1623.gd0e6828a.el9.x86 13/117 2024-09-11T07:07:37.692 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: libradosstriper1-2:17.2.7-1623.gd0e6828a.el9.x86 13/117 2024-09-11T07:07:37.746 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 14/117 2024-09-11T07:07:37.766 INFO:teuthology.orchestra.run.smithi071.stdout:(69/115): python3-prettytable-0.7.2-27.el9.noar 70 kB/s | 42 kB 00:00 2024-09-11T07:07:37.777 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : liboath-2.6.7-2.el9.x86_64 15/117 2024-09-11T07:07:37.807 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 16/117 2024-09-11T07:07:37.848 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 17/117 2024-09-11T07:07:37.920 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 18/117 2024-09-11T07:07:37.965 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 19/117 2024-09-11T07:07:38.000 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 20/117 2024-09-11T07:07:38.069 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 21/117 2024-09-11T07:07:38.122 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 22/117 2024-09-11T07:07:38.162 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 23/117 2024-09-11T07:07:38.205 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 24/117 2024-09-11T07:07:38.280 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 25/117 2024-09-11T07:07:38.283 INFO:teuthology.orchestra.run.smithi071.stdout:(70/115): python3-py-1.10.0-6.el9.noarch.rpm 539 kB/s | 477 kB 00:00 2024-09-11T07:07:38.316 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-ceph-common-2:17.2.7-1623.gd0e6828a.el9. 26/117 2024-09-11T07:07:38.317 INFO:teuthology.orchestra.run.smithi071.stdout:(71/115): python3-pyasn1-0.4.8-6.el9.noarch.rpm 207 kB/s | 159 kB 00:00 2024-09-11T07:07:38.359 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-ceph-argparse-2:17.2.7-1623.gd0e6828a.el 27/117 2024-09-11T07:07:38.434 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-cephfs-2:17.2.7-1623.gd0e6828a.el9.x86_6 28/117 2024-09-11T07:07:38.542 INFO:teuthology.orchestra.run.smithi071.stdout:(72/115): python3-pyasn1-modules-0.4.8-6.el9.no 360 kB/s | 279 kB 00:00 2024-09-11T07:07:38.575 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 29/117 2024-09-11T07:07:38.637 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 30/117 2024-09-11T07:07:38.934 INFO:teuthology.orchestra.run.smithi071.stdout:(73/115): python3-pytz-2021.1-5.el9.noarch.rpm 83 kB/s | 51 kB 00:00 2024-09-11T07:07:39.168 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-rsa-4.9-2.el9.noarch 31/117 2024-09-11T07:07:39.200 INFO:teuthology.orchestra.run.smithi071.stdout:(74/115): python3-pytest-6.2.2-7.el9.noarch.rpm 566 kB/s | 519 kB 00:00 2024-09-11T07:07:39.225 INFO:teuthology.orchestra.run.smithi071.stdout:(75/115): python3-requests-oauthlib-1.3.0-12.el 79 kB/s | 54 kB 00:00 2024-09-11T07:07:39.442 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 32/117 2024-09-11T07:07:39.495 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 33/117 2024-09-11T07:07:39.529 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 34/117 2024-09-11T07:07:39.564 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 35/117 2024-09-11T07:07:39.603 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 36/117 2024-09-11T07:07:39.639 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-rgw-2:17.2.7-1623.gd0e6828a.el9.x86_64 37/117 2024-09-11T07:07:39.677 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : librados-devel-2:17.2.7-1623.gd0e6828a.el9.x86_6 38/117 2024-09-11T07:07:39.708 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 39/117 2024-09-11T07:07:39.744 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 40/117 2024-09-11T07:07:39.767 INFO:teuthology.orchestra.run.smithi071.stdout:(76/115): python3-toml-0.10.2-6.el9.noarch.rpm 74 kB/s | 42 kB 00:00 2024-09-11T07:07:39.800 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 41/117 2024-09-11T07:07:39.873 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 42/117 2024-09-11T07:07:39.912 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 43/117 2024-09-11T07:07:39.948 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 44/117 2024-09-11T07:07:39.995 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 45/117 2024-09-11T07:07:40.041 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 46/117 2024-09-11T07:07:40.083 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 47/117 2024-09-11T07:07:40.185 INFO:teuthology.orchestra.run.smithi071.stdout:(77/115): socat-1.7.4.1-6.el9.x86_64.rpm 317 kB/s | 304 kB 00:00 2024-09-11T07:07:40.220 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 48/117 2024-09-11T07:07:40.243 INFO:teuthology.orchestra.run.smithi071.stdout:(78/115): gperftools-libs-2.9.1-2.el9.x86_64.rp 5.2 MB/s | 309 kB 00:00 2024-09-11T07:07:40.260 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 49/117 2024-09-11T07:07:40.268 INFO:teuthology.orchestra.run.smithi071.stdout:(79/115): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-11T07:07:40.294 INFO:teuthology.orchestra.run.smithi071.stdout:(80/115): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-09-11T07:07:40.301 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 50/117 2024-09-11T07:07:40.327 INFO:teuthology.orchestra.run.smithi071.stdout:(81/115): python3-asyncssh-2.13.2-5.el9.noarch. 16 MB/s | 548 kB 00:00 2024-09-11T07:07:40.352 INFO:teuthology.orchestra.run.smithi071.stdout:(82/115): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 1.7 MB/s | 43 kB 00:00 2024-09-11T07:07:40.378 INFO:teuthology.orchestra.run.smithi071.stdout:(83/115): python3-cachetools-4.2.4-1.el9.noarch 1.3 MB/s | 32 kB 00:00 2024-09-11T07:07:40.403 INFO:teuthology.orchestra.run.smithi071.stdout:(84/115): xmlstarlet-1.6.1-20.el9.x86_64.rpm 100 kB/s | 64 kB 00:00 2024-09-11T07:07:40.403 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 51/117 2024-09-11T07:07:40.428 INFO:teuthology.orchestra.run.smithi071.stdout:(85/115): python3-certifi-2023.05.07-4.el9.noar 289 kB/s | 14 kB 00:00 2024-09-11T07:07:40.443 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 52/117 2024-09-11T07:07:40.470 INFO:teuthology.orchestra.run.smithi071.stdout:(86/115): python3-cherrypy-18.6.1-2.el9.noarch. 10 MB/s | 358 kB 00:00 2024-09-11T07:07:40.492 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 53/117 2024-09-11T07:07:40.504 INFO:teuthology.orchestra.run.smithi071.stdout:(87/115): python3-google-auth-2.34.0-1.el9.noar 6.8 MB/s | 220 kB 00:00 2024-09-11T07:07:40.521 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 54/117 2024-09-11T07:07:40.529 INFO:teuthology.orchestra.run.smithi071.stdout:(88/115): python3-cheroot-8.6.0-4.el9.noarch.rp 1.3 MB/s | 172 kB 00:00 2024-09-11T07:07:40.554 INFO:teuthology.orchestra.run.smithi071.stdout:(89/115): python3-jaraco-8.2.1-3.el9.noarch.rpm 213 kB/s | 11 kB 00:00 2024-09-11T07:07:40.565 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 55/117 2024-09-11T07:07:40.579 INFO:teuthology.orchestra.run.smithi071.stdout:(90/115): python3-jaraco-classes-3.2.1-5.el9.no 353 kB/s | 18 kB 00:00 2024-09-11T07:07:40.605 INFO:teuthology.orchestra.run.smithi071.stdout:(91/115): python3-jaraco-collections-3.0.0-8.el 463 kB/s | 23 kB 00:00 2024-09-11T07:07:40.630 INFO:teuthology.orchestra.run.smithi071.stdout:(92/115): python3-jaraco-functools-3.5.0-2.el9. 387 kB/s | 19 kB 00:00 2024-09-11T07:07:40.655 INFO:teuthology.orchestra.run.smithi071.stdout:(93/115): python3-jaraco-text-3.2.0-6.el9.noarc 392 kB/s | 20 kB 00:00 2024-09-11T07:07:40.681 INFO:teuthology.orchestra.run.smithi071.stdout:(94/115): python3-jwt+crypto-2.4.0-1.el9.noarch 181 kB/s | 9.0 kB 00:00 2024-09-11T07:07:40.690 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 56/117 2024-09-11T07:07:40.706 INFO:teuthology.orchestra.run.smithi071.stdout:(95/115): python3-jwt-2.4.0-1.el9.noarch.rpm 814 kB/s | 41 kB 00:00 2024-09-11T07:07:40.731 INFO:teuthology.orchestra.run.smithi071.stdout:(96/115): python3-logutils-0.3.5-21.el9.noarch. 1.8 MB/s | 46 kB 00:00 2024-09-11T07:07:40.732 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-py-1.10.0-6.el9.noarch 57/117 2024-09-11T07:07:40.756 INFO:teuthology.orchestra.run.smithi071.stdout:(97/115): python3-more-itertools-8.12.0-2.el9.n 3.1 MB/s | 79 kB 00:00 2024-09-11T07:07:40.771 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 58/117 2024-09-11T07:07:40.782 INFO:teuthology.orchestra.run.smithi071.stdout:(98/115): python3-natsort-7.1.1-5.el9.noarch.rp 2.3 MB/s | 58 kB 00:00 2024-09-11T07:07:40.815 INFO:teuthology.orchestra.run.smithi071.stdout:(99/115): python3-pecan-1.4.2-3.el9.noarch.rpm 8.0 MB/s | 272 kB 00:00 2024-09-11T07:07:40.840 INFO:teuthology.orchestra.run.smithi071.stdout:(100/115): python3-portend-3.1.0-2.el9.noarch.r 663 kB/s | 16 kB 00:00 2024-09-11T07:07:40.866 INFO:teuthology.orchestra.run.smithi071.stdout:(101/115): python3-pyOpenSSL-21.0.0-1.el9.noarc 3.5 MB/s | 90 kB 00:00 2024-09-11T07:07:40.879 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-pluggy-0.13.1-7.el9.noarch 59/117 2024-09-11T07:07:40.891 INFO:teuthology.orchestra.run.smithi071.stdout:(102/115): python3-repoze-lru-0.7-16.el9.noarch 1.2 MB/s | 31 kB 00:00 2024-09-11T07:07:40.925 INFO:teuthology.orchestra.run.smithi071.stdout:(103/115): python3-kubernetes-26.1.0-2.el9.noar 4.2 MB/s | 1.0 MB 00:00 2024-09-11T07:07:40.942 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 60/117 2024-09-11T07:07:40.950 INFO:teuthology.orchestra.run.smithi071.stdout:(104/115): python3-routes-2.5.1-5.el9.noarch.rp 3.1 MB/s | 188 kB 00:00 2024-09-11T07:07:40.975 INFO:teuthology.orchestra.run.smithi071.stdout:(105/115): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-09-11T07:07:41.000 INFO:teuthology.orchestra.run.smithi071.stdout:(106/115): python3-tempora-5.0.0-2.el9.noarch.r 716 kB/s | 36 kB 00:00 2024-09-11T07:07:41.026 INFO:teuthology.orchestra.run.smithi071.stdout:(107/115): python3-typing-extensions-4.12.2-2.e 1.4 MB/s | 74 kB 00:00 2024-09-11T07:07:41.051 INFO:teuthology.orchestra.run.smithi071.stdout:(108/115): python3-websocket-client-1.2.3-2.el9 3.5 MB/s | 90 kB 00:00 2024-09-11T07:07:41.077 INFO:teuthology.orchestra.run.smithi071.stdout:(109/115): python3-webob-1.8.8-2.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-09-11T07:07:41.102 INFO:teuthology.orchestra.run.smithi071.stdout:(110/115): python3-zc-lockfile-2.0-10.el9.noarc 805 kB/s | 20 kB 00:00 2024-09-11T07:07:41.135 INFO:teuthology.orchestra.run.smithi071.stdout:(111/115): python3-werkzeug-2.0.3-3.el9.1.noarc 5.0 MB/s | 427 kB 00:00 2024-09-11T07:07:41.211 INFO:teuthology.orchestra.run.smithi071.stdout:(112/115): thrift-0.15.0-2.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-09-11T07:07:41.370 INFO:teuthology.orchestra.run.smithi071.stdout:(113/115): python3-scipy-1.6.2-8.el9.x86_64.rpm 6.7 MB/s | 16 MB 00:02 2024-09-11T07:07:41.496 INFO:teuthology.orchestra.run.smithi071.stdout:(114/115): librados2-17.2.7-1623.gd0e6828a.el9. 8.6 MB/s | 3.1 MB 00:00 2024-09-11T07:07:41.562 INFO:teuthology.orchestra.run.smithi071.stdout:(115/115): librbd1-17.2.7-1623.gd0e6828a.el9.x8 8.4 MB/s | 2.9 MB 00:00 2024-09-11T07:07:41.567 INFO:teuthology.orchestra.run.smithi071.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:07:41.567 INFO:teuthology.orchestra.run.smithi071.stdout:Total 13 MB/s | 191 MB 00:14 2024-09-11T07:07:41.669 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 61/117 2024-09-11T07:07:41.726 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 62/117 2024-09-11T07:07:41.768 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 63/117 2024-09-11T07:07:41.801 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 64/117 2024-09-11T07:07:41.902 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-iniconfig-1.1.1-7.el9.noarch 65/117 2024-09-11T07:07:41.951 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 66/117 2024-09-11T07:07:41.986 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 67/117 2024-09-11T07:07:42.168 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : openblas-0.3.26-2.el9.x86_64 68/117 2024-09-11T07:07:42.201 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 69/117 2024-09-11T07:07:42.285 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction check 2024-09-11T07:07:42.371 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 70/117 2024-09-11T07:07:42.382 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction check succeeded. 2024-09-11T07:07:42.383 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction test 2024-09-11T07:07:43.168 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 71/117 2024-09-11T07:07:43.283 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 72/117 2024-09-11T07:07:44.062 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction test succeeded. 2024-09-11T07:07:44.062 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction 2024-09-11T07:07:44.581 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 73/117 2024-09-11T07:07:44.678 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 74/117 2024-09-11T07:07:44.969 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 75/117 2024-09-11T07:07:45.006 INFO:teuthology.orchestra.run.smithi196.stdout: Upgrading : librbd1-2:17.2.7-1623.gd0e6828a.el9.x86_64 76/117 2024-09-11T07:07:45.080 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: librbd1-2:17.2.7-1623.gd0e6828a.el9.x86_64 76/117 2024-09-11T07:07:45.124 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-rbd-2:17.2.7-1623.gd0e6828a.el9.x86_64 77/117 2024-09-11T07:07:47.588 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-common-2:17.2.7-1623.gd0e6828a.el9.x86_64 78/117 2024-09-11T07:07:47.631 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-common-2:17.2.7-1623.gd0e6828a.el9.x86_64 78/117 2024-09-11T07:07:48.213 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-common-2:17.2.7-1623.gd0e6828a.el9.x86_64 78/117 2024-09-11T07:07:48.251 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-base-2:17.2.7-1623.gd0e6828a.el9.x86_64 79/117 2024-09-11T07:07:48.373 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-base-2:17.2.7-1623.gd0e6828a.el9.x86_64 79/117 2024-09-11T07:07:48.373 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-11T07:07:48.373 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-11T07:07:48.373 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-11T07:07:48.418 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-selinux-2:17.2.7-1623.gd0e6828a.el9.x86_64 80/117 2024-09-11T07:07:48.661 INFO:teuthology.orchestra.run.smithi071.stdout: Preparing : 1/1 2024-09-11T07:07:48.746 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/117 2024-09-11T07:07:48.823 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/117 2024-09-11T07:07:49.035 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/117 2024-09-11T07:07:49.344 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 4/117 2024-09-11T07:07:49.439 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : librados2-2:17.2.7-1623.gd0e6828a.el9.x86_64 5/117 2024-09-11T07:07:49.536 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: librados2-2:17.2.7-1623.gd0e6828a.el9.x86_64 5/117 2024-09-11T07:07:49.574 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libcephfs2-2:17.2.7-1623.gd0e6828a.el9.x86_64 6/117 2024-09-11T07:07:49.670 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: libcephfs2-2:17.2.7-1623.gd0e6828a.el9.x86_64 6/117 2024-09-11T07:07:49.717 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-rados-2:17.2.7-1623.gd0e6828a.el9.x86_64 7/117 2024-09-11T07:07:49.745 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/117 2024-09-11T07:07:50.188 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/117 2024-09-11T07:07:50.218 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : librgw2-2:17.2.7-1623.gd0e6828a.el9.x86_64 10/117 2024-09-11T07:07:50.298 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: librgw2-2:17.2.7-1623.gd0e6828a.el9.x86_64 10/117 2024-09-11T07:07:50.342 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-packaging-20.9-5.el9.noarch 11/117 2024-09-11T07:07:50.370 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libcephsqlite-2:17.2.7-1623.gd0e6828a.el9.x86_64 12/117 2024-09-11T07:07:50.453 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: libcephsqlite-2:17.2.7-1623.gd0e6828a.el9.x86_64 12/117 2024-09-11T07:07:50.479 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libradosstriper1-2:17.2.7-1623.gd0e6828a.el9.x86 13/117 2024-09-11T07:07:50.605 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: libradosstriper1-2:17.2.7-1623.gd0e6828a.el9.x86 13/117 2024-09-11T07:07:50.642 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 14/117 2024-09-11T07:07:50.673 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : liboath-2.6.7-2.el9.x86_64 15/117 2024-09-11T07:07:50.703 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 16/117 2024-09-11T07:07:50.744 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 17/117 2024-09-11T07:07:50.808 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 18/117 2024-09-11T07:07:50.843 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 19/117 2024-09-11T07:07:50.879 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 20/117 2024-09-11T07:07:50.956 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 21/117 2024-09-11T07:07:51.026 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 22/117 2024-09-11T07:07:51.067 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 23/117 2024-09-11T07:07:51.109 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 24/117 2024-09-11T07:07:51.184 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 25/117 2024-09-11T07:07:51.219 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-ceph-common-2:17.2.7-1623.gd0e6828a.el9. 26/117 2024-09-11T07:07:51.262 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-ceph-argparse-2:17.2.7-1623.gd0e6828a.el 27/117 2024-09-11T07:07:51.345 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-cephfs-2:17.2.7-1623.gd0e6828a.el9.x86_6 28/117 2024-09-11T07:07:51.533 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 29/117 2024-09-11T07:07:51.589 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 30/117 2024-09-11T07:07:52.121 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-rsa-4.9-2.el9.noarch 31/117 2024-09-11T07:07:52.193 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 32/117 2024-09-11T07:07:52.232 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 33/117 2024-09-11T07:07:52.266 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 34/117 2024-09-11T07:07:52.300 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 35/117 2024-09-11T07:07:52.339 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 36/117 2024-09-11T07:07:52.385 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-rgw-2:17.2.7-1623.gd0e6828a.el9.x86_64 37/117 2024-09-11T07:07:52.421 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : librados-devel-2:17.2.7-1623.gd0e6828a.el9.x86_6 38/117 2024-09-11T07:07:52.452 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 39/117 2024-09-11T07:07:52.488 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 40/117 2024-09-11T07:07:52.543 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 41/117 2024-09-11T07:07:52.616 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 42/117 2024-09-11T07:07:52.648 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 43/117 2024-09-11T07:07:52.684 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 44/117 2024-09-11T07:07:52.733 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 45/117 2024-09-11T07:07:52.785 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 46/117 2024-09-11T07:07:52.826 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 47/117 2024-09-11T07:07:52.961 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 48/117 2024-09-11T07:07:53.012 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 49/117 2024-09-11T07:07:53.053 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 50/117 2024-09-11T07:07:53.164 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 51/117 2024-09-11T07:07:53.212 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 52/117 2024-09-11T07:07:53.253 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 53/117 2024-09-11T07:07:53.289 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 54/117 2024-09-11T07:07:53.334 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 55/117 2024-09-11T07:07:53.464 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 56/117 2024-09-11T07:07:53.500 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-py-1.10.0-6.el9.noarch 57/117 2024-09-11T07:07:53.539 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 58/117 2024-09-11T07:07:53.637 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-pluggy-0.13.1-7.el9.noarch 59/117 2024-09-11T07:07:53.687 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 60/117 2024-09-11T07:07:54.411 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 61/117 2024-09-11T07:07:54.510 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 62/117 2024-09-11T07:07:54.543 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 63/117 2024-09-11T07:07:54.577 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 64/117 2024-09-11T07:07:54.677 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-iniconfig-1.1.1-7.el9.noarch 65/117 2024-09-11T07:07:54.727 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 66/117 2024-09-11T07:07:54.762 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 67/117 2024-09-11T07:07:54.945 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : openblas-0.3.26-2.el9.x86_64 68/117 2024-09-11T07:07:54.977 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 69/117 2024-09-11T07:07:55.054 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 70/117 2024-09-11T07:07:55.718 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 71/117 2024-09-11T07:07:55.841 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 72/117 2024-09-11T07:07:57.123 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 73/117 2024-09-11T07:07:57.231 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 74/117 2024-09-11T07:07:57.531 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 75/117 2024-09-11T07:07:57.567 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : librbd1-2:17.2.7-1623.gd0e6828a.el9.x86_64 76/117 2024-09-11T07:07:57.658 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: librbd1-2:17.2.7-1623.gd0e6828a.el9.x86_64 76/117 2024-09-11T07:07:57.686 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-rbd-2:17.2.7-1623.gd0e6828a.el9.x86_64 77/117 2024-09-11T07:07:59.761 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-selinux-2:17.2.7-1623.gd0e6828a.el9.x86_64 80/117 2024-09-11T07:07:59.762 INFO:teuthology.orchestra.run.smithi196.stdout:skipping the directory /sys 2024-09-11T07:07:59.762 INFO:teuthology.orchestra.run.smithi196.stdout:skipping the directory /proc 2024-09-11T07:07:59.762 INFO:teuthology.orchestra.run.smithi196.stdout:skipping the directory /mnt 2024-09-11T07:07:59.762 INFO:teuthology.orchestra.run.smithi196.stdout:skipping the directory /var/tmp 2024-09-11T07:07:59.762 INFO:teuthology.orchestra.run.smithi196.stdout:skipping the directory /home 2024-09-11T07:07:59.762 INFO:teuthology.orchestra.run.smithi196.stdout:skipping the directory /root 2024-09-11T07:07:59.762 INFO:teuthology.orchestra.run.smithi196.stdout:skipping the directory /tmp 2024-09-11T07:07:59.762 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-11T07:07:59.928 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-volume-2:17.2.7-1623.gd0e6828a.el9.noarch 81/117 2024-09-11T07:07:59.944 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-volume-2:17.2.7-1623.gd0e6828a.el9.noarch 81/117 2024-09-11T07:07:59.944 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T07:07:59.944 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-11T07:07:59.944 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-11T07:08:00.093 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-common-2:17.2.7-1623.gd0e6828a.el9.x86_64 78/117 2024-09-11T07:08:00.124 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-common-2:17.2.7-1623.gd0e6828a.el9.x86_64 78/117 2024-09-11T07:08:00.638 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-common-2:17.2.7-1623.gd0e6828a.el9.x86_64 78/117 2024-09-11T07:08:00.686 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-base-2:17.2.7-1623.gd0e6828a.el9.x86_64 79/117 2024-09-11T07:08:00.789 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-base-2:17.2.7-1623.gd0e6828a.el9.x86_64 79/117 2024-09-11T07:08:00.790 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-11T07:08:00.790 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-11T07:08:00.790 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T07:08:00.836 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-selinux-2:17.2.7-1623.gd0e6828a.el9.x86_64 80/117 2024-09-11T07:08:01.307 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-osd-2:17.2.7-1623.gd0e6828a.el9.x86_64 82/117 2024-09-11T07:08:01.334 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-osd-2:17.2.7-1623.gd0e6828a.el9.x86_64 82/117 2024-09-11T07:08:01.334 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T07:08:01.334 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-11T07:08:01.334 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-11T07:08:01.334 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-11T07:08:01.334 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-11T07:08:01.540 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mds-2:17.2.7-1623.gd0e6828a.el9.x86_64 83/117 2024-09-11T07:08:01.564 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-mds-2:17.2.7-1623.gd0e6828a.el9.x86_64 83/117 2024-09-11T07:08:01.564 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T07:08:01.564 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-11T07:08:01.564 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-11T07:08:01.564 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-11T07:08:01.564 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-11T07:08:01.933 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mon-2:17.2.7-1623.gd0e6828a.el9.x86_64 84/117 2024-09-11T07:08:01.957 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-mon-2:17.2.7-1623.gd0e6828a.el9.x86_64 84/117 2024-09-11T07:08:01.957 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T07:08:01.957 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-11T07:08:01.957 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-11T07:08:01.957 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-11T07:08:01.957 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-11T07:08:02.019 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-ply-3.11-14.el9.noarch 85/117 2024-09-11T07:08:02.075 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 86/117 2024-09-11T07:08:02.267 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 87/117 2024-09-11T07:08:02.315 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 88/117 2024-09-11T07:08:02.385 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 89/117 2024-09-11T07:08:02.474 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 90/117 2024-09-11T07:08:02.603 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 91/117 2024-09-11T07:08:02.654 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 92/117 2024-09-11T07:08:02.683 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 93/117 2024-09-11T07:08:02.725 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 94/117 2024-09-11T07:08:02.757 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 95/117 2024-09-11T07:08:02.785 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : fuse-2.9.9-16.el9.x86_64 96/117 2024-09-11T07:08:03.185 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: cephadm-2:17.2.7-1623.gd0e6828a.el9.noarch 97/117 2024-09-11T07:08:03.217 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : cephadm-2:17.2.7-1623.gd0e6828a.el9.noarch 97/117 2024-09-11T07:08:03.245 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-prometheus-alerts-2:17.2.7-1623.gd0e6828a.e 98/117 2024-09-11T07:08:03.302 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-grafana-dashboards-2:17.2.7-1623.gd0e6828a. 99/117 2024-09-11T07:08:03.332 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mgr-cephadm-2:17.2.7-1623.gd0e6828a.el9.noa 100/117 2024-09-11T07:08:04.310 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.7-1623.gd0e6828a.el9.noa 100/117 2024-09-11T07:08:04.342 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mgr-dashboard-2:17.2.7-1623.gd0e6828a.el9.n 101/117 2024-09-11T07:08:05.298 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.7-1623.gd0e6828a.el9.n 101/117 2024-09-11T07:08:05.325 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.7-1623.gd0e 102/117 2024-09-11T07:08:05.425 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.7-1623.gd0e 102/117 2024-09-11T07:08:05.580 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mgr-modules-core-2:17.2.7-1623.gd0e6828a.el 103/117 2024-09-11T07:08:05.611 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mgr-2:17.2.7-1623.gd0e6828a.el9.x86_64 104/117 2024-09-11T07:08:05.635 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-mgr-2:17.2.7-1623.gd0e6828a.el9.x86_64 104/117 2024-09-11T07:08:05.635 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T07:08:05.635 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-11T07:08:05.635 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-11T07:08:05.635 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-11T07:08:05.636 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-11T07:08:05.678 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-mgr-rook-2:17.2.7-1623.gd0e6828a.el9.noarch 105/117 2024-09-11T07:08:05.690 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.7-1623.gd0e6828a.el9.noarch 105/117 2024-09-11T07:08:05.781 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-2:17.2.7-1623.gd0e6828a.el9.x86_64 106/117 2024-09-11T07:08:07.037 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-fuse-2:17.2.7-1623.gd0e6828a.el9.x86_64 107/117 2024-09-11T07:08:07.063 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-radosgw-2:17.2.7-1623.gd0e6828a.el9.x86_64 108/117 2024-09-11T07:08:07.142 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-radosgw-2:17.2.7-1623.gd0e6828a.el9.x86_64 108/117 2024-09-11T07:08:07.142 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T07:08:07.142 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-11T07:08:07.142 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-11T07:08:07.142 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-11T07:08:07.142 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-11T07:08:07.182 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-immutable-object-cache-2:17.2.7-1623.gd0e68 109/117 2024-09-11T07:08:07.205 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.7-1623.gd0e68 109/117 2024-09-11T07:08:07.205 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T07:08:07.205 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-11T07:08:07.205 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-11T07:08:07.490 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : rbd-mirror-2:17.2.7-1623.gd0e6828a.el9.x86_64 110/117 2024-09-11T07:08:07.513 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: rbd-mirror-2:17.2.7-1623.gd0e6828a.el9.x86_64 110/117 2024-09-11T07:08:07.514 INFO:teuthology.orchestra.run.smithi196.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T07:08:07.514 INFO:teuthology.orchestra.run.smithi196.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-11T07:08:07.514 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-11T07:08:07.514 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-11T07:08:07.514 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-11T07:08:11.901 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : ceph-test-2:17.2.7-1623.gd0e6828a.el9.x86_64 111/117 2024-09-11T07:08:11.959 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : rbd-fuse-2:17.2.7-1623.gd0e6828a.el9.x86_64 112/117 2024-09-11T07:08:12.012 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-selinux-2:17.2.7-1623.gd0e6828a.el9.x86_64 80/117 2024-09-11T07:08:12.012 INFO:teuthology.orchestra.run.smithi071.stdout:skipping the directory /sys 2024-09-11T07:08:12.012 INFO:teuthology.orchestra.run.smithi071.stdout:skipping the directory /proc 2024-09-11T07:08:12.012 INFO:teuthology.orchestra.run.smithi071.stdout:skipping the directory /mnt 2024-09-11T07:08:12.012 INFO:teuthology.orchestra.run.smithi071.stdout:skipping the directory /var/tmp 2024-09-11T07:08:12.012 INFO:teuthology.orchestra.run.smithi071.stdout:skipping the directory /home 2024-09-11T07:08:12.013 INFO:teuthology.orchestra.run.smithi071.stdout:skipping the directory /root 2024-09-11T07:08:12.013 INFO:teuthology.orchestra.run.smithi071.stdout:skipping the directory /tmp 2024-09-11T07:08:12.013 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T07:08:12.078 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : rbd-nbd-2:17.2.7-1623.gd0e6828a.el9.x86_64 113/117 2024-09-11T07:08:12.125 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : python3-pytest-6.2.2-7.el9.noarch 114/117 2024-09-11T07:08:12.148 INFO:teuthology.orchestra.run.smithi196.stdout: Installing : libcephfs-devel-2:17.2.7-1623.gd0e6828a.el9.x86_ 115/117 2024-09-11T07:08:12.148 INFO:teuthology.orchestra.run.smithi196.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 116/117 2024-09-11T07:08:12.179 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-volume-2:17.2.7-1623.gd0e6828a.el9.noarch 81/117 2024-09-11T07:08:12.195 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-volume-2:17.2.7-1623.gd0e6828a.el9.noarch 81/117 2024-09-11T07:08:12.195 INFO:teuthology.orchestra.run.smithi071.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T07:08:12.195 INFO:teuthology.orchestra.run.smithi071.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-11T07:08:12.195 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T07:08:12.232 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 116/117 2024-09-11T07:08:12.232 INFO:teuthology.orchestra.run.smithi196.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 117/117 2024-09-11T07:08:13.549 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-osd-2:17.2.7-1623.gd0e6828a.el9.x86_64 82/117 2024-09-11T07:08:13.575 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-osd-2:17.2.7-1623.gd0e6828a.el9.x86_64 82/117 2024-09-11T07:08:13.575 INFO:teuthology.orchestra.run.smithi071.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T07:08:13.575 INFO:teuthology.orchestra.run.smithi071.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-11T07:08:13.575 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-11T07:08:13.575 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-11T07:08:13.575 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T07:08:13.783 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-mds-2:17.2.7-1623.gd0e6828a.el9.x86_64 83/117 2024-09-11T07:08:13.805 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-mds-2:17.2.7-1623.gd0e6828a.el9.x86_64 83/117 2024-09-11T07:08:13.805 INFO:teuthology.orchestra.run.smithi071.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T07:08:13.806 INFO:teuthology.orchestra.run.smithi071.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-11T07:08:13.806 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-11T07:08:13.806 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-11T07:08:13.806 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T07:08:14.175 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-mon-2:17.2.7-1623.gd0e6828a.el9.x86_64 84/117 2024-09-11T07:08:14.197 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-mon-2:17.2.7-1623.gd0e6828a.el9.x86_64 84/117 2024-09-11T07:08:14.198 INFO:teuthology.orchestra.run.smithi071.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T07:08:14.198 INFO:teuthology.orchestra.run.smithi071.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-11T07:08:14.198 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-11T07:08:14.198 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-11T07:08:14.198 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T07:08:14.259 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-ply-3.11-14.el9.noarch 85/117 2024-09-11T07:08:14.315 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 86/117 2024-09-11T07:08:14.507 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 87/117 2024-09-11T07:08:14.557 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 88/117 2024-09-11T07:08:14.626 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 89/117 2024-09-11T07:08:14.715 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 90/117 2024-09-11T07:08:14.852 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 91/117 2024-09-11T07:08:14.903 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 92/117 2024-09-11T07:08:14.933 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 93/117 2024-09-11T07:08:14.966 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 94/117 2024-09-11T07:08:15.006 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 95/117 2024-09-11T07:08:15.034 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : fuse-2.9.9-16.el9.x86_64 96/117 2024-09-11T07:08:15.468 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: cephadm-2:17.2.7-1623.gd0e6828a.el9.noarch 97/117 2024-09-11T07:08:15.505 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : cephadm-2:17.2.7-1623.gd0e6828a.el9.noarch 97/117 2024-09-11T07:08:15.536 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-prometheus-alerts-2:17.2.7-1623.gd0e6828a.e 98/117 2024-09-11T07:08:15.593 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-grafana-dashboards-2:17.2.7-1623.gd0e6828a. 99/117 2024-09-11T07:08:15.631 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-mgr-cephadm-2:17.2.7-1623.gd0e6828a.el9.noa 100/117 2024-09-11T07:08:16.599 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.7-1623.gd0e6828a.el9.noa 100/117 2024-09-11T07:08:16.632 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-mgr-dashboard-2:17.2.7-1623.gd0e6828a.el9.n 101/117 2024-09-11T07:08:17.367 INFO:teuthology.orchestra.run.smithi196.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 117/117 2024-09-11T07:08:17.367 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/117 2024-09-11T07:08:17.367 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-base-2:17.2.7-1623.gd0e6828a.el9.x86_64 2/117 2024-09-11T07:08:17.367 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-common-2:17.2.7-1623.gd0e6828a.el9.x86_64 3/117 2024-09-11T07:08:17.367 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-fuse-2:17.2.7-1623.gd0e6828a.el9.x86_64 4/117 2024-09-11T07:08:17.367 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-immutable-object-cache-2:17.2.7-1623.gd0e68 5/117 2024-09-11T07:08:17.367 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mds-2:17.2.7-1623.gd0e6828a.el9.x86_64 6/117 2024-09-11T07:08:17.367 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mgr-2:17.2.7-1623.gd0e6828a.el9.x86_64 7/117 2024-09-11T07:08:17.368 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mon-2:17.2.7-1623.gd0e6828a.el9.x86_64 8/117 2024-09-11T07:08:17.368 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-osd-2:17.2.7-1623.gd0e6828a.el9.x86_64 9/117 2024-09-11T07:08:17.368 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-radosgw-2:17.2.7-1623.gd0e6828a.el9.x86_64 10/117 2024-09-11T07:08:17.368 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-selinux-2:17.2.7-1623.gd0e6828a.el9.x86_64 11/117 2024-09-11T07:08:17.368 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-test-2:17.2.7-1623.gd0e6828a.el9.x86_64 12/117 2024-09-11T07:08:17.378 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libcephfs-devel-2:17.2.7-1623.gd0e6828a.el9.x86_ 13/117 2024-09-11T07:08:17.378 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libcephfs2-2:17.2.7-1623.gd0e6828a.el9.x86_64 14/117 2024-09-11T07:08:17.378 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libcephsqlite-2:17.2.7-1623.gd0e6828a.el9.x86_64 15/117 2024-09-11T07:08:17.378 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librados-devel-2:17.2.7-1623.gd0e6828a.el9.x86_6 16/117 2024-09-11T07:08:17.379 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libradosstriper1-2:17.2.7-1623.gd0e6828a.el9.x86 17/117 2024-09-11T07:08:17.379 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librgw2-2:17.2.7-1623.gd0e6828a.el9.x86_64 18/117 2024-09-11T07:08:17.379 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-ceph-argparse-2:17.2.7-1623.gd0e6828a.el 19/117 2024-09-11T07:08:17.379 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-ceph-common-2:17.2.7-1623.gd0e6828a.el9. 20/117 2024-09-11T07:08:17.379 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-cephfs-2:17.2.7-1623.gd0e6828a.el9.x86_6 21/117 2024-09-11T07:08:17.379 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-rados-2:17.2.7-1623.gd0e6828a.el9.x86_64 22/117 2024-09-11T07:08:17.379 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-rbd-2:17.2.7-1623.gd0e6828a.el9.x86_64 23/117 2024-09-11T07:08:17.379 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-rgw-2:17.2.7-1623.gd0e6828a.el9.x86_64 24/117 2024-09-11T07:08:17.379 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : rbd-fuse-2:17.2.7-1623.gd0e6828a.el9.x86_64 25/117 2024-09-11T07:08:17.379 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : rbd-mirror-2:17.2.7-1623.gd0e6828a.el9.x86_64 26/117 2024-09-11T07:08:17.379 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : rbd-nbd-2:17.2.7-1623.gd0e6828a.el9.x86_64 27/117 2024-09-11T07:08:17.380 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-grafana-dashboards-2:17.2.7-1623.gd0e6828a. 28/117 2024-09-11T07:08:17.380 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mgr-cephadm-2:17.2.7-1623.gd0e6828a.el9.noa 29/117 2024-09-11T07:08:17.380 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mgr-dashboard-2:17.2.7-1623.gd0e6828a.el9.n 30/117 2024-09-11T07:08:17.380 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.7-1623.gd0e 31/117 2024-09-11T07:08:17.380 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mgr-modules-core-2:17.2.7-1623.gd0e6828a.el 32/117 2024-09-11T07:08:17.380 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-mgr-rook-2:17.2.7-1623.gd0e6828a.el9.noarch 33/117 2024-09-11T07:08:17.380 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-prometheus-alerts-2:17.2.7-1623.gd0e6828a.e 34/117 2024-09-11T07:08:17.380 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : ceph-volume-2:17.2.7-1623.gd0e6828a.el9.noarch 35/117 2024-09-11T07:08:17.380 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : cephadm-2:17.2.7-1623.gd0e6828a.el9.noarch 36/117 2024-09-11T07:08:17.380 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 37/117 2024-09-11T07:08:17.380 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 38/117 2024-09-11T07:08:17.380 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 39/117 2024-09-11T07:08:17.380 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 40/117 2024-09-11T07:08:17.381 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 41/117 2024-09-11T07:08:17.381 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-ply-3.11-14.el9.noarch 42/117 2024-09-11T07:08:17.381 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 43/117 2024-09-11T07:08:17.381 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 44/117 2024-09-11T07:08:17.381 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 45/117 2024-09-11T07:08:17.381 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 46/117 2024-09-11T07:08:17.381 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 47/117 2024-09-11T07:08:17.381 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 48/117 2024-09-11T07:08:17.381 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 49/117 2024-09-11T07:08:17.381 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 50/117 2024-09-11T07:08:17.381 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 51/117 2024-09-11T07:08:17.382 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 52/117 2024-09-11T07:08:17.382 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 53/117 2024-09-11T07:08:17.382 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 54/117 2024-09-11T07:08:17.382 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 55/117 2024-09-11T07:08:17.382 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 56/117 2024-09-11T07:08:17.382 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 57/117 2024-09-11T07:08:17.382 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-iniconfig-1.1.1-7.el9.noarch 58/117 2024-09-11T07:08:17.382 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 59/117 2024-09-11T07:08:17.382 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 60/117 2024-09-11T07:08:17.382 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 61/117 2024-09-11T07:08:17.382 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 62/117 2024-09-11T07:08:17.382 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 63/117 2024-09-11T07:08:17.382 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 64/117 2024-09-11T07:08:17.383 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 65/117 2024-09-11T07:08:17.383 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 66/117 2024-09-11T07:08:17.383 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 67/117 2024-09-11T07:08:17.383 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-pluggy-0.13.1-7.el9.noarch 68/117 2024-09-11T07:08:17.383 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 69/117 2024-09-11T07:08:17.383 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-py-1.10.0-6.el9.noarch 70/117 2024-09-11T07:08:17.383 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 71/117 2024-09-11T07:08:17.383 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 72/117 2024-09-11T07:08:17.383 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-pytest-6.2.2-7.el9.noarch 73/117 2024-09-11T07:08:17.383 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 74/117 2024-09-11T07:08:17.384 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 75/117 2024-09-11T07:08:17.384 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 76/117 2024-09-11T07:08:17.384 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 77/117 2024-09-11T07:08:17.384 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 78/117 2024-09-11T07:08:17.384 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 79/117 2024-09-11T07:08:17.384 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 80/117 2024-09-11T07:08:17.384 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 81/117 2024-09-11T07:08:17.384 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 82/117 2024-09-11T07:08:17.384 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 83/117 2024-09-11T07:08:17.384 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 84/117 2024-09-11T07:08:17.384 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 85/117 2024-09-11T07:08:17.385 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 86/117 2024-09-11T07:08:17.385 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 87/117 2024-09-11T07:08:17.385 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 88/117 2024-09-11T07:08:17.385 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 89/117 2024-09-11T07:08:17.385 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 90/117 2024-09-11T07:08:17.385 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 91/117 2024-09-11T07:08:17.385 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 92/117 2024-09-11T07:08:17.385 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 93/117 2024-09-11T07:08:17.385 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 94/117 2024-09-11T07:08:17.385 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 95/117 2024-09-11T07:08:17.385 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 96/117 2024-09-11T07:08:17.386 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 97/117 2024-09-11T07:08:17.386 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 98/117 2024-09-11T07:08:17.386 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 99/117 2024-09-11T07:08:17.386 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 100/117 2024-09-11T07:08:17.386 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 101/117 2024-09-11T07:08:17.386 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 102/117 2024-09-11T07:08:17.386 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 103/117 2024-09-11T07:08:17.386 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 104/117 2024-09-11T07:08:17.386 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 105/117 2024-09-11T07:08:17.386 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 106/117 2024-09-11T07:08:17.386 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 107/117 2024-09-11T07:08:17.387 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 108/117 2024-09-11T07:08:17.387 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 109/117 2024-09-11T07:08:17.387 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 110/117 2024-09-11T07:08:17.387 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 111/117 2024-09-11T07:08:17.387 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 112/117 2024-09-11T07:08:17.387 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 113/117 2024-09-11T07:08:17.387 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librados2-2:17.2.7-1623.gd0e6828a.el9.x86_64 114/117 2024-09-11T07:08:17.387 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 115/117 2024-09-11T07:08:17.387 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librbd1-2:17.2.7-1623.gd0e6828a.el9.x86_64 116/117 2024-09-11T07:08:17.586 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.7-1623.gd0e6828a.el9.n 101/117 2024-09-11T07:08:17.616 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.7-1623.gd0e 102/117 2024-09-11T07:08:17.715 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.7-1623.gd0e 102/117 2024-09-11T07:08:17.875 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-mgr-modules-core-2:17.2.7-1623.gd0e6828a.el 103/117 2024-09-11T07:08:17.901 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-mgr-2:17.2.7-1623.gd0e6828a.el9.x86_64 104/117 2024-09-11T07:08:17.924 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-mgr-2:17.2.7-1623.gd0e6828a.el9.x86_64 104/117 2024-09-11T07:08:17.925 INFO:teuthology.orchestra.run.smithi071.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T07:08:17.925 INFO:teuthology.orchestra.run.smithi071.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-11T07:08:17.925 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-11T07:08:17.925 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-11T07:08:17.925 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T07:08:17.969 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-mgr-rook-2:17.2.7-1623.gd0e6828a.el9.noarch 105/117 2024-09-11T07:08:17.979 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.7-1623.gd0e6828a.el9.noarch 105/117 2024-09-11T07:08:18.073 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-2:17.2.7-1623.gd0e6828a.el9.x86_64 106/117 2024-09-11T07:08:19.348 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-fuse-2:17.2.7-1623.gd0e6828a.el9.x86_64 107/117 2024-09-11T07:08:19.378 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-radosgw-2:17.2.7-1623.gd0e6828a.el9.x86_64 108/117 2024-09-11T07:08:19.457 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-radosgw-2:17.2.7-1623.gd0e6828a.el9.x86_64 108/117 2024-09-11T07:08:19.457 INFO:teuthology.orchestra.run.smithi071.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T07:08:19.457 INFO:teuthology.orchestra.run.smithi071.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-11T07:08:19.457 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-11T07:08:19.457 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-11T07:08:19.457 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T07:08:19.496 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-immutable-object-cache-2:17.2.7-1623.gd0e68 109/117 2024-09-11T07:08:19.519 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.7-1623.gd0e68 109/117 2024-09-11T07:08:19.519 INFO:teuthology.orchestra.run.smithi071.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T07:08:19.519 INFO:teuthology.orchestra.run.smithi071.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-11T07:08:19.519 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T07:08:19.689 INFO:teuthology.orchestra.run.smithi196.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 117/117 2024-09-11T07:08:19.689 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-11T07:08:19.689 INFO:teuthology.orchestra.run.smithi196.stdout:Upgraded: 2024-09-11T07:08:19.689 INFO:teuthology.orchestra.run.smithi196.stdout: librados2-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:08:19.690 INFO:teuthology.orchestra.run.smithi196.stdout: librbd1-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:08:19.690 INFO:teuthology.orchestra.run.smithi196.stdout:Installed: 2024-09-11T07:08:19.690 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:08:19.690 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-base-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:08:19.690 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-common-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:08:19.690 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-fuse-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:08:19.690 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-grafana-dashboards-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T07:08:19.690 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-immutable-object-cache-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:08:19.690 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mds-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:08:19.690 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:08:19.690 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-cephadm-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T07:08:19.690 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-dashboard-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T07:08:19.691 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-diskprediction-local-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T07:08:19.691 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-modules-core-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T07:08:19.691 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-rook-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T07:08:19.691 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mon-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:08:19.691 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-osd-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:08:19.691 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-prometheus-alerts-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T07:08:19.691 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-radosgw-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:08:19.691 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-selinux-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:08:19.691 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-test-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:08:19.691 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-volume-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T07:08:19.691 INFO:teuthology.orchestra.run.smithi196.stdout: cephadm-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T07:08:19.691 INFO:teuthology.orchestra.run.smithi196.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-11T07:08:19.691 INFO:teuthology.orchestra.run.smithi196.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-11T07:08:19.692 INFO:teuthology.orchestra.run.smithi196.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-11T07:08:19.692 INFO:teuthology.orchestra.run.smithi196.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-11T07:08:19.692 INFO:teuthology.orchestra.run.smithi196.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-11T07:08:19.692 INFO:teuthology.orchestra.run.smithi196.stdout: libcephfs-devel-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:08:19.692 INFO:teuthology.orchestra.run.smithi196.stdout: libcephfs2-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:08:19.692 INFO:teuthology.orchestra.run.smithi196.stdout: libcephsqlite-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:08:19.692 INFO:teuthology.orchestra.run.smithi196.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-11T07:08:19.692 INFO:teuthology.orchestra.run.smithi196.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-11T07:08:19.692 INFO:teuthology.orchestra.run.smithi196.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-11T07:08:19.692 INFO:teuthology.orchestra.run.smithi196.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-11T07:08:19.692 INFO:teuthology.orchestra.run.smithi196.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-11T07:08:19.692 INFO:teuthology.orchestra.run.smithi196.stdout: librados-devel-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:08:19.692 INFO:teuthology.orchestra.run.smithi196.stdout: libradosstriper1-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:08:19.693 INFO:teuthology.orchestra.run.smithi196.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-11T07:08:19.693 INFO:teuthology.orchestra.run.smithi196.stdout: librgw2-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:08:19.693 INFO:teuthology.orchestra.run.smithi196.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-11T07:08:19.693 INFO:teuthology.orchestra.run.smithi196.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-11T07:08:19.693 INFO:teuthology.orchestra.run.smithi196.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-11T07:08:19.693 INFO:teuthology.orchestra.run.smithi196.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-11T07:08:19.693 INFO:teuthology.orchestra.run.smithi196.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-11T07:08:19.693 INFO:teuthology.orchestra.run.smithi196.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-11T07:08:19.693 INFO:teuthology.orchestra.run.smithi196.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-11T07:08:19.694 INFO:teuthology.orchestra.run.smithi196.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-11T07:08:19.694 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-11T07:08:19.694 INFO:teuthology.orchestra.run.smithi196.stdout: python3-ceph-argparse-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:08:19.694 INFO:teuthology.orchestra.run.smithi196.stdout: python3-ceph-common-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:08:19.694 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cephfs-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:08:19.694 INFO:teuthology.orchestra.run.smithi196.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-11T07:08:19.694 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-11T07:08:19.694 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-11T07:08:19.694 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-11T07:08:19.694 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-11T07:08:19.694 INFO:teuthology.orchestra.run.smithi196.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-11T07:08:19.695 INFO:teuthology.orchestra.run.smithi196.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-11T07:08:19.695 INFO:teuthology.orchestra.run.smithi196.stdout: python3-iniconfig-1.1.1-7.el9.noarch 2024-09-11T07:08:19.695 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-11T07:08:19.695 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-11T07:08:19.695 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-11T07:08:19.695 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-11T07:08:19.695 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-11T07:08:19.695 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-11T07:08:19.695 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-11T07:08:19.695 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-11T07:08:19.695 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-09-11T07:08:19.695 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-09-11T07:08:19.695 INFO:teuthology.orchestra.run.smithi196.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-11T07:08:19.695 INFO:teuthology.orchestra.run.smithi196.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-11T07:08:19.696 INFO:teuthology.orchestra.run.smithi196.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-11T07:08:19.696 INFO:teuthology.orchestra.run.smithi196.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-11T07:08:19.696 INFO:teuthology.orchestra.run.smithi196.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-11T07:08:19.696 INFO:teuthology.orchestra.run.smithi196.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-11T07:08:19.696 INFO:teuthology.orchestra.run.smithi196.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-11T07:08:19.696 INFO:teuthology.orchestra.run.smithi196.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-11T07:08:19.696 INFO:teuthology.orchestra.run.smithi196.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-11T07:08:19.696 INFO:teuthology.orchestra.run.smithi196.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-11T07:08:19.696 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-11T07:08:19.696 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pluggy-0.13.1-7.el9.noarch 2024-09-11T07:08:19.696 INFO:teuthology.orchestra.run.smithi196.stdout: python3-ply-3.11-14.el9.noarch 2024-09-11T07:08:19.696 INFO:teuthology.orchestra.run.smithi196.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-11T07:08:19.696 INFO:teuthology.orchestra.run.smithi196.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-11T07:08:19.697 INFO:teuthology.orchestra.run.smithi196.stdout: python3-py-1.10.0-6.el9.noarch 2024-09-11T07:08:19.697 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-11T07:08:19.697 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-11T07:08:19.697 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-11T07:08:19.697 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-11T07:08:19.697 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pytest-6.2.2-7.el9.noarch 2024-09-11T07:08:19.697 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-11T07:08:19.697 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rados-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:08:19.697 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rbd-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:08:19.697 INFO:teuthology.orchestra.run.smithi196.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-11T07:08:19.697 INFO:teuthology.orchestra.run.smithi196.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-11T07:08:19.698 INFO:teuthology.orchestra.run.smithi196.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-11T07:08:19.698 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rgw-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:08:19.698 INFO:teuthology.orchestra.run.smithi196.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-11T07:08:19.698 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-11T07:08:19.698 INFO:teuthology.orchestra.run.smithi196.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-11T07:08:19.698 INFO:teuthology.orchestra.run.smithi196.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-11T07:08:19.698 INFO:teuthology.orchestra.run.smithi196.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-11T07:08:19.698 INFO:teuthology.orchestra.run.smithi196.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-11T07:08:19.698 INFO:teuthology.orchestra.run.smithi196.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-11T07:08:19.698 INFO:teuthology.orchestra.run.smithi196.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-11T07:08:19.698 INFO:teuthology.orchestra.run.smithi196.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-11T07:08:19.698 INFO:teuthology.orchestra.run.smithi196.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-11T07:08:19.698 INFO:teuthology.orchestra.run.smithi196.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-11T07:08:19.699 INFO:teuthology.orchestra.run.smithi196.stdout: rbd-fuse-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:08:19.699 INFO:teuthology.orchestra.run.smithi196.stdout: rbd-mirror-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:08:19.699 INFO:teuthology.orchestra.run.smithi196.stdout: rbd-nbd-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:08:19.699 INFO:teuthology.orchestra.run.smithi196.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-11T07:08:19.699 INFO:teuthology.orchestra.run.smithi196.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-11T07:08:19.699 INFO:teuthology.orchestra.run.smithi196.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-11T07:08:19.699 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-11T07:08:19.700 INFO:teuthology.orchestra.run.smithi196.stdout:Complete! 2024-09-11T07:08:19.805 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : rbd-mirror-2:17.2.7-1623.gd0e6828a.el9.x86_64 110/117 2024-09-11T07:08:19.829 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: rbd-mirror-2:17.2.7-1623.gd0e6828a.el9.x86_64 110/117 2024-09-11T07:08:19.829 INFO:teuthology.orchestra.run.smithi071.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T07:08:19.829 INFO:teuthology.orchestra.run.smithi071.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-11T07:08:19.829 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-11T07:08:19.829 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-11T07:08:19.829 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T07:08:20.012 DEBUG:teuthology.parallel:result is None 2024-09-11T07:08:24.192 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-test-2:17.2.7-1623.gd0e6828a.el9.x86_64 111/117 2024-09-11T07:08:24.240 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : rbd-fuse-2:17.2.7-1623.gd0e6828a.el9.x86_64 112/117 2024-09-11T07:08:24.360 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : rbd-nbd-2:17.2.7-1623.gd0e6828a.el9.x86_64 113/117 2024-09-11T07:08:24.389 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-pytest-6.2.2-7.el9.noarch 114/117 2024-09-11T07:08:24.413 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libcephfs-devel-2:17.2.7-1623.gd0e6828a.el9.x86_ 115/117 2024-09-11T07:08:24.413 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 116/117 2024-09-11T07:08:24.497 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 116/117 2024-09-11T07:08:24.497 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 117/117 2024-09-11T07:08:29.818 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 117/117 2024-09-11T07:08:29.818 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-2:17.2.7-1623.gd0e6828a.el9.x86_64 1/117 2024-09-11T07:08:29.818 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-base-2:17.2.7-1623.gd0e6828a.el9.x86_64 2/117 2024-09-11T07:08:29.818 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-common-2:17.2.7-1623.gd0e6828a.el9.x86_64 3/117 2024-09-11T07:08:29.818 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-fuse-2:17.2.7-1623.gd0e6828a.el9.x86_64 4/117 2024-09-11T07:08:29.818 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-immutable-object-cache-2:17.2.7-1623.gd0e68 5/117 2024-09-11T07:08:29.818 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-mds-2:17.2.7-1623.gd0e6828a.el9.x86_64 6/117 2024-09-11T07:08:29.819 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-mgr-2:17.2.7-1623.gd0e6828a.el9.x86_64 7/117 2024-09-11T07:08:29.819 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-mon-2:17.2.7-1623.gd0e6828a.el9.x86_64 8/117 2024-09-11T07:08:29.819 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-osd-2:17.2.7-1623.gd0e6828a.el9.x86_64 9/117 2024-09-11T07:08:29.819 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-radosgw-2:17.2.7-1623.gd0e6828a.el9.x86_64 10/117 2024-09-11T07:08:29.819 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-selinux-2:17.2.7-1623.gd0e6828a.el9.x86_64 11/117 2024-09-11T07:08:29.819 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-test-2:17.2.7-1623.gd0e6828a.el9.x86_64 12/117 2024-09-11T07:08:29.819 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libcephfs-devel-2:17.2.7-1623.gd0e6828a.el9.x86_ 13/117 2024-09-11T07:08:29.819 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libcephfs2-2:17.2.7-1623.gd0e6828a.el9.x86_64 14/117 2024-09-11T07:08:29.819 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libcephsqlite-2:17.2.7-1623.gd0e6828a.el9.x86_64 15/117 2024-09-11T07:08:29.819 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : librados-devel-2:17.2.7-1623.gd0e6828a.el9.x86_6 16/117 2024-09-11T07:08:29.819 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libradosstriper1-2:17.2.7-1623.gd0e6828a.el9.x86 17/117 2024-09-11T07:08:29.819 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : librgw2-2:17.2.7-1623.gd0e6828a.el9.x86_64 18/117 2024-09-11T07:08:29.819 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-ceph-argparse-2:17.2.7-1623.gd0e6828a.el 19/117 2024-09-11T07:08:29.820 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-ceph-common-2:17.2.7-1623.gd0e6828a.el9. 20/117 2024-09-11T07:08:29.820 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-cephfs-2:17.2.7-1623.gd0e6828a.el9.x86_6 21/117 2024-09-11T07:08:29.820 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-rados-2:17.2.7-1623.gd0e6828a.el9.x86_64 22/117 2024-09-11T07:08:29.820 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-rbd-2:17.2.7-1623.gd0e6828a.el9.x86_64 23/117 2024-09-11T07:08:29.820 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-rgw-2:17.2.7-1623.gd0e6828a.el9.x86_64 24/117 2024-09-11T07:08:29.820 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : rbd-fuse-2:17.2.7-1623.gd0e6828a.el9.x86_64 25/117 2024-09-11T07:08:29.820 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : rbd-mirror-2:17.2.7-1623.gd0e6828a.el9.x86_64 26/117 2024-09-11T07:08:29.820 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : rbd-nbd-2:17.2.7-1623.gd0e6828a.el9.x86_64 27/117 2024-09-11T07:08:29.828 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-grafana-dashboards-2:17.2.7-1623.gd0e6828a. 28/117 2024-09-11T07:08:29.828 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-mgr-cephadm-2:17.2.7-1623.gd0e6828a.el9.noa 29/117 2024-09-11T07:08:29.828 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-mgr-dashboard-2:17.2.7-1623.gd0e6828a.el9.n 30/117 2024-09-11T07:08:29.828 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.7-1623.gd0e 31/117 2024-09-11T07:08:29.828 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-mgr-modules-core-2:17.2.7-1623.gd0e6828a.el 32/117 2024-09-11T07:08:29.828 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-mgr-rook-2:17.2.7-1623.gd0e6828a.el9.noarch 33/117 2024-09-11T07:08:29.829 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-prometheus-alerts-2:17.2.7-1623.gd0e6828a.e 34/117 2024-09-11T07:08:29.829 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-volume-2:17.2.7-1623.gd0e6828a.el9.noarch 35/117 2024-09-11T07:08:29.829 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : cephadm-2:17.2.7-1623.gd0e6828a.el9.noarch 36/117 2024-09-11T07:08:29.829 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 37/117 2024-09-11T07:08:29.829 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 38/117 2024-09-11T07:08:29.829 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 39/117 2024-09-11T07:08:29.829 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 40/117 2024-09-11T07:08:29.829 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 41/117 2024-09-11T07:08:29.829 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-ply-3.11-14.el9.noarch 42/117 2024-09-11T07:08:29.829 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 43/117 2024-09-11T07:08:29.829 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 44/117 2024-09-11T07:08:29.829 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 45/117 2024-09-11T07:08:29.830 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 46/117 2024-09-11T07:08:29.830 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 47/117 2024-09-11T07:08:29.830 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 48/117 2024-09-11T07:08:29.830 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 49/117 2024-09-11T07:08:29.830 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 50/117 2024-09-11T07:08:29.830 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 51/117 2024-09-11T07:08:29.830 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 52/117 2024-09-11T07:08:29.830 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 53/117 2024-09-11T07:08:29.830 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 54/117 2024-09-11T07:08:29.830 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 55/117 2024-09-11T07:08:29.830 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 56/117 2024-09-11T07:08:29.830 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 57/117 2024-09-11T07:08:29.831 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-iniconfig-1.1.1-7.el9.noarch 58/117 2024-09-11T07:08:29.831 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 59/117 2024-09-11T07:08:29.831 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 60/117 2024-09-11T07:08:29.831 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 61/117 2024-09-11T07:08:29.831 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 62/117 2024-09-11T07:08:29.831 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 63/117 2024-09-11T07:08:29.831 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 64/117 2024-09-11T07:08:29.831 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 65/117 2024-09-11T07:08:29.831 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 66/117 2024-09-11T07:08:29.831 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 67/117 2024-09-11T07:08:29.831 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-pluggy-0.13.1-7.el9.noarch 68/117 2024-09-11T07:08:29.831 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 69/117 2024-09-11T07:08:29.831 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-py-1.10.0-6.el9.noarch 70/117 2024-09-11T07:08:29.832 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 71/117 2024-09-11T07:08:29.832 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 72/117 2024-09-11T07:08:29.832 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-pytest-6.2.2-7.el9.noarch 73/117 2024-09-11T07:08:29.832 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 74/117 2024-09-11T07:08:29.832 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 75/117 2024-09-11T07:08:29.832 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 76/117 2024-09-11T07:08:29.832 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 77/117 2024-09-11T07:08:29.832 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 78/117 2024-09-11T07:08:29.832 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 79/117 2024-09-11T07:08:29.832 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 80/117 2024-09-11T07:08:29.832 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 81/117 2024-09-11T07:08:29.832 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 82/117 2024-09-11T07:08:29.832 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 83/117 2024-09-11T07:08:29.833 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 84/117 2024-09-11T07:08:29.833 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 85/117 2024-09-11T07:08:29.833 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 86/117 2024-09-11T07:08:29.833 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 87/117 2024-09-11T07:08:29.833 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 88/117 2024-09-11T07:08:29.833 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 89/117 2024-09-11T07:08:29.833 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 90/117 2024-09-11T07:08:29.833 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 91/117 2024-09-11T07:08:29.833 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 92/117 2024-09-11T07:08:29.833 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 93/117 2024-09-11T07:08:29.833 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 94/117 2024-09-11T07:08:29.833 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 95/117 2024-09-11T07:08:29.833 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 96/117 2024-09-11T07:08:29.834 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 97/117 2024-09-11T07:08:29.834 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 98/117 2024-09-11T07:08:29.834 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 99/117 2024-09-11T07:08:29.834 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 100/117 2024-09-11T07:08:29.834 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 101/117 2024-09-11T07:08:29.834 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 102/117 2024-09-11T07:08:29.834 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 103/117 2024-09-11T07:08:29.834 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 104/117 2024-09-11T07:08:29.834 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 105/117 2024-09-11T07:08:29.834 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 106/117 2024-09-11T07:08:29.834 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 107/117 2024-09-11T07:08:29.834 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 108/117 2024-09-11T07:08:29.834 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 109/117 2024-09-11T07:08:29.835 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 110/117 2024-09-11T07:08:29.835 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 111/117 2024-09-11T07:08:29.835 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 112/117 2024-09-11T07:08:29.835 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 113/117 2024-09-11T07:08:29.835 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : librados2-2:17.2.7-1623.gd0e6828a.el9.x86_64 114/117 2024-09-11T07:08:29.835 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 115/117 2024-09-11T07:08:29.835 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : librbd1-2:17.2.7-1623.gd0e6828a.el9.x86_64 116/117 2024-09-11T07:08:32.020 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 117/117 2024-09-11T07:08:32.021 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T07:08:32.021 INFO:teuthology.orchestra.run.smithi071.stdout:Upgraded: 2024-09-11T07:08:32.021 INFO:teuthology.orchestra.run.smithi071.stdout: librados2-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:08:32.021 INFO:teuthology.orchestra.run.smithi071.stdout: librbd1-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:08:32.021 INFO:teuthology.orchestra.run.smithi071.stdout:Installed: 2024-09-11T07:08:32.021 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:08:32.021 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-base-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:08:32.021 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-common-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:08:32.021 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-fuse-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:08:32.021 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-grafana-dashboards-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T07:08:32.022 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-immutable-object-cache-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:08:32.022 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mds-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:08:32.022 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:08:32.022 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-cephadm-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T07:08:32.022 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-dashboard-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T07:08:32.022 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-diskprediction-local-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T07:08:32.022 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-modules-core-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T07:08:32.022 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-rook-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T07:08:32.022 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mon-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:08:32.022 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-osd-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:08:32.022 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-prometheus-alerts-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T07:08:32.022 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-radosgw-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:08:32.023 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-selinux-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:08:32.023 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-test-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:08:32.023 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-volume-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T07:08:32.023 INFO:teuthology.orchestra.run.smithi071.stdout: cephadm-2:17.2.7-1623.gd0e6828a.el9.noarch 2024-09-11T07:08:32.023 INFO:teuthology.orchestra.run.smithi071.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-11T07:08:32.023 INFO:teuthology.orchestra.run.smithi071.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-11T07:08:32.023 INFO:teuthology.orchestra.run.smithi071.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-11T07:08:32.023 INFO:teuthology.orchestra.run.smithi071.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-11T07:08:32.023 INFO:teuthology.orchestra.run.smithi071.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-11T07:08:32.023 INFO:teuthology.orchestra.run.smithi071.stdout: libcephfs-devel-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:08:32.023 INFO:teuthology.orchestra.run.smithi071.stdout: libcephfs2-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:08:32.023 INFO:teuthology.orchestra.run.smithi071.stdout: libcephsqlite-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:08:32.024 INFO:teuthology.orchestra.run.smithi071.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-11T07:08:32.024 INFO:teuthology.orchestra.run.smithi071.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-11T07:08:32.024 INFO:teuthology.orchestra.run.smithi071.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-11T07:08:32.024 INFO:teuthology.orchestra.run.smithi071.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-11T07:08:32.024 INFO:teuthology.orchestra.run.smithi071.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-11T07:08:32.024 INFO:teuthology.orchestra.run.smithi071.stdout: librados-devel-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:08:32.024 INFO:teuthology.orchestra.run.smithi071.stdout: libradosstriper1-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:08:32.024 INFO:teuthology.orchestra.run.smithi071.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-11T07:08:32.024 INFO:teuthology.orchestra.run.smithi071.stdout: librgw2-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:08:32.024 INFO:teuthology.orchestra.run.smithi071.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-11T07:08:32.024 INFO:teuthology.orchestra.run.smithi071.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-11T07:08:32.024 INFO:teuthology.orchestra.run.smithi071.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-11T07:08:32.025 INFO:teuthology.orchestra.run.smithi071.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-11T07:08:32.025 INFO:teuthology.orchestra.run.smithi071.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-11T07:08:32.025 INFO:teuthology.orchestra.run.smithi071.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-11T07:08:32.025 INFO:teuthology.orchestra.run.smithi071.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-11T07:08:32.025 INFO:teuthology.orchestra.run.smithi071.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-11T07:08:32.025 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-11T07:08:32.025 INFO:teuthology.orchestra.run.smithi071.stdout: python3-ceph-argparse-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:08:32.025 INFO:teuthology.orchestra.run.smithi071.stdout: python3-ceph-common-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:08:32.025 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cephfs-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:08:32.025 INFO:teuthology.orchestra.run.smithi071.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-11T07:08:32.025 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-11T07:08:32.025 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-11T07:08:32.025 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-11T07:08:32.025 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-11T07:08:32.026 INFO:teuthology.orchestra.run.smithi071.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-11T07:08:32.026 INFO:teuthology.orchestra.run.smithi071.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-11T07:08:32.026 INFO:teuthology.orchestra.run.smithi071.stdout: python3-iniconfig-1.1.1-7.el9.noarch 2024-09-11T07:08:32.026 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-11T07:08:32.026 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-11T07:08:32.026 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-11T07:08:32.026 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-11T07:08:32.026 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-11T07:08:32.026 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-11T07:08:32.026 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-11T07:08:32.026 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-11T07:08:32.026 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-09-11T07:08:32.026 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-09-11T07:08:32.027 INFO:teuthology.orchestra.run.smithi071.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-11T07:08:32.027 INFO:teuthology.orchestra.run.smithi071.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-11T07:08:32.027 INFO:teuthology.orchestra.run.smithi071.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-11T07:08:32.027 INFO:teuthology.orchestra.run.smithi071.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-11T07:08:32.027 INFO:teuthology.orchestra.run.smithi071.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-11T07:08:32.027 INFO:teuthology.orchestra.run.smithi071.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-11T07:08:32.027 INFO:teuthology.orchestra.run.smithi071.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-11T07:08:32.027 INFO:teuthology.orchestra.run.smithi071.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-11T07:08:32.027 INFO:teuthology.orchestra.run.smithi071.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-11T07:08:32.027 INFO:teuthology.orchestra.run.smithi071.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-11T07:08:32.027 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-11T07:08:32.027 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pluggy-0.13.1-7.el9.noarch 2024-09-11T07:08:32.027 INFO:teuthology.orchestra.run.smithi071.stdout: python3-ply-3.11-14.el9.noarch 2024-09-11T07:08:32.028 INFO:teuthology.orchestra.run.smithi071.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-11T07:08:32.028 INFO:teuthology.orchestra.run.smithi071.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-11T07:08:32.028 INFO:teuthology.orchestra.run.smithi071.stdout: python3-py-1.10.0-6.el9.noarch 2024-09-11T07:08:32.028 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-11T07:08:32.028 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-11T07:08:32.028 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-11T07:08:32.028 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-11T07:08:32.028 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pytest-6.2.2-7.el9.noarch 2024-09-11T07:08:32.028 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-11T07:08:32.028 INFO:teuthology.orchestra.run.smithi071.stdout: python3-rados-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:08:32.028 INFO:teuthology.orchestra.run.smithi071.stdout: python3-rbd-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:08:32.028 INFO:teuthology.orchestra.run.smithi071.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-11T07:08:32.029 INFO:teuthology.orchestra.run.smithi071.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-11T07:08:32.029 INFO:teuthology.orchestra.run.smithi071.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-11T07:08:32.029 INFO:teuthology.orchestra.run.smithi071.stdout: python3-rgw-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:08:32.029 INFO:teuthology.orchestra.run.smithi071.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-11T07:08:32.029 INFO:teuthology.orchestra.run.smithi071.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-11T07:08:32.029 INFO:teuthology.orchestra.run.smithi071.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-11T07:08:32.029 INFO:teuthology.orchestra.run.smithi071.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-11T07:08:32.029 INFO:teuthology.orchestra.run.smithi071.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-11T07:08:32.029 INFO:teuthology.orchestra.run.smithi071.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-11T07:08:32.029 INFO:teuthology.orchestra.run.smithi071.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-11T07:08:32.029 INFO:teuthology.orchestra.run.smithi071.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-11T07:08:32.029 INFO:teuthology.orchestra.run.smithi071.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-11T07:08:32.029 INFO:teuthology.orchestra.run.smithi071.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-11T07:08:32.030 INFO:teuthology.orchestra.run.smithi071.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-11T07:08:32.030 INFO:teuthology.orchestra.run.smithi071.stdout: rbd-fuse-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:08:32.030 INFO:teuthology.orchestra.run.smithi071.stdout: rbd-mirror-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:08:32.030 INFO:teuthology.orchestra.run.smithi071.stdout: rbd-nbd-2:17.2.7-1623.gd0e6828a.el9.x86_64 2024-09-11T07:08:32.030 INFO:teuthology.orchestra.run.smithi071.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-11T07:08:32.030 INFO:teuthology.orchestra.run.smithi071.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-11T07:08:32.030 INFO:teuthology.orchestra.run.smithi071.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-11T07:08:32.030 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T07:08:32.030 INFO:teuthology.orchestra.run.smithi071.stdout:Complete! 2024-09-11T07:08:32.348 DEBUG:teuthology.parallel:result is None 2024-09-11T07:08:32.349 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d0e6828a2016d48cf25ad84064e50742bb1c39b9 2024-09-11T07:08:32.495 DEBUG:teuthology.orchestra.run.smithi071:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-11T07:08:32.517 INFO:teuthology.orchestra.run.smithi071.stdout:17.2.7-1623.gd0e6828a.el9 2024-09-11T07:08:32.518 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-1623.gd0e6828a.el9 2024-09-11T07:08:32.518 INFO:teuthology.task.install:The correct ceph version 17.2.7-1623.gd0e6828a is installed. 2024-09-11T07:08:32.520 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d0e6828a2016d48cf25ad84064e50742bb1c39b9 2024-09-11T07:08:32.673 DEBUG:teuthology.orchestra.run.smithi196:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-11T07:08:32.701 INFO:teuthology.orchestra.run.smithi196.stdout:17.2.7-1623.gd0e6828a.el9 2024-09-11T07:08:32.701 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-1623.gd0e6828a.el9 2024-09-11T07:08:32.701 INFO:teuthology.task.install:The correct ceph version 17.2.7-1623.gd0e6828a is installed. 2024-09-11T07:08:32.703 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-11T07:08:32.703 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-11T07:08:32.703 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-11T07:08:32.730 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-11T07:08:32.730 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-11T07:08:32.773 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-11T07:08:32.773 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-11T07:08:32.773 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/usr/bin/daemon-helper 2024-09-11T07:08:32.797 DEBUG:teuthology.orchestra.run.smithi071:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-11T07:08:32.865 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-11T07:08:32.865 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/usr/bin/daemon-helper 2024-09-11T07:08:32.894 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-11T07:08:32.962 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-11T07:08:32.962 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-11T07:08:32.962 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-11T07:08:32.986 DEBUG:teuthology.orchestra.run.smithi071:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-11T07:08:33.051 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-11T07:08:33.051 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-11T07:08:33.078 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-11T07:08:33.144 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-11T07:08:33.144 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-11T07:08:33.144 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/usr/bin/stdin-killer 2024-09-11T07:08:33.168 DEBUG:teuthology.orchestra.run.smithi071:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-11T07:08:33.233 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-11T07:08:33.233 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/usr/bin/stdin-killer 2024-09-11T07:08:33.259 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-11T07:08:33.325 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-11T07:08:33.416 INFO:tasks.cephadm:Config: {'conf': {'mgr': {'debug mgr': 20, 'debug ms': 1}, 'global': {'mon election default strategy': 3, 'ms bind msgr2': False, 'ms type': 'async'}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'debug ms': 1, 'debug osd': 20, 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'but it is still running', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(PG_', '\\(OSD_', '\\(OBJECT_', '\\(POOL_APP_NOT_ENABLED\\)'], 'mon_bind_msgr2': False, 'sha1': 'd0e6828a2016d48cf25ad84064e50742bb1c39b9', 'cephadm_mode': 'root'} 2024-09-11T07:08:33.416 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 2024-09-11T07:08:33.416 INFO:tasks.cephadm:Cluster fsid is a81cffe6-700c-11ef-bcea-c7b262605968 2024-09-11T07:08:33.416 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-11T07:08:33.417 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '[v1:172.21.15.71:6789]', 'mon.c': '[v1:172.21.15.71:6790]', 'mon.b': '[v1:172.21.15.196:6789]'} 2024-09-11T07:08:33.417 INFO:tasks.cephadm:First mon is mon.a on smithi071 2024-09-11T07:08:33.417 INFO:tasks.cephadm:First mgr is y 2024-09-11T07:08:33.417 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-11T07:08:33.417 DEBUG:teuthology.orchestra.run.smithi071:> sudo hostname $(hostname -s) 2024-09-11T07:08:33.442 DEBUG:teuthology.orchestra.run.smithi196:> sudo hostname $(hostname -s) 2024-09-11T07:08:33.468 INFO:tasks.cephadm:Downloading cephadm (repo https://git.ceph.com/ceph-ci.git ref d0e6828a2016d48cf25ad84064e50742bb1c39b9)... 2024-09-11T07:08:33.468 DEBUG:teuthology.orchestra.run.smithi071:> git clone https://git.ceph.com/ceph-ci.git testrepo && cd testrepo && git show d0e6828a2016d48cf25ad84064e50742bb1c39b9:src/cephadm/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-11T07:08:33.497 INFO:teuthology.orchestra.run.smithi071.stderr:Cloning into 'testrepo'... 2024-09-11T07:09:30.412 INFO:teuthology.orchestra.run.smithi071.stdout:-rw-r--r--. 1 ubuntu ubuntu 378585 Sep 11 07:09 /home/ubuntu/cephtest/cephadm 2024-09-11T07:09:30.413 DEBUG:teuthology.orchestra.run.smithi196:> git clone https://git.ceph.com/ceph-ci.git testrepo && cd testrepo && git show d0e6828a2016d48cf25ad84064e50742bb1c39b9:src/cephadm/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-11T07:09:30.433 INFO:teuthology.orchestra.run.smithi196.stderr:Cloning into 'testrepo'... 2024-09-11T07:10:27.637 INFO:teuthology.orchestra.run.smithi196.stdout:-rw-r--r--. 1 ubuntu ubuntu 378585 Sep 11 07:10 /home/ubuntu/cephtest/cephadm 2024-09-11T07:10:27.638 DEBUG:teuthology.orchestra.run.smithi071:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-11T07:10:27.659 DEBUG:teuthology.orchestra.run.smithi196:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-11T07:10:27.707 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 on all hosts... 2024-09-11T07:10:27.707 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 pull 2024-09-11T07:10:27.710 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 pull 2024-09-11T07:10:29.344 INFO:teuthology.orchestra.run.smithi071.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9... 2024-09-11T07:10:30.990 INFO:teuthology.orchestra.run.smithi196.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9... 2024-09-11T07:10:58.448 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-11T07:10:58.449 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_version": "ceph version 17.2.7-1623-gd0e6828a (d0e6828a2016d48cf25ad84064e50742bb1c39b9) quincy (stable)", 2024-09-11T07:10:58.449 INFO:teuthology.orchestra.run.smithi071.stdout: "image_id": "4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b", 2024-09-11T07:10:58.449 INFO:teuthology.orchestra.run.smithi071.stdout: "repo_digests": [ 2024-09-11T07:10:58.449 INFO:teuthology.orchestra.run.smithi071.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40" 2024-09-11T07:10:58.449 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-11T07:10:58.449 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-11T07:10:59.840 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-09-11T07:10:59.873 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_version": "ceph version 17.2.7-1623-gd0e6828a (d0e6828a2016d48cf25ad84064e50742bb1c39b9) quincy (stable)", 2024-09-11T07:10:59.873 INFO:teuthology.orchestra.run.smithi196.stdout: "image_id": "4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b", 2024-09-11T07:10:59.873 INFO:teuthology.orchestra.run.smithi196.stdout: "repo_digests": [ 2024-09-11T07:10:59.873 INFO:teuthology.orchestra.run.smithi196.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40" 2024-09-11T07:10:59.873 INFO:teuthology.orchestra.run.smithi196.stdout: ] 2024-09-11T07:10:59.874 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-09-11T07:10:59.875 DEBUG:teuthology.orchestra.run.smithi071:> sudo mkdir -p /etc/ceph 2024-09-11T07:10:59.909 DEBUG:teuthology.orchestra.run.smithi196:> sudo mkdir -p /etc/ceph 2024-09-11T07:10:59.938 DEBUG:teuthology.orchestra.run.smithi071:> sudo chmod 777 /etc/ceph 2024-09-11T07:10:59.977 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod 777 /etc/ceph 2024-09-11T07:11:00.002 INFO:tasks.cephadm:Writing seed config... 2024-09-11T07:11:00.004 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-11T07:11:00.004 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-11T07:11:00.004 INFO:tasks.cephadm: override: [global] mon election default strategy = 3 2024-09-11T07:11:00.004 INFO:tasks.cephadm: override: [global] ms bind msgr2 = False 2024-09-11T07:11:00.004 INFO:tasks.cephadm: override: [global] ms type = async 2024-09-11T07:11:00.004 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-11T07:11:00.004 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-11T07:11:00.004 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-11T07:11:00.004 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-11T07:11:00.004 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-11T07:11:00.004 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-09-11T07:11:00.005 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-11T07:11:00.005 DEBUG:teuthology.orchestra.run.smithi071:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-11T07:11:00.035 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = a81cffe6-700c-11ef-bcea-c7b262605968 mon election default strategy = 3 ms bind msgr2 = False ms type = async [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true debug ms = 1 debug osd = 20 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true 2024-09-11T07:11:00.035 DEBUG:teuthology.orchestra.run.smithi071:mon.a> sudo journalctl -f -n 0 -u ceph-a81cffe6-700c-11ef-bcea-c7b262605968@mon.a.service 2024-09-11T07:11:00.077 DEBUG:teuthology.orchestra.run.smithi071:mgr.y> sudo journalctl -f -n 0 -u ceph-a81cffe6-700c-11ef-bcea-c7b262605968@mgr.y.service 2024-09-11T07:11:00.121 INFO:tasks.cephadm:Bootstrapping... 2024-09-11T07:11:00.121 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 -v bootstrap --fsid a81cffe6-700c-11ef-bcea-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id y --orphan-initial-daemons --skip-monitoring-stack --mon-addrv '[v1:172.21.15.71:6789]' --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-11T07:11:00.319 INFO:teuthology.orchestra.run.smithi071.stdout:-------------------------------------------------------------------------------- 2024-09-11T07:11:00.319 INFO:teuthology.orchestra.run.smithi071.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9', '-v', 'bootstrap', '--fsid', 'a81cffe6-700c-11ef-bcea-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'y', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-addrv', '[v1:172.21.15.71:6789]', '--skip-admin-label'] 2024-09-11T07:11:00.343 INFO:teuthology.orchestra.run.smithi071.stdout:/bin/podman: stdout 5.2.2 2024-09-11T07:11:00.343 INFO:teuthology.orchestra.run.smithi071.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-11T07:11:00.343 INFO:teuthology.orchestra.run.smithi071.stdout:Verifying podman|docker is present... 2024-09-11T07:11:00.366 INFO:teuthology.orchestra.run.smithi071.stdout:/bin/podman: stdout 5.2.2 2024-09-11T07:11:00.366 INFO:teuthology.orchestra.run.smithi071.stdout:Verifying lvm2 is present... 2024-09-11T07:11:00.366 INFO:teuthology.orchestra.run.smithi071.stdout:Verifying time synchronization is in place... 2024-09-11T07:11:00.374 INFO:teuthology.orchestra.run.smithi071.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-11T07:11:00.374 INFO:teuthology.orchestra.run.smithi071.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-11T07:11:00.381 INFO:teuthology.orchestra.run.smithi071.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-11T07:11:00.381 INFO:teuthology.orchestra.run.smithi071.stdout:systemctl: stdout inactive 2024-09-11T07:11:00.388 INFO:teuthology.orchestra.run.smithi071.stdout:systemctl: stdout enabled 2024-09-11T07:11:00.395 INFO:teuthology.orchestra.run.smithi071.stdout:systemctl: stdout active 2024-09-11T07:11:00.395 INFO:teuthology.orchestra.run.smithi071.stdout:Unit chronyd.service is enabled and running 2024-09-11T07:11:00.395 INFO:teuthology.orchestra.run.smithi071.stdout:Repeating the final host check... 2024-09-11T07:11:00.417 INFO:teuthology.orchestra.run.smithi071.stdout:/bin/podman: stdout 5.2.2 2024-09-11T07:11:00.417 INFO:teuthology.orchestra.run.smithi071.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-11T07:11:00.417 INFO:teuthology.orchestra.run.smithi071.stdout:systemctl is present 2024-09-11T07:11:00.417 INFO:teuthology.orchestra.run.smithi071.stdout:lvcreate is present 2024-09-11T07:11:00.424 INFO:teuthology.orchestra.run.smithi071.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-11T07:11:00.424 INFO:teuthology.orchestra.run.smithi071.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-11T07:11:00.430 INFO:teuthology.orchestra.run.smithi071.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-11T07:11:00.430 INFO:teuthology.orchestra.run.smithi071.stdout:systemctl: stdout inactive 2024-09-11T07:11:00.437 INFO:teuthology.orchestra.run.smithi071.stdout:systemctl: stdout enabled 2024-09-11T07:11:00.444 INFO:teuthology.orchestra.run.smithi071.stdout:systemctl: stdout active 2024-09-11T07:11:00.444 INFO:teuthology.orchestra.run.smithi071.stdout:Unit chronyd.service is enabled and running 2024-09-11T07:11:00.444 INFO:teuthology.orchestra.run.smithi071.stdout:Host looks OK 2024-09-11T07:11:00.444 INFO:teuthology.orchestra.run.smithi071.stdout:Cluster fsid: a81cffe6-700c-11ef-bcea-c7b262605968 2024-09-11T07:11:00.445 INFO:teuthology.orchestra.run.smithi071.stdout:Acquiring lock 139917045265312 on /run/cephadm/a81cffe6-700c-11ef-bcea-c7b262605968.lock 2024-09-11T07:11:00.445 INFO:teuthology.orchestra.run.smithi071.stdout:Lock 139917045265312 acquired on /run/cephadm/a81cffe6-700c-11ef-bcea-c7b262605968.lock 2024-09-11T07:11:00.445 INFO:teuthology.orchestra.run.smithi071.stdout:Verifying IP 172.21.15.71 port 6789 ... 2024-09-11T07:11:00.445 INFO:teuthology.orchestra.run.smithi071.stdout:Base mon IP(s) is [172.21.15.71:6789], mon addrv is [v1:172.21.15.71:6789] 2024-09-11T07:11:00.449 INFO:teuthology.orchestra.run.smithi071.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.71 metric 100 2024-09-11T07:11:00.449 INFO:teuthology.orchestra.run.smithi071.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.71 metric 100 2024-09-11T07:11:00.451 INFO:teuthology.orchestra.run.smithi071.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-11T07:11:00.452 INFO:teuthology.orchestra.run.smithi071.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2024-09-11T07:11:00.452 INFO:teuthology.orchestra.run.smithi071.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2024-09-11T07:11:00.454 INFO:teuthology.orchestra.run.smithi071.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-11T07:11:00.454 INFO:teuthology.orchestra.run.smithi071.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-11T07:11:00.454 INFO:teuthology.orchestra.run.smithi071.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-11T07:11:00.455 INFO:teuthology.orchestra.run.smithi071.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-09-11T07:11:00.455 INFO:teuthology.orchestra.run.smithi071.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:d765/64 scope link noprefixroute 2024-09-11T07:11:00.455 INFO:teuthology.orchestra.run.smithi071.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-11T07:11:00.455 INFO:teuthology.orchestra.run.smithi071.stdout:Mon IP `172.21.15.71` is in CIDR network `172.21.0.0/20` 2024-09-11T07:11:00.455 INFO:teuthology.orchestra.run.smithi071.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20'] 2024-09-11T07:11:00.455 INFO:teuthology.orchestra.run.smithi071.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-11T07:11:00.456 INFO:teuthology.orchestra.run.smithi071.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9... 2024-09-11T07:11:00.898 INFO:teuthology.orchestra.run.smithi071.stdout:/bin/podman: stdout 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b 2024-09-11T07:11:00.898 INFO:teuthology.orchestra.run.smithi071.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9... 2024-09-11T07:11:00.898 INFO:teuthology.orchestra.run.smithi071.stdout:/bin/podman: stderr Getting image source signatures 2024-09-11T07:11:00.898 INFO:teuthology.orchestra.run.smithi071.stdout:/bin/podman: stderr Copying blob sha256:0d2f264d63cec2f6fb0f85eed7b2c8c5b4db3ab9edd9d71a74c04647294a553f 2024-09-11T07:11:00.898 INFO:teuthology.orchestra.run.smithi071.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-11T07:11:00.898 INFO:teuthology.orchestra.run.smithi071.stdout:/bin/podman: stderr Copying config sha256:4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b 2024-09-11T07:11:00.898 INFO:teuthology.orchestra.run.smithi071.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-11T07:11:01.786 INFO:teuthology.orchestra.run.smithi071.stdout:ceph: stdout ceph version 17.2.7-1623-gd0e6828a (d0e6828a2016d48cf25ad84064e50742bb1c39b9) quincy (stable) 2024-09-11T07:11:01.786 INFO:teuthology.orchestra.run.smithi071.stdout:Ceph version: ceph version 17.2.7-1623-gd0e6828a (d0e6828a2016d48cf25ad84064e50742bb1c39b9) quincy (stable) 2024-09-11T07:11:01.786 INFO:teuthology.orchestra.run.smithi071.stdout:Extracting ceph user uid/gid from container image... 2024-09-11T07:11:02.725 INFO:teuthology.orchestra.run.smithi071.stdout:stat: stdout 167 167 2024-09-11T07:11:02.725 INFO:teuthology.orchestra.run.smithi071.stdout:Creating initial keys... 2024-09-11T07:11:03.655 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph-authtool: stdout AQCHQuFm+5K5AhAAKhnGtT2JkRmASOqukJchYQ== 2024-09-11T07:11:04.569 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph-authtool: stdout AQCHQuFmFiU5OhAAmTYYXqBqxzEcwDcqx3sCSw== 2024-09-11T07:11:05.408 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph-authtool: stdout AQCIQuFmZusQMxAAw+5/Nkz/5MoZG+S7K2kcCw== 2024-09-11T07:11:05.409 INFO:teuthology.orchestra.run.smithi071.stdout:Creating initial monmap... 2024-09-11T07:11:06.372 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-11T07:11:06.372 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = octopus 2024-09-11T07:11:06.372 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to a81cffe6-700c-11ef-bcea-c7b262605968 2024-09-11T07:11:06.372 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-11T07:11:06.372 INFO:teuthology.orchestra.run.smithi071.stdout:monmaptool for a [v1:172.21.15.71:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-11T07:11:06.372 INFO:teuthology.orchestra.run.smithi071.stdout:setting min_mon_release = octopus 2024-09-11T07:11:06.372 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/monmaptool: set fsid to a81cffe6-700c-11ef-bcea-c7b262605968 2024-09-11T07:11:06.372 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-11T07:11:06.372 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T07:11:06.372 INFO:teuthology.orchestra.run.smithi071.stdout:Creating mon... 2024-09-11T07:11:07.361 INFO:teuthology.orchestra.run.smithi071.stdout:create mon.a on 2024-09-11T07:11:07.576 INFO:teuthology.orchestra.run.smithi071.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-09-11T07:11:07.746 INFO:teuthology.orchestra.run.smithi071.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-11T07:11:07.901 INFO:teuthology.orchestra.run.smithi071.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-a81cffe6-700c-11ef-bcea-c7b262605968.target → /etc/systemd/system/ceph-a81cffe6-700c-11ef-bcea-c7b262605968.target. 2024-09-11T07:11:07.901 INFO:teuthology.orchestra.run.smithi071.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-a81cffe6-700c-11ef-bcea-c7b262605968.target → /etc/systemd/system/ceph-a81cffe6-700c-11ef-bcea-c7b262605968.target. 2024-09-11T07:11:08.079 INFO:teuthology.orchestra.run.smithi071.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-a81cffe6-700c-11ef-bcea-c7b262605968@mon.a 2024-09-11T07:11:08.080 INFO:teuthology.orchestra.run.smithi071.stdout:systemctl: stderr Failed to reset failed state of unit ceph-a81cffe6-700c-11ef-bcea-c7b262605968@mon.a.service: Unit ceph-a81cffe6-700c-11ef-bcea-c7b262605968@mon.a.service not loaded. 2024-09-11T07:11:08.252 INFO:teuthology.orchestra.run.smithi071.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-a81cffe6-700c-11ef-bcea-c7b262605968.target.wants/ceph-a81cffe6-700c-11ef-bcea-c7b262605968@mon.a.service → /etc/systemd/system/ceph-a81cffe6-700c-11ef-bcea-c7b262605968@.service. 2024-09-11T07:11:08.776 INFO:teuthology.orchestra.run.smithi071.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-11T07:11:08.776 INFO:teuthology.orchestra.run.smithi071.stdout:systemctl: stdout disabled 2024-09-11T07:11:08.783 INFO:teuthology.orchestra.run.smithi071.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-11T07:11:08.783 INFO:teuthology.orchestra.run.smithi071.stdout:systemctl: stdout inactive 2024-09-11T07:11:08.783 INFO:teuthology.orchestra.run.smithi071.stdout:firewalld.service is not enabled 2024-09-11T07:11:08.784 INFO:teuthology.orchestra.run.smithi071.stdout:Not possible to enable service . firewalld.service is not available 2024-09-11T07:11:08.784 INFO:teuthology.orchestra.run.smithi071.stdout:Waiting for mon to start... 2024-09-11T07:11:08.784 INFO:teuthology.orchestra.run.smithi071.stdout:Waiting for mon... 2024-09-11T07:11:08.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:08 smithi071 systemd[1]: Started Ceph mon.a for a81cffe6-700c-11ef-bcea-c7b262605968. 2024-09-11T07:11:09.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:09 smithi071 ceph-mon[26645]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-11T07:11:09.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:09 smithi071 ceph-mon[26645]: monmap e1: 1 mons at {a=v1:172.21.15.71:6789/0} removed_ranks: {} disallowed_leaders: {} 2024-09-11T07:11:09.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:09 smithi071 ceph-mon[26645]: fsmap 2024-09-11T07:11:09.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:09 smithi071 ceph-mon[26645]: osdmap e1: 0 total, 0 up, 0 in 2024-09-11T07:11:09.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:09 smithi071 ceph-mon[26645]: mgrmap e1: no daemons active 2024-09-11T07:11:09.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:09 smithi071 ceph-mon[26645]: from='client.? v1:172.21.15.71:0/2424206573' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-09-11T07:11:09.864 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout cluster: 2024-09-11T07:11:09.864 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout id: a81cffe6-700c-11ef-bcea-c7b262605968 2024-09-11T07:11:09.864 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-11T07:11:09.864 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout 2024-09-11T07:11:09.864 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout services: 2024-09-11T07:11:09.864 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.551696s) 2024-09-11T07:11:09.864 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-11T07:11:09.865 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-11T07:11:09.865 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout 2024-09-11T07:11:09.865 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout data: 2024-09-11T07:11:09.865 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-11T07:11:09.865 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-11T07:11:09.865 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-11T07:11:09.865 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout pgs: 2024-09-11T07:11:09.865 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout 2024-09-11T07:11:09.865 INFO:teuthology.orchestra.run.smithi071.stdout:mon is available 2024-09-11T07:11:09.865 INFO:teuthology.orchestra.run.smithi071.stdout:Assimilating anything we can from ceph.conf... 2024-09-11T07:11:10.817 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:10 smithi071 ceph-mon[26645]: from='client.? v1:172.21.15.71:0/3878989289' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-11T07:11:10.817 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:10 smithi071 ceph-mon[26645]: from='client.? v1:172.21.15.71:0/3878989289' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-11T07:11:10.820 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout 2024-09-11T07:11:10.820 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout [global] 2024-09-11T07:11:10.821 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout fsid = a81cffe6-700c-11ef-bcea-c7b262605968 2024-09-11T07:11:10.821 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-09-11T07:11:10.821 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout mon_host = [v1:172.21.15.71:6789] 2024-09-11T07:11:10.821 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-11T07:11:10.821 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-11T07:11:10.821 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-11T07:11:10.821 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-11T07:11:10.821 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout 2024-09-11T07:11:10.821 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-11T07:11:10.821 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-11T07:11:10.821 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout 2024-09-11T07:11:10.821 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout [osd] 2024-09-11T07:11:10.822 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-11T07:11:10.822 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-11T07:11:10.822 INFO:teuthology.orchestra.run.smithi071.stdout:Generating new minimal ceph.conf... 2024-09-11T07:11:11.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:11 smithi071 ceph-mon[26645]: from='client.? v1:172.21.15.71:0/579367126' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:11:11.842 INFO:teuthology.orchestra.run.smithi071.stdout:Restarting the monitor... 2024-09-11T07:11:12.144 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:11 smithi071 systemd[1]: Stopping Ceph mon.a for a81cffe6-700c-11ef-bcea-c7b262605968... 2024-09-11T07:11:12.477 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:12 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mon-a[26641]: 2024-09-11T07:11:12.141+0000 7f340ee84640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T07:11:12.477 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:12 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mon-a[26641]: 2024-09-11T07:11:12.141+0000 7f340ee84640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-09-11T07:11:12.477 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:12 smithi071 podman[26890]: 2024-09-11 07:11:12.213473791 +0000 UTC m=+0.206061819 container died d67a0f2ac88dd3e9662aaaa6251f53c39fee6d7ad60f540ec8ddf3b905b2e8b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mon-a, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-11T07:11:12.835 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:12 smithi071 podman[26890]: 2024-09-11 07:11:12.574560776 +0000 UTC m=+0.567148806 container cleanup d67a0f2ac88dd3e9662aaaa6251f53c39fee6d7ad60f540ec8ddf3b905b2e8b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-09-11T07:11:12.836 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:12 smithi071 bash[26890]: ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mon-a 2024-09-11T07:11:13.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:12 smithi071 podman[26904]: 2024-09-11 07:11:12.834003664 +0000 UTC m=+0.618440657 container remove d67a0f2ac88dd3e9662aaaa6251f53c39fee6d7ad60f540ec8ddf3b905b2e8b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mon-a, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T07:11:13.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:12 smithi071 systemd[1]: ceph-a81cffe6-700c-11ef-bcea-c7b262605968@mon.a.service: Deactivated successfully. 2024-09-11T07:11:13.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:12 smithi071 systemd[1]: Stopped Ceph mon.a for a81cffe6-700c-11ef-bcea-c7b262605968. 2024-09-11T07:11:13.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:12 smithi071 systemd[1]: Starting Ceph mon.a for a81cffe6-700c-11ef-bcea-c7b262605968... 2024-09-11T07:11:13.438 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 podman[26994]: 2024-09-11 07:11:13.184848085 +0000 UTC m=+0.105569960 container create 4d63348e9b889c37707e770b9469f1b00d91f29a1cd77a6e2d735c6dcb084047 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mon-a, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2) 2024-09-11T07:11:13.438 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 podman[26994]: 2024-09-11 07:11:13.094658273 +0000 UTC m=+0.015380150 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 2024-09-11T07:11:13.438 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 podman[26994]: 2024-09-11 07:11:13.412602899 +0000 UTC m=+0.333324765 container init 4d63348e9b889c37707e770b9469f1b00d91f29a1cd77a6e2d735c6dcb084047 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mon-a, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-09-11T07:11:13.438 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 podman[26994]: 2024-09-11 07:11:13.415743776 +0000 UTC m=+0.336465645 container start 4d63348e9b889c37707e770b9469f1b00d91f29a1cd77a6e2d735c6dcb084047 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.vendor=CentOS) 2024-09-11T07:11:13.438 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: set uid:gid to 167:167 (ceph:ceph) 2024-09-11T07:11:13.438 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: ceph version 17.2.7-1623-gd0e6828a (d0e6828a2016d48cf25ad84064e50742bb1c39b9) quincy (stable), process ceph-mon, pid 2 2024-09-11T07:11:13.438 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: pidfile_write: ignore empty --pid-file 2024-09-11T07:11:13.438 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: load: jerasure load: lrc 2024-09-11T07:11:13.543 INFO:teuthology.orchestra.run.smithi071.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-11T07:11:13.764 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: RocksDB version: 6.15.5 2024-09-11T07:11:13.764 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Git sha rocksdb_build_git_sha:@0@ 2024-09-11T07:11:13.764 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Compile date Sep 10 2024 2024-09-11T07:11:13.764 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: DB SUMMARY 2024-09-11T07:11:13.764 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: DB Session ID: F487NXXX6BRDLUTKKIMI 2024-09-11T07:11:13.764 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: CURRENT file: CURRENT 2024-09-11T07:11:13.764 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: IDENTITY file: IDENTITY 2024-09-11T07:11:13.764 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: MANIFEST file: MANIFEST-000009 size: 131 Bytes 2024-09-11T07:11:13.765 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-09-11T07:11:13.765 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000010.log size: 81984 ; 2024-09-11T07:11:13.765 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.error_if_exists: 0 2024-09-11T07:11:13.765 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.create_if_missing: 0 2024-09-11T07:11:13.765 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.paranoid_checks: 1 2024-09-11T07:11:13.765 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-11T07:11:13.765 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.env: 0x5636c2a97440 2024-09-11T07:11:13.765 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.fs: Posix File System 2024-09-11T07:11:13.765 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.info_log: 0x5636c3332e60 2024-09-11T07:11:13.765 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.max_file_opening_threads: 16 2024-09-11T07:11:13.765 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.statistics: (nil) 2024-09-11T07:11:13.765 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.use_fsync: 0 2024-09-11T07:11:13.766 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.max_log_file_size: 0 2024-09-11T07:11:13.766 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-11T07:11:13.766 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-11T07:11:13.766 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.keep_log_file_num: 1000 2024-09-11T07:11:13.766 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.recycle_log_file_num: 0 2024-09-11T07:11:13.766 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.allow_fallocate: 1 2024-09-11T07:11:13.766 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.allow_mmap_reads: 0 2024-09-11T07:11:13.766 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.allow_mmap_writes: 0 2024-09-11T07:11:13.766 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.use_direct_reads: 0 2024-09-11T07:11:13.766 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-11T07:11:13.766 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.create_missing_column_families: 0 2024-09-11T07:11:13.766 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.db_log_dir: 2024-09-11T07:11:13.766 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-a/store.db 2024-09-11T07:11:13.766 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-11T07:11:13.767 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-11T07:11:13.767 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-11T07:11:13.767 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-11T07:11:13.767 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-11T07:11:13.767 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-11T07:11:13.767 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.advise_random_on_open: 1 2024-09-11T07:11:13.767 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.db_write_buffer_size: 0 2024-09-11T07:11:13.767 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.write_buffer_manager: 0x5636c33f0780 2024-09-11T07:11:13.767 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-11T07:11:13.767 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2024-09-11T07:11:13.767 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-11T07:11:13.767 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-11T07:11:13.767 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.rate_limiter: (nil) 2024-09-11T07:11:13.768 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-11T07:11:13.768 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.wal_recovery_mode: 2 2024-09-11T07:11:13.768 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.enable_thread_tracking: 0 2024-09-11T07:11:13.768 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.enable_pipelined_write: 0 2024-09-11T07:11:13.768 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.unordered_write: 0 2024-09-11T07:11:13.768 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-11T07:11:13.768 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-11T07:11:13.768 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-11T07:11:13.768 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-11T07:11:13.768 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.row_cache: None 2024-09-11T07:11:13.768 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.wal_filter: None 2024-09-11T07:11:13.768 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-11T07:11:13.768 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.allow_ingest_behind: 0 2024-09-11T07:11:13.769 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.preserve_deletes: 0 2024-09-11T07:11:13.769 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.two_write_queues: 0 2024-09-11T07:11:13.769 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.manual_wal_flush: 0 2024-09-11T07:11:13.769 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.atomic_flush: 0 2024-09-11T07:11:13.769 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-11T07:11:13.769 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-11T07:11:13.769 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-11T07:11:13.769 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.log_readahead_size: 0 2024-09-11T07:11:13.769 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-11T07:11:13.769 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.best_efforts_recovery: 0 2024-09-11T07:11:13.769 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-11T07:11:13.769 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-11T07:11:13.770 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.allow_data_in_errors: 0 2024-09-11T07:11:13.770 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.db_host_id: __hostname__ 2024-09-11T07:11:13.770 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.max_background_jobs: 2 2024-09-11T07:11:13.770 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.max_background_compactions: -1 2024-09-11T07:11:13.770 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.max_subcompactions: 1 2024-09-11T07:11:13.770 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-11T07:11:13.770 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-11T07:11:13.771 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-11T07:11:13.771 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.max_total_wal_size: 0 2024-09-11T07:11:13.772 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-11T07:11:13.772 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-11T07:11:13.772 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-11T07:11:13.772 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-11T07:11:13.772 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.max_open_files: -1 2024-09-11T07:11:13.772 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.bytes_per_sync: 0 2024-09-11T07:11:13.772 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-11T07:11:13.772 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-11T07:11:13.772 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.compaction_readahead_size: 0 2024-09-11T07:11:13.772 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.max_background_flushes: -1 2024-09-11T07:11:13.772 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Compression algorithms supported: 2024-09-11T07:11:13.772 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: kZSTD supported: 0 2024-09-11T07:11:13.773 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: kXpressCompression supported: 0 2024-09-11T07:11:13.773 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: kBZip2Compression supported: 0 2024-09-11T07:11:13.773 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-11T07:11:13.773 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: kLZ4Compression supported: 1 2024-09-11T07:11:13.773 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: kZlibCompression supported: 1 2024-09-11T07:11:13.773 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: kLZ4HCCompression supported: 1 2024-09-11T07:11:13.773 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: kSnappyCompression supported: 1 2024-09-11T07:11:13.773 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-11T07:11:13.773 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: [db/version_set.cc:4724] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000009 2024-09-11T07:11:13.773 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: [db/column_family.cc:595] --------------- Options for column family [default]: 2024-09-11T07:11:13.773 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-11T07:11:13.773 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.merge_operator: 2024-09-11T07:11:13.773 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.compaction_filter: None 2024-09-11T07:11:13.774 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.compaction_filter_factory: None 2024-09-11T07:11:13.774 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.sst_partitioner_factory: None 2024-09-11T07:11:13.774 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-11T07:11:13.774 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-11T07:11:13.774 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5636c32d9a38) 2024-09-11T07:11:13.774 INFO:journalctl@ceph.mon.a.smithi071.stdout: cache_index_and_filter_blocks: 1 2024-09-11T07:11:13.774 INFO:journalctl@ceph.mon.a.smithi071.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-11T07:11:13.774 INFO:journalctl@ceph.mon.a.smithi071.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-11T07:11:13.774 INFO:journalctl@ceph.mon.a.smithi071.stdout: pin_top_level_index_and_filter: 1 2024-09-11T07:11:13.774 INFO:journalctl@ceph.mon.a.smithi071.stdout: index_type: 0 2024-09-11T07:11:13.774 INFO:journalctl@ceph.mon.a.smithi071.stdout: data_block_index_type: 0 2024-09-11T07:11:13.774 INFO:journalctl@ceph.mon.a.smithi071.stdout: index_shortening: 1 2024-09-11T07:11:13.774 INFO:journalctl@ceph.mon.a.smithi071.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-11T07:11:13.775 INFO:journalctl@ceph.mon.a.smithi071.stdout: hash_index_allow_collision: 1 2024-09-11T07:11:13.775 INFO:journalctl@ceph.mon.a.smithi071.stdout: checksum: 1 2024-09-11T07:11:13.775 INFO:journalctl@ceph.mon.a.smithi071.stdout: no_block_cache: 0 2024-09-11T07:11:13.775 INFO:journalctl@ceph.mon.a.smithi071.stdout: block_cache: 0x5636c330f1f0 2024-09-11T07:11:13.775 INFO:journalctl@ceph.mon.a.smithi071.stdout: block_cache_name: BinnedLRUCache 2024-09-11T07:11:13.775 INFO:journalctl@ceph.mon.a.smithi071.stdout: block_cache_options: 2024-09-11T07:11:13.775 INFO:journalctl@ceph.mon.a.smithi071.stdout: capacity : 536870912 2024-09-11T07:11:13.775 INFO:journalctl@ceph.mon.a.smithi071.stdout: num_shard_bits : 4 2024-09-11T07:11:13.775 INFO:journalctl@ceph.mon.a.smithi071.stdout: strict_capacity_limit : 0 2024-09-11T07:11:13.775 INFO:journalctl@ceph.mon.a.smithi071.stdout: high_pri_pool_ratio: 0.000 2024-09-11T07:11:13.775 INFO:journalctl@ceph.mon.a.smithi071.stdout: block_cache_compressed: (nil) 2024-09-11T07:11:13.775 INFO:journalctl@ceph.mon.a.smithi071.stdout: persistent_cache: (nil) 2024-09-11T07:11:13.775 INFO:journalctl@ceph.mon.a.smithi071.stdout: block_size: 4096 2024-09-11T07:11:13.775 INFO:journalctl@ceph.mon.a.smithi071.stdout: block_size_deviation: 10 2024-09-11T07:11:13.776 INFO:journalctl@ceph.mon.a.smithi071.stdout: block_restart_interval: 16 2024-09-11T07:11:13.776 INFO:journalctl@ceph.mon.a.smithi071.stdout: index_block_restart_interval: 1 2024-09-11T07:11:13.776 INFO:journalctl@ceph.mon.a.smithi071.stdout: metadata_block_size: 4096 2024-09-11T07:11:13.776 INFO:journalctl@ceph.mon.a.smithi071.stdout: partition_filters: 0 2024-09-11T07:11:13.776 INFO:journalctl@ceph.mon.a.smithi071.stdout: use_delta_encoding: 1 2024-09-11T07:11:13.776 INFO:journalctl@ceph.mon.a.smithi071.stdout: filter_policy: rocksdb.BuiltinBloomFilter 2024-09-11T07:11:13.776 INFO:journalctl@ceph.mon.a.smithi071.stdout: whole_key_filtering: 1 2024-09-11T07:11:13.776 INFO:journalctl@ceph.mon.a.smithi071.stdout: verify_compression: 0 2024-09-11T07:11:13.776 INFO:journalctl@ceph.mon.a.smithi071.stdout: read_amp_bytes_per_bit: 0 2024-09-11T07:11:13.776 INFO:journalctl@ceph.mon.a.smithi071.stdout: format_version: 4 2024-09-11T07:11:13.776 INFO:journalctl@ceph.mon.a.smithi071.stdout: enable_index_compression: 1 2024-09-11T07:11:13.776 INFO:journalctl@ceph.mon.a.smithi071.stdout: block_align: 0 2024-09-11T07:11:13.777 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.write_buffer_size: 33554432 2024-09-11T07:11:13.777 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.max_write_buffer_number: 2 2024-09-11T07:11:13.777 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.compression: NoCompression 2024-09-11T07:11:13.777 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.bottommost_compression: Disabled 2024-09-11T07:11:13.777 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.prefix_extractor: nullptr 2024-09-11T07:11:13.777 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-11T07:11:13.777 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.num_levels: 7 2024-09-11T07:11:13.777 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-11T07:11:13.777 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-11T07:11:13.777 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-11T07:11:13.778 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-11T07:11:13.778 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-11T07:11:13.778 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-11T07:11:13.778 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-11T07:11:13.778 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-11T07:11:13.778 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-11T07:11:13.778 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-11T07:11:13.778 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-11T07:11:13.778 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.compression_opts.level: 32767 2024-09-11T07:11:13.778 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.compression_opts.strategy: 0 2024-09-11T07:11:13.778 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-11T07:11:13.778 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-11T07:11:13.778 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-11T07:11:13.778 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.compression_opts.enabled: false 2024-09-11T07:11:13.779 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-11T07:11:13.779 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-11T07:11:13.779 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-11T07:11:13.779 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.target_file_size_base: 67108864 2024-09-11T07:11:13.779 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-11T07:11:13.779 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-11T07:11:13.779 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-11T07:11:13.779 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-11T07:11:13.779 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-11T07:11:13.779 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-11T07:11:13.779 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-11T07:11:13.779 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-11T07:11:13.779 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-11T07:11:13.780 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-11T07:11:13.780 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-11T07:11:13.780 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-11T07:11:13.780 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-11T07:11:13.780 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.arena_block_size: 4194304 2024-09-11T07:11:13.780 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-11T07:11:13.780 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-11T07:11:13.780 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2024-09-11T07:11:13.780 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.disable_auto_compactions: 0 2024-09-11T07:11:13.780 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-11T07:11:13.780 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-11T07:11:13.780 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-11T07:11:13.781 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-11T07:11:13.781 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-11T07:11:13.781 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-11T07:11:13.781 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-11T07:11:13.781 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-11T07:11:13.781 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-11T07:11:13.781 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-11T07:11:13.781 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-11T07:11:13.781 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.inplace_update_support: 0 2024-09-11T07:11:13.781 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-11T07:11:13.781 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-11T07:11:13.781 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-11T07:11:13.781 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-11T07:11:13.781 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.bloom_locality: 0 2024-09-11T07:11:13.782 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.max_successive_merges: 0 2024-09-11T07:11:13.782 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-11T07:11:13.782 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.paranoid_file_checks: 0 2024-09-11T07:11:13.782 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.force_consistency_checks: 1 2024-09-11T07:11:13.782 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.report_bg_io_stats: 0 2024-09-11T07:11:13.782 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.ttl: 2592000 2024-09-11T07:11:13.782 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-11T07:11:13.782 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.enable_blob_files: false 2024-09-11T07:11:13.782 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.min_blob_size: 0 2024-09-11T07:11:13.782 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.blob_file_size: 268435456 2024-09-11T07:11:13.782 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-11T07:11:13.782 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-11T07:11:13.782 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-11T07:11:13.782 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: [db/version_set.cc:4764] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000009 succeeded,manifest_file_number is 9, next_file_number is 11, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-11T07:11:13.783 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: [db/version_set.cc:4779] Column family [default] (ID 0), log number is 5 2024-09-11T07:11:13.783 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: [db/version_set.cc:4082] Creating manifest 13 2024-09-11T07:11:13.783 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726038673438889, "job": 1, "event": "recovery_started", "wal_files": [10]} 2024-09-11T07:11:13.783 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: [db/db_impl/db_impl_open.cc:845] Recovering log #10 mode 2 2024-09-11T07:11:13.783 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: [table/block_based/filter_policy.cc:991] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2024-09-11T07:11:13.783 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726038673439590, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 14, "file_size": 78583, "file_checksum": "", "file_checksum_func_name": "Unknown", "table_properties": {"data_size": 76845, "index_size": 200, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 581, "raw_key_size": 9969, "raw_average_key_size": 47, "raw_value_size": 71167, "raw_average_value_size": 337, "num_data_blocks": 10, "num_entries": 211, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1726038673, "oldest_key_time": 0, "file_creation_time": 0, "db_id": "04732c35-d272-405e-89df-38018a2ff482", "db_session_id": "F487NXXX6BRDLUTKKIMI"}} 2024-09-11T07:11:13.783 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: [db/version_set.cc:4082] Creating manifest 15 2024-09-11T07:11:13.783 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726038673439924, "job": 1, "event": "recovery_finished"} 2024-09-11T07:11:13.783 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: [file/delete_scheduler.cc:69] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000010.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-11T07:11:13.783 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: [db/db_impl/db_impl_open.cc:1700] SstFileManager instance 0x5636c3342700 2024-09-11T07:11:13.783 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: DB pointer 0x5636c3312000 2024-09-11T07:11:13.783 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: [db/db_impl/db_impl.cc:901] ------- DUMPING STATS ------- 2024-09-11T07:11:13.784 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: rocksdb: [db/db_impl/db_impl.cc:903] 2024-09-11T07:11:13.784 INFO:journalctl@ceph.mon.a.smithi071.stdout: ** DB Stats ** 2024-09-11T07:11:13.784 INFO:journalctl@ceph.mon.a.smithi071.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-11T07:11:13.784 INFO:journalctl@ceph.mon.a.smithi071.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-11T07:11:13.784 INFO:journalctl@ceph.mon.a.smithi071.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-11T07:11:13.784 INFO:journalctl@ceph.mon.a.smithi071.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-11T07:11:13.784 INFO:journalctl@ceph.mon.a.smithi071.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-11T07:11:13.784 INFO:journalctl@ceph.mon.a.smithi071.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2024-09-11T07:11:13.784 INFO:journalctl@ceph.mon.a.smithi071.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-11T07:11:13.785 INFO:journalctl@ceph.mon.a.smithi071.stdout: 2024-09-11T07:11:13.785 INFO:journalctl@ceph.mon.a.smithi071.stdout: ** Compaction Stats [default] ** 2024-09-11T07:11:13.785 INFO:journalctl@ceph.mon.a.smithi071.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-11T07:11:13.785 INFO:journalctl@ceph.mon.a.smithi071.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-11T07:11:13.785 INFO:journalctl@ceph.mon.a.smithi071.stdout: L0 2/0 78.49 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 160.1 0.00 0.00 1 0.000 0 0 2024-09-11T07:11:13.785 INFO:journalctl@ceph.mon.a.smithi071.stdout: Sum 2/0 78.49 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 160.1 0.00 0.00 1 0.000 0 0 2024-09-11T07:11:13.785 INFO:journalctl@ceph.mon.a.smithi071.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 160.1 0.00 0.00 1 0.000 0 0 2024-09-11T07:11:13.785 INFO:journalctl@ceph.mon.a.smithi071.stdout: 2024-09-11T07:11:13.785 INFO:journalctl@ceph.mon.a.smithi071.stdout: ** Compaction Stats [default] ** 2024-09-11T07:11:13.785 INFO:journalctl@ceph.mon.a.smithi071.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-11T07:11:13.785 INFO:journalctl@ceph.mon.a.smithi071.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-11T07:11:13.785 INFO:journalctl@ceph.mon.a.smithi071.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 160.1 0.00 0.00 1 0.000 0 0 2024-09-11T07:11:13.785 INFO:journalctl@ceph.mon.a.smithi071.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-11T07:11:13.786 INFO:journalctl@ceph.mon.a.smithi071.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-11T07:11:13.786 INFO:journalctl@ceph.mon.a.smithi071.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-11T07:11:13.786 INFO:journalctl@ceph.mon.a.smithi071.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-11T07:11:13.786 INFO:journalctl@ceph.mon.a.smithi071.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-11T07:11:13.786 INFO:journalctl@ceph.mon.a.smithi071.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-11T07:11:13.786 INFO:journalctl@ceph.mon.a.smithi071.stdout: Cumulative compaction: 0.00 GB write, 25.17 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-11T07:11:13.786 INFO:journalctl@ceph.mon.a.smithi071.stdout: Interval compaction: 0.00 GB write, 25.17 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-11T07:11:13.786 INFO:journalctl@ceph.mon.a.smithi071.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-11T07:11:13.786 INFO:journalctl@ceph.mon.a.smithi071.stdout: 2024-09-11T07:11:13.786 INFO:journalctl@ceph.mon.a.smithi071.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-11T07:11:13.786 INFO:journalctl@ceph.mon.a.smithi071.stdout: 2024-09-11T07:11:13.786 INFO:journalctl@ceph.mon.a.smithi071.stdout: ** Compaction Stats [default] ** 2024-09-11T07:11:13.786 INFO:journalctl@ceph.mon.a.smithi071.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-11T07:11:13.786 INFO:journalctl@ceph.mon.a.smithi071.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-11T07:11:13.787 INFO:journalctl@ceph.mon.a.smithi071.stdout: L0 2/0 78.49 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 160.1 0.00 0.00 1 0.000 0 0 2024-09-11T07:11:13.787 INFO:journalctl@ceph.mon.a.smithi071.stdout: Sum 2/0 78.49 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 160.1 0.00 0.00 1 0.000 0 0 2024-09-11T07:11:13.787 INFO:journalctl@ceph.mon.a.smithi071.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-09-11T07:11:13.787 INFO:journalctl@ceph.mon.a.smithi071.stdout: 2024-09-11T07:11:13.787 INFO:journalctl@ceph.mon.a.smithi071.stdout: ** Compaction Stats [default] ** 2024-09-11T07:11:13.787 INFO:journalctl@ceph.mon.a.smithi071.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-11T07:11:13.787 INFO:journalctl@ceph.mon.a.smithi071.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-11T07:11:13.787 INFO:journalctl@ceph.mon.a.smithi071.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 160.1 0.00 0.00 1 0.000 0 0 2024-09-11T07:11:13.787 INFO:journalctl@ceph.mon.a.smithi071.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-11T07:11:13.787 INFO:journalctl@ceph.mon.a.smithi071.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-11T07:11:13.787 INFO:journalctl@ceph.mon.a.smithi071.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-11T07:11:13.787 INFO:journalctl@ceph.mon.a.smithi071.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-11T07:11:13.787 INFO:journalctl@ceph.mon.a.smithi071.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-11T07:11:13.787 INFO:journalctl@ceph.mon.a.smithi071.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-11T07:11:13.788 INFO:journalctl@ceph.mon.a.smithi071.stdout: Cumulative compaction: 0.00 GB write, 24.91 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-11T07:11:13.788 INFO:journalctl@ceph.mon.a.smithi071.stdout: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-11T07:11:13.788 INFO:journalctl@ceph.mon.a.smithi071.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-11T07:11:13.788 INFO:journalctl@ceph.mon.a.smithi071.stdout: 2024-09-11T07:11:13.788 INFO:journalctl@ceph.mon.a.smithi071.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-11T07:11:13.788 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: starting mon.a rank 0 at public addrs v1:172.21.15.71:6789/0 at bind addrs v1:172.21.15.71:6789/0 mon_data /var/lib/ceph/mon/ceph-a fsid a81cffe6-700c-11ef-bcea-c7b262605968 2024-09-11T07:11:13.788 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: mon.a@-1(???) e1 preinit fsid a81cffe6-700c-11ef-bcea-c7b262605968 2024-09-11T07:11:13.788 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: mon.a@-1(???).mds e1 new map 2024-09-11T07:11:13.788 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: mon.a@-1(???).mds e1 print_map 2024-09-11T07:11:13.788 INFO:journalctl@ceph.mon.a.smithi071.stdout: e1 2024-09-11T07:11:13.788 INFO:journalctl@ceph.mon.a.smithi071.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-11T07:11:13.788 INFO:journalctl@ceph.mon.a.smithi071.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-11T07:11:13.789 INFO:journalctl@ceph.mon.a.smithi071.stdout: legacy client fscid: -1 2024-09-11T07:11:13.789 INFO:journalctl@ceph.mon.a.smithi071.stdout: 2024-09-11T07:11:13.789 INFO:journalctl@ceph.mon.a.smithi071.stdout: No filesystems configured 2024-09-11T07:11:13.789 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-11T07:11:13.789 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-11T07:11:13.789 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-11T07:11:13.789 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-11T07:11:13.789 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-11T07:11:13.789 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-11T07:11:13.789 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: monmap e1: 1 mons at {a=v1:172.21.15.71:6789/0} removed_ranks: {} disallowed_leaders: {} 2024-09-11T07:11:13.789 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: fsmap 2024-09-11T07:11:13.789 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: osdmap e1: 0 total, 0 up, 0 in 2024-09-11T07:11:13.789 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 ceph-mon[27013]: mgrmap e1: no daemons active 2024-09-11T07:11:13.789 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 bash[26994]: 4d63348e9b889c37707e770b9469f1b00d91f29a1cd77a6e2d735c6dcb084047 2024-09-11T07:11:13.790 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:13 smithi071 systemd[1]: Started Ceph mon.a for a81cffe6-700c-11ef-bcea-c7b262605968. 2024-09-11T07:11:14.514 INFO:teuthology.orchestra.run.smithi071.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-11T07:11:14.515 INFO:teuthology.orchestra.run.smithi071.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-11T07:11:14.515 INFO:teuthology.orchestra.run.smithi071.stdout:Creating mgr... 2024-09-11T07:11:14.515 INFO:teuthology.orchestra.run.smithi071.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-11T07:11:14.515 INFO:teuthology.orchestra.run.smithi071.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-11T07:11:14.701 INFO:teuthology.orchestra.run.smithi071.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-a81cffe6-700c-11ef-bcea-c7b262605968@mgr.y 2024-09-11T07:11:14.701 INFO:teuthology.orchestra.run.smithi071.stdout:systemctl: stderr Failed to reset failed state of unit ceph-a81cffe6-700c-11ef-bcea-c7b262605968@mgr.y.service: Unit ceph-a81cffe6-700c-11ef-bcea-c7b262605968@mgr.y.service not loaded. 2024-09-11T07:11:14.857 INFO:teuthology.orchestra.run.smithi071.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-a81cffe6-700c-11ef-bcea-c7b262605968.target.wants/ceph-a81cffe6-700c-11ef-bcea-c7b262605968@mgr.y.service → /etc/systemd/system/ceph-a81cffe6-700c-11ef-bcea-c7b262605968@.service. 2024-09-11T07:11:14.992 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:14 smithi071 systemd[1]: Starting Ceph mgr.y for a81cffe6-700c-11ef-bcea-c7b262605968... 2024-09-11T07:11:15.281 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:14 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/1076783721' entity='client.admin' 2024-09-11T07:11:15.282 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:15 smithi071 podman[27224]: 2024-09-11 07:11:15.086695496 +0000 UTC m=+0.112445403 container create 677f18b0b24aebb06dc2008e6a45a4ea7658d9ea333d2fd55527a4345eab00a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-11T07:11:15.282 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:15 smithi071 podman[27224]: 2024-09-11 07:11:14.989455955 +0000 UTC m=+0.015205833 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 2024-09-11T07:11:15.282 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:15 smithi071 podman[27224]: 2024-09-11 07:11:15.280499514 +0000 UTC m=+0.306249386 container init 677f18b0b24aebb06dc2008e6a45a4ea7658d9ea333d2fd55527a4345eab00a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, io.buildah.version=1.37.2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2) 2024-09-11T07:11:15.404 INFO:teuthology.orchestra.run.smithi071.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-11T07:11:15.404 INFO:teuthology.orchestra.run.smithi071.stdout:systemctl: stdout disabled 2024-09-11T07:11:15.412 INFO:teuthology.orchestra.run.smithi071.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-11T07:11:15.413 INFO:teuthology.orchestra.run.smithi071.stdout:systemctl: stdout inactive 2024-09-11T07:11:15.413 INFO:teuthology.orchestra.run.smithi071.stdout:firewalld.service is not enabled 2024-09-11T07:11:15.413 INFO:teuthology.orchestra.run.smithi071.stdout:Not possible to enable service . firewalld.service is not available 2024-09-11T07:11:15.423 INFO:teuthology.orchestra.run.smithi071.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-11T07:11:15.423 INFO:teuthology.orchestra.run.smithi071.stdout:systemctl: stdout disabled 2024-09-11T07:11:15.432 INFO:teuthology.orchestra.run.smithi071.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-11T07:11:15.432 INFO:teuthology.orchestra.run.smithi071.stdout:systemctl: stdout inactive 2024-09-11T07:11:15.433 INFO:teuthology.orchestra.run.smithi071.stdout:firewalld.service is not enabled 2024-09-11T07:11:15.433 INFO:teuthology.orchestra.run.smithi071.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-09-11T07:11:15.433 INFO:teuthology.orchestra.run.smithi071.stdout:Waiting for mgr to start... 2024-09-11T07:11:15.433 INFO:teuthology.orchestra.run.smithi071.stdout:Waiting for mgr... 2024-09-11T07:11:15.549 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:15 smithi071 podman[27224]: 2024-09-11 07:11:15.283334148 +0000 UTC m=+0.309084016 container start 677f18b0b24aebb06dc2008e6a45a4ea7658d9ea333d2fd55527a4345eab00a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.37.2) 2024-09-11T07:11:15.549 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:15 smithi071 bash[27224]: 677f18b0b24aebb06dc2008e6a45a4ea7658d9ea333d2fd55527a4345eab00a5 2024-09-11T07:11:15.550 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:15 smithi071 systemd[1]: Started Ceph mgr.y for a81cffe6-700c-11ef-bcea-c7b262605968. 2024-09-11T07:11:15.550 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:15 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: 2024-09-11T07:11:15.393+0000 7f9288508ec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-11T07:11:15.550 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:15 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: 2024-09-11T07:11:15.460+0000 7f9288508ec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-11T07:11:15.550 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:15 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: 2024-09-11T07:11:15.526+0000 7f9288508ec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-11T07:11:15.839 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:15 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: 2024-09-11T07:11:15.655+0000 7f9288508ec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-11T07:11:15.839 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:15 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: 2024-09-11T07:11:15.795+0000 7f9288508ec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-11T07:11:16.127 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:15 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/2562821776' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-11T07:11:16.128 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:16 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: 2024-09-11T07:11:16.066+0000 7f9288508ec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-11T07:11:16.128 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:16 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: 2024-09-11T07:11:16.125+0000 7f9288508ec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-11T07:11:16.530 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:16 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: 2024-09-11T07:11:16.292+0000 7f9288508ec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-11T07:11:16.532 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout 2024-09-11T07:11:16.535 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout { 2024-09-11T07:11:16.535 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "fsid": "a81cffe6-700c-11ef-bcea-c7b262605968", 2024-09-11T07:11:16.535 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "health": { 2024-09-11T07:11:16.535 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-11T07:11:16.535 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-11T07:11:16.535 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-11T07:11:16.535 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout }, 2024-09-11T07:11:16.535 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-11T07:11:16.536 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-11T07:11:16.536 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout 0 2024-09-11T07:11:16.536 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout ], 2024-09-11T07:11:16.536 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-11T07:11:16.536 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "a" 2024-09-11T07:11:16.536 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout ], 2024-09-11T07:11:16.536 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-09-11T07:11:16.536 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-11T07:11:16.536 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T07:11:16.536 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-09-11T07:11:16.536 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-11T07:11:16.536 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout }, 2024-09-11T07:11:16.536 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-11T07:11:16.536 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T07:11:16.537 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-11T07:11:16.537 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-11T07:11:16.537 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-11T07:11:16.537 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-11T07:11:16.537 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-11T07:11:16.537 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-11T07:11:16.540 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout }, 2024-09-11T07:11:16.540 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-11T07:11:16.540 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-11T07:11:16.540 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-11T07:11:16.540 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-11T07:11:16.540 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-11T07:11:16.540 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-11T07:11:16.540 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-11T07:11:16.541 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-11T07:11:16.541 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-11T07:11:16.541 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout }, 2024-09-11T07:11:16.541 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-11T07:11:16.541 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T07:11:16.541 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-11T07:11:16.541 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-11T07:11:16.541 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout }, 2024-09-11T07:11:16.541 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-11T07:11:16.541 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-11T07:11:16.541 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-11T07:11:16.541 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-11T07:11:16.542 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-11T07:11:16.542 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-11T07:11:16.542 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "restful" 2024-09-11T07:11:16.542 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout ], 2024-09-11T07:11:16.542 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-11T07:11:16.542 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout }, 2024-09-11T07:11:16.542 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-11T07:11:16.542 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T07:11:16.542 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "modified": "2024-09-11T07:11:08.733552+0000", 2024-09-11T07:11:16.542 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-11T07:11:16.542 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout }, 2024-09-11T07:11:16.542 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-11T07:11:16.542 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout } 2024-09-11T07:11:16.542 INFO:teuthology.orchestra.run.smithi071.stdout:mgr not available, waiting (1/15)... 2024-09-11T07:11:17.589 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:17 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: 2024-09-11T07:11:17.129+0000 7f9288508ec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-11T07:11:17.589 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:17 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: 2024-09-11T07:11:17.192+0000 7f9288508ec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-11T07:11:18.181 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:17 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: 2024-09-11T07:11:17.896+0000 7f9288508ec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-11T07:11:18.181 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:18 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: 2024-09-11T07:11:18.004+0000 7f9288508ec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-11T07:11:18.181 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:18 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: 2024-09-11T07:11:18.121+0000 7f9288508ec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-11T07:11:18.433 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:18 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: 2024-09-11T07:11:18.178+0000 7f9288508ec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-11T07:11:18.433 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:18 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: 2024-09-11T07:11:18.302+0000 7f9288508ec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-11T07:11:18.433 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:18 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: 2024-09-11T07:11:18.373+0000 7f9288508ec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-11T07:11:18.434 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:18 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: 2024-09-11T07:11:18.431+0000 7f9288508ec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-11T07:11:18.700 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:18 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: 2024-09-11T07:11:18.567+0000 7f9288508ec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-11T07:11:18.700 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:18 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: 2024-09-11T07:11:18.632+0000 7f9288508ec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-11T07:11:18.701 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:18 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: 2024-09-11T07:11:18.698+0000 7f9288508ec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-11T07:11:18.966 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:18 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: 2024-09-11T07:11:18.779+0000 7f9288508ec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-11T07:11:18.966 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:18 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: 2024-09-11T07:11:18.841+0000 7f9288508ec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-11T07:11:18.966 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:18 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: 2024-09-11T07:11:18.900+0000 7f9288508ec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-11T07:11:19.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:19 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/128451933' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-11T07:11:19.670 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout 2024-09-11T07:11:19.670 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout { 2024-09-11T07:11:19.670 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "fsid": "a81cffe6-700c-11ef-bcea-c7b262605968", 2024-09-11T07:11:19.670 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "health": { 2024-09-11T07:11:19.670 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-11T07:11:19.670 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-11T07:11:19.670 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-11T07:11:19.671 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout }, 2024-09-11T07:11:19.671 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-11T07:11:19.671 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-11T07:11:19.671 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout 0 2024-09-11T07:11:19.671 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout ], 2024-09-11T07:11:19.671 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-11T07:11:19.671 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "a" 2024-09-11T07:11:19.671 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout ], 2024-09-11T07:11:19.671 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-09-11T07:11:19.671 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-11T07:11:19.671 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T07:11:19.671 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-09-11T07:11:19.671 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-11T07:11:19.671 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout }, 2024-09-11T07:11:19.672 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-11T07:11:19.672 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T07:11:19.672 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-11T07:11:19.672 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-11T07:11:19.672 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-11T07:11:19.672 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-11T07:11:19.672 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-11T07:11:19.672 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-11T07:11:19.672 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout }, 2024-09-11T07:11:19.672 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-11T07:11:19.672 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-11T07:11:19.672 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-11T07:11:19.672 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-11T07:11:19.672 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-11T07:11:19.672 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-11T07:11:19.673 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-11T07:11:19.673 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-11T07:11:19.673 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-11T07:11:19.673 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout }, 2024-09-11T07:11:19.673 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-11T07:11:19.673 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T07:11:19.673 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-11T07:11:19.673 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-11T07:11:19.673 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout }, 2024-09-11T07:11:19.673 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-11T07:11:19.673 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-11T07:11:19.673 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-11T07:11:19.673 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-11T07:11:19.673 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-11T07:11:19.674 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-11T07:11:19.674 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "restful" 2024-09-11T07:11:19.674 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout ], 2024-09-11T07:11:19.674 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-11T07:11:19.674 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout }, 2024-09-11T07:11:19.674 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-11T07:11:19.674 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T07:11:19.674 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "modified": "2024-09-11T07:11:08.733552+0000", 2024-09-11T07:11:19.674 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-11T07:11:19.674 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout }, 2024-09-11T07:11:19.674 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-11T07:11:19.674 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout } 2024-09-11T07:11:19.674 INFO:teuthology.orchestra.run.smithi071.stdout:mgr not available, waiting (2/15)... 2024-09-11T07:11:20.089 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:19 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: 2024-09-11T07:11:19.669+0000 7f9288508ec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-11T07:11:20.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:20 smithi071 ceph-mon[27013]: Activating manager daemon y 2024-09-11T07:11:20.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:20 smithi071 ceph-mon[27013]: mgrmap e2: y(active, starting, since 0.00285748s) 2024-09-11T07:11:20.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:20 smithi071 ceph-mon[27013]: from='mgr.14100 v1:172.21.15.71:0/2398848162' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-11T07:11:20.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:20 smithi071 ceph-mon[27013]: from='mgr.14100 v1:172.21.15.71:0/2398848162' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-11T07:11:20.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:20 smithi071 ceph-mon[27013]: from='mgr.14100 v1:172.21.15.71:0/2398848162' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-11T07:11:20.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:20 smithi071 ceph-mon[27013]: from='mgr.14100 v1:172.21.15.71:0/2398848162' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-11T07:11:20.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:20 smithi071 ceph-mon[27013]: from='mgr.14100 v1:172.21.15.71:0/2398848162' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-11T07:11:20.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:20 smithi071 ceph-mon[27013]: from='mgr.14100 v1:172.21.15.71:0/2398848162' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-11T07:11:20.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:20 smithi071 ceph-mon[27013]: from='mgr.14100 v1:172.21.15.71:0/2398848162' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T07:11:20.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:20 smithi071 ceph-mon[27013]: from='mgr.14100 v1:172.21.15.71:0/2398848162' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-11T07:11:20.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:20 smithi071 ceph-mon[27013]: Manager daemon y is now available 2024-09-11T07:11:20.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:20 smithi071 ceph-mon[27013]: from='mgr.14100 v1:172.21.15.71:0/2398848162' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-11T07:11:20.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:20 smithi071 ceph-mon[27013]: from='mgr.14100 v1:172.21.15.71:0/2398848162' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-11T07:11:20.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:20 smithi071 ceph-mon[27013]: from='mgr.14100 v1:172.21.15.71:0/2398848162' entity='mgr.y' 2024-09-11T07:11:20.591 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:20 smithi071 ceph-mon[27013]: from='mgr.14100 v1:172.21.15.71:0/2398848162' entity='mgr.y' 2024-09-11T07:11:20.591 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:20 smithi071 ceph-mon[27013]: from='mgr.14100 v1:172.21.15.71:0/2398848162' entity='mgr.y' 2024-09-11T07:11:22.005 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:21 smithi071 ceph-mon[27013]: mgrmap e3: y(active, since 1.00605s) 2024-09-11T07:11:22.881 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout 2024-09-11T07:11:22.882 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout { 2024-09-11T07:11:22.882 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "fsid": "a81cffe6-700c-11ef-bcea-c7b262605968", 2024-09-11T07:11:22.882 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "health": { 2024-09-11T07:11:22.882 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-11T07:11:22.882 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-11T07:11:22.882 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-11T07:11:22.882 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout }, 2024-09-11T07:11:22.882 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-11T07:11:22.882 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-11T07:11:22.882 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout 0 2024-09-11T07:11:22.882 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout ], 2024-09-11T07:11:22.883 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-11T07:11:22.883 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "a" 2024-09-11T07:11:22.883 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout ], 2024-09-11T07:11:22.883 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-09-11T07:11:22.883 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-11T07:11:22.883 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T07:11:22.883 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-09-11T07:11:22.883 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-11T07:11:22.883 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout }, 2024-09-11T07:11:22.883 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-11T07:11:22.883 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T07:11:22.883 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-11T07:11:22.883 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-11T07:11:22.883 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-11T07:11:22.884 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-11T07:11:22.884 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-11T07:11:22.884 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-11T07:11:22.887 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout }, 2024-09-11T07:11:22.887 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-11T07:11:22.887 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-11T07:11:22.887 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-11T07:11:22.887 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-11T07:11:22.887 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-11T07:11:22.887 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-11T07:11:22.887 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-11T07:11:22.887 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-11T07:11:22.888 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-11T07:11:22.888 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout }, 2024-09-11T07:11:22.888 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-11T07:11:22.888 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T07:11:22.888 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-11T07:11:22.888 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-11T07:11:22.888 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout }, 2024-09-11T07:11:22.888 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-11T07:11:22.888 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-11T07:11:22.888 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-11T07:11:22.888 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-11T07:11:22.888 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-11T07:11:22.888 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-11T07:11:22.888 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "restful" 2024-09-11T07:11:22.889 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout ], 2024-09-11T07:11:22.889 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-11T07:11:22.889 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout }, 2024-09-11T07:11:22.889 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-11T07:11:22.889 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-11T07:11:22.889 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "modified": "2024-09-11T07:11:08.733552+0000", 2024-09-11T07:11:22.889 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-11T07:11:22.889 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout }, 2024-09-11T07:11:22.889 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-11T07:11:22.889 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout } 2024-09-11T07:11:22.889 INFO:teuthology.orchestra.run.smithi071.stdout:mgr is available 2024-09-11T07:11:23.000 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:22 smithi071 ceph-mon[27013]: mgrmap e4: y(active, since 2s) 2024-09-11T07:11:23.001 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:22 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/1084889970' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-11T07:11:24.048 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:23 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/869536788' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-11T07:11:24.052 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout 2024-09-11T07:11:24.052 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout [global] 2024-09-11T07:11:24.052 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout fsid = a81cffe6-700c-11ef-bcea-c7b262605968 2024-09-11T07:11:24.052 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-09-11T07:11:24.052 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-11T07:11:24.053 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-11T07:11:24.053 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-11T07:11:24.053 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-11T07:11:24.053 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout 2024-09-11T07:11:24.053 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-11T07:11:24.053 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-11T07:11:24.053 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout 2024-09-11T07:11:24.053 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout [osd] 2024-09-11T07:11:24.053 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-11T07:11:24.053 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-11T07:11:24.053 INFO:teuthology.orchestra.run.smithi071.stdout:Enabling cephadm module... 2024-09-11T07:11:24.939 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:24 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/1616564863' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-09-11T07:11:24.940 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:24 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: ignoring --setuser ceph since I am not root 2024-09-11T07:11:24.940 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:24 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: ignoring --setgroup ceph since I am not root 2024-09-11T07:11:24.940 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:24 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: 2024-09-11T07:11:24.819+0000 7fa63cf7aec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-11T07:11:24.940 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:24 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: 2024-09-11T07:11:24.873+0000 7fa63cf7aec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-11T07:11:24.940 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:24 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: 2024-09-11T07:11:24.937+0000 7fa63cf7aec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-11T07:11:25.194 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:25 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: 2024-09-11T07:11:25.058+0000 7fa63cf7aec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-11T07:11:25.194 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:25 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: 2024-09-11T07:11:25.191+0000 7fa63cf7aec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-11T07:11:25.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:25 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/1616564863' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-11T07:11:25.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:25 smithi071 ceph-mon[27013]: mgrmap e5: y(active, since 5s) 2024-09-11T07:11:25.839 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:25 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: 2024-09-11T07:11:25.444+0000 7fa63cf7aec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-11T07:11:25.840 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:25 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: 2024-09-11T07:11:25.504+0000 7fa63cf7aec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-11T07:11:25.840 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:25 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: 2024-09-11T07:11:25.663+0000 7fa63cf7aec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-11T07:11:26.839 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:26 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: 2024-09-11T07:11:26.518+0000 7fa63cf7aec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-11T07:11:26.839 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:26 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: 2024-09-11T07:11:26.580+0000 7fa63cf7aec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-11T07:11:27.577 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:27 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: 2024-09-11T07:11:27.285+0000 7fa63cf7aec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-11T07:11:27.577 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:27 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: 2024-09-11T07:11:27.395+0000 7fa63cf7aec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-11T07:11:27.577 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:27 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: 2024-09-11T07:11:27.516+0000 7fa63cf7aec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-11T07:11:27.830 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:27 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: 2024-09-11T07:11:27.575+0000 7fa63cf7aec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-11T07:11:27.830 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:27 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: 2024-09-11T07:11:27.700+0000 7fa63cf7aec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-11T07:11:27.831 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:27 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: 2024-09-11T07:11:27.768+0000 7fa63cf7aec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-11T07:11:27.831 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:27 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: 2024-09-11T07:11:27.828+0000 7fa63cf7aec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-11T07:11:28.089 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:27 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: 2024-09-11T07:11:27.962+0000 7fa63cf7aec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-11T07:11:28.089 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:28 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: 2024-09-11T07:11:28.024+0000 7fa63cf7aec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-11T07:11:28.354 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:28 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: 2024-09-11T07:11:28.089+0000 7fa63cf7aec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-11T07:11:28.354 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:28 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: 2024-09-11T07:11:28.174+0000 7fa63cf7aec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-11T07:11:28.354 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:28 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: 2024-09-11T07:11:28.234+0000 7fa63cf7aec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-11T07:11:28.354 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:28 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: 2024-09-11T07:11:28.293+0000 7fa63cf7aec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-11T07:11:29.339 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:29 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: 2024-09-11T07:11:29.069+0000 7fa63cf7aec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-11T07:11:29.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:29 smithi071 ceph-mon[27013]: Active manager daemon y restarted 2024-09-11T07:11:29.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:29 smithi071 ceph-mon[27013]: Activating manager daemon y 2024-09-11T07:11:29.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:29 smithi071 ceph-mon[27013]: osdmap e2: 0 total, 0 up, 0 in 2024-09-11T07:11:29.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:29 smithi071 ceph-mon[27013]: mgrmap e6: y(active, starting, since 0.00411294s) 2024-09-11T07:11:29.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:29 smithi071 ceph-mon[27013]: from='mgr.14118 v1:172.21.15.71:0/97007527' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T07:11:29.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:29 smithi071 ceph-mon[27013]: from='mgr.14118 v1:172.21.15.71:0/97007527' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-11T07:11:29.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:29 smithi071 ceph-mon[27013]: from='mgr.14118 v1:172.21.15.71:0/97007527' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-11T07:11:29.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:29 smithi071 ceph-mon[27013]: from='mgr.14118 v1:172.21.15.71:0/97007527' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-11T07:11:29.341 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:29 smithi071 ceph-mon[27013]: from='mgr.14118 v1:172.21.15.71:0/97007527' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-11T07:11:29.341 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:29 smithi071 ceph-mon[27013]: Manager daemon y is now available 2024-09-11T07:11:29.341 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:29 smithi071 ceph-mon[27013]: from='mgr.14118 v1:172.21.15.71:0/97007527' entity='mgr.y' 2024-09-11T07:11:29.341 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:29 smithi071 ceph-mon[27013]: from='mgr.14118 v1:172.21.15.71:0/97007527' entity='mgr.y' 2024-09-11T07:11:29.341 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:29 smithi071 ceph-mon[27013]: from='mgr.14118 v1:172.21.15.71:0/97007527' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:11:29.341 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:29 smithi071 ceph-mon[27013]: from='mgr.14118 v1:172.21.15.71:0/97007527' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:11:29.341 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:29 smithi071 ceph-mon[27013]: from='mgr.14118 v1:172.21.15.71:0/97007527' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-11T07:11:29.341 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:29 smithi071 ceph-mon[27013]: from='mgr.14118 v1:172.21.15.71:0/97007527' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-11T07:11:30.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:30 smithi071 ceph-mon[27013]: Found migration_current of "None". Setting to last migration. 2024-09-11T07:11:30.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:30 smithi071 ceph-mon[27013]: mgrmap e7: y(active, since 1.00531s) 2024-09-11T07:11:31.340 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:31 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: 2024-09-11T07:11:31.053+0000 7fa5ef368640 -1 log_channel(cephadm) log [ERR] : [11/Sep/2024:07:11:31] ENGINE Error in HTTPServer.serve 2024-09-11T07:11:31.340 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:31 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: Traceback (most recent call last): 2024-09-11T07:11:31.340 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:31 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T07:11:31.340 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:31 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: self._connections.run(self.expiration_interval) 2024-09-11T07:11:31.340 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:31 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T07:11:31.340 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:31 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: self._run(expiration_interval) 2024-09-11T07:11:31.340 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:31 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T07:11:31.340 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:31 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T07:11:31.340 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:31 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T07:11:31.340 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:31 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T07:11:31.341 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:31 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T07:11:31.341 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:31 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: s = self.context.wrap_socket( 2024-09-11T07:11:31.341 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:31 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T07:11:31.341 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:31 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: return self.sslsocket_class._create( 2024-09-11T07:11:31.341 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:31 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T07:11:31.341 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:31 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: self.do_handshake() 2024-09-11T07:11:31.341 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:31 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T07:11:31.341 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:31 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: self._sslobj.do_handshake() 2024-09-11T07:11:31.341 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:31 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T07:11:31.341 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:31 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: 2024-09-11T07:11:32.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:32 smithi071 ceph-mon[27013]: [11/Sep/2024:07:11:30] ENGINE Bus STARTING 2024-09-11T07:11:32.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:32 smithi071 ceph-mon[27013]: [11/Sep/2024:07:11:31] ENGINE Serving on https://172.21.15.71:7150 2024-09-11T07:11:32.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:32 smithi071 ceph-mon[27013]: [11/Sep/2024:07:11:31] ENGINE Bus STARTED 2024-09-11T07:11:32.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:32 smithi071 ceph-mon[27013]: [11/Sep/2024:07:11:31] ENGINE Error in HTTPServer.serve 2024-09-11T07:11:32.340 INFO:journalctl@ceph.mon.a.smithi071.stdout: Traceback (most recent call last): 2024-09-11T07:11:32.340 INFO:journalctl@ceph.mon.a.smithi071.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T07:11:32.340 INFO:journalctl@ceph.mon.a.smithi071.stdout: self._connections.run(self.expiration_interval) 2024-09-11T07:11:32.340 INFO:journalctl@ceph.mon.a.smithi071.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T07:11:32.340 INFO:journalctl@ceph.mon.a.smithi071.stdout: self._run(expiration_interval) 2024-09-11T07:11:32.340 INFO:journalctl@ceph.mon.a.smithi071.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T07:11:32.340 INFO:journalctl@ceph.mon.a.smithi071.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T07:11:32.340 INFO:journalctl@ceph.mon.a.smithi071.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T07:11:32.340 INFO:journalctl@ceph.mon.a.smithi071.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T07:11:32.340 INFO:journalctl@ceph.mon.a.smithi071.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T07:11:32.340 INFO:journalctl@ceph.mon.a.smithi071.stdout: s = self.context.wrap_socket( 2024-09-11T07:11:32.340 INFO:journalctl@ceph.mon.a.smithi071.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T07:11:32.341 INFO:journalctl@ceph.mon.a.smithi071.stdout: return self.sslsocket_class._create( 2024-09-11T07:11:32.341 INFO:journalctl@ceph.mon.a.smithi071.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T07:11:32.341 INFO:journalctl@ceph.mon.a.smithi071.stdout: self.do_handshake() 2024-09-11T07:11:32.341 INFO:journalctl@ceph.mon.a.smithi071.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T07:11:32.341 INFO:journalctl@ceph.mon.a.smithi071.stdout: self._sslobj.do_handshake() 2024-09-11T07:11:32.341 INFO:journalctl@ceph.mon.a.smithi071.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T07:11:32.341 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:32 smithi071 ceph-mon[27013]: from='mgr.14118 v1:172.21.15.71:0/97007527' entity='mgr.y' 2024-09-11T07:11:32.341 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:32 smithi071 ceph-mon[27013]: from='mgr.14118 v1:172.21.15.71:0/97007527' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:11:33.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:33 smithi071 ceph-mon[27013]: mgrmap e8: y(active, since 2s) 2024-09-11T07:11:34.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:34 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/4232251115' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-11T07:11:35.502 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout { 2024-09-11T07:11:35.502 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "epoch": 8, 2024-09-11T07:11:35.503 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-11T07:11:35.503 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "active_name": "y", 2024-09-11T07:11:35.503 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-11T07:11:35.503 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout } 2024-09-11T07:11:35.503 INFO:teuthology.orchestra.run.smithi071.stdout:Waiting for the mgr to restart... 2024-09-11T07:11:35.503 INFO:teuthology.orchestra.run.smithi071.stdout:Waiting for mgr epoch 8... 2024-09-11T07:11:36.499 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout { 2024-09-11T07:11:36.499 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 8, 2024-09-11T07:11:36.499 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-11T07:11:36.500 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout } 2024-09-11T07:11:36.500 INFO:teuthology.orchestra.run.smithi071.stdout:mgr epoch 8 is available 2024-09-11T07:11:36.500 INFO:teuthology.orchestra.run.smithi071.stdout:Setting orchestrator backend to cephadm... 2024-09-11T07:11:37.063 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:36 smithi071 ceph-mon[27013]: from='client.14128 v1:172.21.15.71:0/2054116864' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-11T07:11:37.063 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:36 smithi071 ceph-mon[27013]: from='client.14128 v1:172.21.15.71:0/2054116864' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-11T07:11:38.342 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:38 smithi071 ceph-mon[27013]: from='client.14130 v1:172.21.15.71:0/1661063526' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:11:38.342 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:38 smithi071 ceph-mon[27013]: from='mgr.14118 v1:172.21.15.71:0/97007527' entity='mgr.y' 2024-09-11T07:11:38.342 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:38 smithi071 ceph-mon[27013]: from='mgr.14118 v1:172.21.15.71:0/97007527' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:11:38.736 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-11T07:11:38.736 INFO:teuthology.orchestra.run.smithi071.stdout:Generating ssh key... 2024-09-11T07:11:39.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:39 smithi071 ceph-mon[27013]: from='client.14132 v1:172.21.15.71:0/2183160632' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:11:39.806 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:39 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: Generating public/private rsa key pair. 2024-09-11T07:11:39.807 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:39 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: Your identification has been saved in /tmp/tmpsx8alhmj/key 2024-09-11T07:11:39.807 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:39 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: Your public key has been saved in /tmp/tmpsx8alhmj/key.pub 2024-09-11T07:11:39.807 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:39 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: The key fingerprint is: 2024-09-11T07:11:39.807 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:39 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: SHA256:3Yt6qM7opxUzJ0XRqgGvFgfFCvtmxExxMB9OpPS85d8 ceph-a81cffe6-700c-11ef-bcea-c7b262605968 2024-09-11T07:11:39.807 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:39 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: The key's randomart image is: 2024-09-11T07:11:39.807 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:39 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: +---[RSA 3072]----+ 2024-09-11T07:11:39.807 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:39 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: | =** oo | 2024-09-11T07:11:39.807 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:39 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: | ..+X.o . | 2024-09-11T07:11:39.807 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:39 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: | *.== o. | 2024-09-11T07:11:39.807 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:39 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: | . * +=.. . | 2024-09-11T07:11:39.807 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:39 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: | o +*oS . . | 2024-09-11T07:11:39.807 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:39 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: | * .* . o . | 2024-09-11T07:11:39.807 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:39 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: | + . .o E | 2024-09-11T07:11:39.808 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:39 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: | +. ... | 2024-09-11T07:11:39.808 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:39 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: | .+++... | 2024-09-11T07:11:39.808 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:39 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: +----[SHA256]-----+ 2024-09-11T07:11:40.765 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:40 smithi071 ceph-mon[27013]: from='client.14134 v1:172.21.15.71:0/2907646939' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:11:40.765 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:40 smithi071 ceph-mon[27013]: Generating ssh key... 2024-09-11T07:11:40.765 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:40 smithi071 ceph-mon[27013]: from='mgr.14118 v1:172.21.15.71:0/97007527' entity='mgr.y' 2024-09-11T07:11:40.765 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:40 smithi071 ceph-mon[27013]: from='mgr.14118 v1:172.21.15.71:0/97007527' entity='mgr.y' 2024-09-11T07:11:41.649 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCq057JKHuLEMKGMKM1VFEthlHBNprIh7HO1VP02lOx8wzsmsB2fswV00dzDSKlqJ9L2tqDQ5SKQBtBNgr3dPn+UArG3qEMDMItGTV+Wg563E6TTEgXlHtwl9pawKCkIK75GQXUJcQI68vPpbeR2i2HF1HJ3yUSJNue8seEIf5Spbc+/D5AKv6xJyLmvaSY+grHy2hkrCQ4zGqzuMiiArHr74759CCqSQqWOPMOhX5eMLo730uV1Z8LtaUybvN6n0AZbOt0i3rAzs1FEek5l6Fc/msTIUJSKRV70njG0cGnk6IHP+G0sNlfPi5yUH6mG0LSUutQEabyIORkFR7qwpwxTwMFuDYZq80wDbePG5/2TEQ7Y8uP0+RGHHCexGs99Ngps4xkJiBVGykt8jHcmfewchymJZbEtMAjB5g+RXtdO0S1BOygHBdrECz9EDyiYTZhJjM7QUl8CtLJTq+9cLL5T+sU5bVXtO1V7AMJUKF+0aYHUN9dg1BCuJuZoWN2z90= ceph-a81cffe6-700c-11ef-bcea-c7b262605968 2024-09-11T07:11:41.649 INFO:teuthology.orchestra.run.smithi071.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-11T07:11:41.649 INFO:teuthology.orchestra.run.smithi071.stdout:Adding key to root@localhost authorized_keys... 2024-09-11T07:11:41.655 INFO:teuthology.orchestra.run.smithi071.stdout:Adding host smithi071... 2024-09-11T07:11:41.907 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:41 smithi071 ceph-mon[27013]: from='client.14136 v1:172.21.15.71:0/561140534' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:11:43.031 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:42 smithi071 ceph-mon[27013]: from='client.14138 v1:172.21.15.71:0/854506834' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi071", "addr": "172.21.15.71", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:11:43.954 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:43 smithi071 ceph-mon[27013]: Deploying cephadm binary to smithi071 2024-09-11T07:11:45.489 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout Added host 'smithi071' with addr '172.21.15.71' 2024-09-11T07:11:45.490 INFO:teuthology.orchestra.run.smithi071.stdout:Deploying unmanaged mon service... 2024-09-11T07:11:46.112 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:45 smithi071 ceph-mon[27013]: from='mgr.14118 v1:172.21.15.71:0/97007527' entity='mgr.y' 2024-09-11T07:11:46.113 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:45 smithi071 ceph-mon[27013]: Added host smithi071 2024-09-11T07:11:46.113 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:45 smithi071 ceph-mon[27013]: from='mgr.14118 v1:172.21.15.71:0/97007527' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:11:46.832 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-11T07:11:46.832 INFO:teuthology.orchestra.run.smithi071.stdout:Deploying unmanaged mgr service... 2024-09-11T07:11:47.409 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:47 smithi071 ceph-mon[27013]: from='client.14140 v1:172.21.15.71:0/1948717358' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:11:47.410 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:47 smithi071 ceph-mon[27013]: Saving service mon spec with placement count:5 2024-09-11T07:11:47.410 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:47 smithi071 ceph-mon[27013]: from='mgr.14118 v1:172.21.15.71:0/97007527' entity='mgr.y' 2024-09-11T07:11:48.742 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:48 smithi071 ceph-mon[27013]: from='client.14142 v1:172.21.15.71:0/3009091918' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:11:48.743 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:48 smithi071 ceph-mon[27013]: Saving service mgr spec with placement count:2 2024-09-11T07:11:48.743 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:48 smithi071 ceph-mon[27013]: from='mgr.14118 v1:172.21.15.71:0/97007527' entity='mgr.y' 2024-09-11T07:11:48.743 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:48 smithi071 ceph-mon[27013]: from='mgr.14118 v1:172.21.15.71:0/97007527' entity='mgr.y' 2024-09-11T07:11:48.745 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-11T07:11:50.452 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:50 smithi071 ceph-mon[27013]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:11:50.452 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:50 smithi071 ceph-mon[27013]: from='mgr.14118 v1:172.21.15.71:0/97007527' entity='mgr.y' 2024-09-11T07:11:50.452 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:50 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/1472415152' entity='client.admin' 2024-09-11T07:11:51.644 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:51 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/491194987' entity='client.admin' 2024-09-11T07:11:51.644 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:51 smithi071 ceph-mon[27013]: from='mgr.14118 v1:172.21.15.71:0/97007527' entity='mgr.y' 2024-09-11T07:11:51.646 INFO:teuthology.orchestra.run.smithi071.stdout:Enabling the dashboard module... 2024-09-11T07:11:52.717 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:52 smithi071 ceph-mon[27013]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:11:52.717 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:52 smithi071 ceph-mon[27013]: from='mgr.14118 v1:172.21.15.71:0/97007527' entity='mgr.y' 2024-09-11T07:11:52.717 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:52 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/1013549224' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-09-11T07:11:53.385 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:53 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: ignoring --setuser ceph since I am not root 2024-09-11T07:11:53.385 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:53 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: ignoring --setgroup ceph since I am not root 2024-09-11T07:11:53.386 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:53 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: 2024-09-11T07:11:53.247+0000 7fc07010bec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-11T07:11:53.386 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:53 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: 2024-09-11T07:11:53.303+0000 7fc07010bec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-11T07:11:53.386 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:53 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: 2024-09-11T07:11:53.367+0000 7fc07010bec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-11T07:11:53.681 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:53 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: 2024-09-11T07:11:53.492+0000 7fc07010bec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-11T07:11:53.681 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:53 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: 2024-09-11T07:11:53.629+0000 7fc07010bec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-11T07:11:53.956 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:53 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: 2024-09-11T07:11:53.893+0000 7fc07010bec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-11T07:11:53.956 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:53 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: 2024-09-11T07:11:53.952+0000 7fc07010bec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-11T07:11:54.286 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:54 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/1013549224' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-11T07:11:54.286 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:54 smithi071 ceph-mon[27013]: mgrmap e9: y(active, since 24s) 2024-09-11T07:11:54.286 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:54 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: 2024-09-11T07:11:54.127+0000 7fc07010bec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-11T07:11:55.299 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:55 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/3101445631' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-11T07:11:55.299 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:54 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: 2024-09-11T07:11:54.995+0000 7fc07010bec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-11T07:11:55.300 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:55 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: 2024-09-11T07:11:55.060+0000 7fc07010bec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-11T07:11:55.302 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout { 2024-09-11T07:11:55.302 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-09-11T07:11:55.302 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-11T07:11:55.302 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "active_name": "y", 2024-09-11T07:11:55.302 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-11T07:11:55.303 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout } 2024-09-11T07:11:55.303 INFO:teuthology.orchestra.run.smithi071.stdout:Waiting for the mgr to restart... 2024-09-11T07:11:55.303 INFO:teuthology.orchestra.run.smithi071.stdout:Waiting for mgr epoch 9... 2024-09-11T07:11:55.902 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:55 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: 2024-09-11T07:11:55.788+0000 7fc07010bec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-11T07:11:55.902 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:55 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: 2024-09-11T07:11:55.899+0000 7fc07010bec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-11T07:11:56.205 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:56 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: 2024-09-11T07:11:56.019+0000 7fc07010bec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-11T07:11:56.206 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:56 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: 2024-09-11T07:11:56.078+0000 7fc07010bec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-11T07:11:56.206 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:56 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: 2024-09-11T07:11:56.202+0000 7fc07010bec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-11T07:11:56.464 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:56 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: 2024-09-11T07:11:56.270+0000 7fc07010bec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-11T07:11:56.464 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:56 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: 2024-09-11T07:11:56.328+0000 7fc07010bec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-11T07:11:56.464 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:56 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: 2024-09-11T07:11:56.461+0000 7fc07010bec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-11T07:11:56.728 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:56 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: 2024-09-11T07:11:56.524+0000 7fc07010bec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-11T07:11:56.728 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:56 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: 2024-09-11T07:11:56.587+0000 7fc07010bec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-11T07:11:56.728 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:56 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: 2024-09-11T07:11:56.666+0000 7fc07010bec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-11T07:11:56.728 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:56 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: 2024-09-11T07:11:56.724+0000 7fc07010bec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-11T07:11:57.089 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:56 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: 2024-09-11T07:11:56.782+0000 7fc07010bec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-11T07:11:57.839 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:57 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: 2024-09-11T07:11:57.526+0000 7fc07010bec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-11T07:11:57.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:57 smithi071 ceph-mon[27013]: Active manager daemon y restarted 2024-09-11T07:11:57.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:57 smithi071 ceph-mon[27013]: Activating manager daemon y 2024-09-11T07:11:57.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:57 smithi071 ceph-mon[27013]: osdmap e3: 0 total, 0 up, 0 in 2024-09-11T07:11:57.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:57 smithi071 ceph-mon[27013]: mgrmap e10: y(active, starting, since 0.00322156s) 2024-09-11T07:11:57.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:57 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T07:11:57.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:57 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-11T07:11:57.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:57 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-11T07:11:57.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:57 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-11T07:11:57.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:57 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-11T07:11:57.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:57 smithi071 ceph-mon[27013]: Manager daemon y is now available 2024-09-11T07:11:57.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:57 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:11:57.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:57 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-11T07:11:57.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:57 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-11T07:11:59.090 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:58 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: 2024-09-11T07:11:58.831+0000 7fc020c68640 -1 log_channel(cephadm) log [ERR] : [11/Sep/2024:07:11:58] ENGINE Error in HTTPServer.serve 2024-09-11T07:11:59.090 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:58 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: Traceback (most recent call last): 2024-09-11T07:11:59.090 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:58 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T07:11:59.090 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:58 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: self._connections.run(self.expiration_interval) 2024-09-11T07:11:59.090 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:58 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T07:11:59.090 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:58 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: self._run(expiration_interval) 2024-09-11T07:11:59.090 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:58 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T07:11:59.090 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:58 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T07:11:59.090 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:58 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T07:11:59.090 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:58 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T07:11:59.090 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:58 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T07:11:59.090 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:58 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: s = self.context.wrap_socket( 2024-09-11T07:11:59.091 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:58 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T07:11:59.091 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:58 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: return self.sslsocket_class._create( 2024-09-11T07:11:59.091 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:58 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T07:11:59.091 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:58 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: self.do_handshake() 2024-09-11T07:11:59.091 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:58 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T07:11:59.091 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:58 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: self._sslobj.do_handshake() 2024-09-11T07:11:59.091 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:58 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T07:11:59.091 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:11:58 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: 2024-09-11T07:11:59.371 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout { 2024-09-11T07:11:59.371 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-09-11T07:11:59.371 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-11T07:11:59.372 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout } 2024-09-11T07:11:59.372 INFO:teuthology.orchestra.run.smithi071.stdout:mgr epoch 9 is available 2024-09-11T07:11:59.372 INFO:teuthology.orchestra.run.smithi071.stdout:Generating a dashboard self-signed certificate... 2024-09-11T07:11:59.813 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:59 smithi071 ceph-mon[27013]: mgrmap e11: y(active, since 1.0054s) 2024-09-11T07:11:59.814 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:59 smithi071 ceph-mon[27013]: [11/Sep/2024:07:11:58] ENGINE Bus STARTING 2024-09-11T07:11:59.814 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:59 smithi071 ceph-mon[27013]: [11/Sep/2024:07:11:58] ENGINE Serving on https://172.21.15.71:7150 2024-09-11T07:11:59.814 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:59 smithi071 ceph-mon[27013]: [11/Sep/2024:07:11:58] ENGINE Bus STARTED 2024-09-11T07:11:59.814 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:59 smithi071 ceph-mon[27013]: [11/Sep/2024:07:11:58] ENGINE Error in HTTPServer.serve 2024-09-11T07:11:59.814 INFO:journalctl@ceph.mon.a.smithi071.stdout: Traceback (most recent call last): 2024-09-11T07:11:59.814 INFO:journalctl@ceph.mon.a.smithi071.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T07:11:59.814 INFO:journalctl@ceph.mon.a.smithi071.stdout: self._connections.run(self.expiration_interval) 2024-09-11T07:11:59.814 INFO:journalctl@ceph.mon.a.smithi071.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T07:11:59.814 INFO:journalctl@ceph.mon.a.smithi071.stdout: self._run(expiration_interval) 2024-09-11T07:11:59.814 INFO:journalctl@ceph.mon.a.smithi071.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T07:11:59.814 INFO:journalctl@ceph.mon.a.smithi071.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T07:11:59.815 INFO:journalctl@ceph.mon.a.smithi071.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T07:11:59.815 INFO:journalctl@ceph.mon.a.smithi071.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T07:11:59.815 INFO:journalctl@ceph.mon.a.smithi071.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T07:11:59.815 INFO:journalctl@ceph.mon.a.smithi071.stdout: s = self.context.wrap_socket( 2024-09-11T07:11:59.815 INFO:journalctl@ceph.mon.a.smithi071.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T07:11:59.815 INFO:journalctl@ceph.mon.a.smithi071.stdout: return self.sslsocket_class._create( 2024-09-11T07:11:59.815 INFO:journalctl@ceph.mon.a.smithi071.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T07:11:59.815 INFO:journalctl@ceph.mon.a.smithi071.stdout: self.do_handshake() 2024-09-11T07:11:59.815 INFO:journalctl@ceph.mon.a.smithi071.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T07:11:59.815 INFO:journalctl@ceph.mon.a.smithi071.stdout: self._sslobj.do_handshake() 2024-09-11T07:11:59.815 INFO:journalctl@ceph.mon.a.smithi071.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T07:11:59.815 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:59 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:11:59.815 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:11:59 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:12:00.704 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-09-11T07:12:00.704 INFO:teuthology.orchestra.run.smithi071.stdout:Creating initial admin user... 2024-09-11T07:12:01.279 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:00 smithi071 ceph-mon[27013]: mgrmap e12: y(active, since 2s) 2024-09-11T07:12:01.279 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:00 smithi071 ceph-mon[27013]: from='client.14162 v1:172.21.15.71:0/3773884840' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:12:01.279 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:00 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:12:01.279 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:00 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:12:01.279 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:00 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:12:02.807 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:02 smithi071 ceph-mon[27013]: from='client.14164 v1:172.21.15.71:0/878145405' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:12:02.808 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:02 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:12:02.809 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$Y.L7vBKGqgt1fiP62NReeuPOIxBGdEF/ID0QAeMtyn.lV//8BGuwm", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1726038721, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-11T07:12:02.809 INFO:teuthology.orchestra.run.smithi071.stdout:Fetching dashboard port number... 2024-09-11T07:12:03.697 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:03 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/413343730' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-09-11T07:12:03.958 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout 8443 2024-09-11T07:12:03.970 INFO:teuthology.orchestra.run.smithi071.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-11T07:12:03.970 INFO:teuthology.orchestra.run.smithi071.stdout:systemctl: stdout disabled 2024-09-11T07:12:03.978 INFO:teuthology.orchestra.run.smithi071.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-11T07:12:03.978 INFO:teuthology.orchestra.run.smithi071.stdout:systemctl: stdout inactive 2024-09-11T07:12:03.978 INFO:teuthology.orchestra.run.smithi071.stdout:firewalld.service is not enabled 2024-09-11T07:12:03.978 INFO:teuthology.orchestra.run.smithi071.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-11T07:12:03.980 INFO:teuthology.orchestra.run.smithi071.stdout:Ceph Dashboard is now available at: 2024-09-11T07:12:03.980 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T07:12:03.980 INFO:teuthology.orchestra.run.smithi071.stdout: URL: https://smithi071.front.sepia.ceph.com:8443/ 2024-09-11T07:12:03.980 INFO:teuthology.orchestra.run.smithi071.stdout: User: admin 2024-09-11T07:12:03.980 INFO:teuthology.orchestra.run.smithi071.stdout: Password: igoroabdy9 2024-09-11T07:12:03.980 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T07:12:03.981 INFO:teuthology.orchestra.run.smithi071.stdout:Saving cluster configuration to /var/lib/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/config directory 2024-09-11T07:12:03.981 INFO:teuthology.orchestra.run.smithi071.stdout:Enabling autotune for osd_memory_target 2024-09-11T07:12:04.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:04 smithi071 ceph-mon[27013]: mgrmap e13: y(active, since 6s) 2024-09-11T07:12:06.340 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-11T07:12:06.340 INFO:teuthology.orchestra.run.smithi071.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-11T07:12:06.340 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T07:12:06.340 INFO:teuthology.orchestra.run.smithi071.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid a81cffe6-700c-11ef-bcea-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-11T07:12:06.340 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T07:12:06.340 INFO:teuthology.orchestra.run.smithi071.stdout:Or, if you are only running a single cluster on this host: 2024-09-11T07:12:06.340 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T07:12:06.340 INFO:teuthology.orchestra.run.smithi071.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-09-11T07:12:06.340 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T07:12:06.340 INFO:teuthology.orchestra.run.smithi071.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-11T07:12:06.340 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T07:12:06.341 INFO:teuthology.orchestra.run.smithi071.stdout: ceph telemetry on 2024-09-11T07:12:06.341 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T07:12:06.341 INFO:teuthology.orchestra.run.smithi071.stdout:For more information see: 2024-09-11T07:12:06.341 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T07:12:06.341 INFO:teuthology.orchestra.run.smithi071.stdout: https://docs.ceph.com/docs/master/mgr/telemetry/ 2024-09-11T07:12:06.341 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T07:12:06.341 INFO:teuthology.orchestra.run.smithi071.stdout:Bootstrap complete. 2024-09-11T07:12:06.381 INFO:tasks.cephadm:Fetching config... 2024-09-11T07:12:06.382 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-11T07:12:06.382 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-11T07:12:06.399 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-11T07:12:06.399 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-11T07:12:06.399 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-11T07:12:06.454 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-11T07:12:06.455 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-11T07:12:06.455 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd if=/var/lib/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/mon.a/keyring of=/dev/stdout 2024-09-11T07:12:06.519 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-11T07:12:06.519 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-11T07:12:06.519 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-11T07:12:06.577 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-11T07:12:06.577 DEBUG:teuthology.orchestra.run.smithi071:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCq057JKHuLEMKGMKM1VFEthlHBNprIh7HO1VP02lOx8wzsmsB2fswV00dzDSKlqJ9L2tqDQ5SKQBtBNgr3dPn+UArG3qEMDMItGTV+Wg563E6TTEgXlHtwl9pawKCkIK75GQXUJcQI68vPpbeR2i2HF1HJ3yUSJNue8seEIf5Spbc+/D5AKv6xJyLmvaSY+grHy2hkrCQ4zGqzuMiiArHr74759CCqSQqWOPMOhX5eMLo730uV1Z8LtaUybvN6n0AZbOt0i3rAzs1FEek5l6Fc/msTIUJSKRV70njG0cGnk6IHP+G0sNlfPi5yUH6mG0LSUutQEabyIORkFR7qwpwxTwMFuDYZq80wDbePG5/2TEQ7Y8uP0+RGHHCexGs99Ngps4xkJiBVGykt8jHcmfewchymJZbEtMAjB5g+RXtdO0S1BOygHBdrECz9EDyiYTZhJjM7QUl8CtLJTq+9cLL5T+sU5bVXtO1V7AMJUKF+0aYHUN9dg1BCuJuZoWN2z90= ceph-a81cffe6-700c-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-11T07:12:06.650 INFO:teuthology.orchestra.run.smithi071.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCq057JKHuLEMKGMKM1VFEthlHBNprIh7HO1VP02lOx8wzsmsB2fswV00dzDSKlqJ9L2tqDQ5SKQBtBNgr3dPn+UArG3qEMDMItGTV+Wg563E6TTEgXlHtwl9pawKCkIK75GQXUJcQI68vPpbeR2i2HF1HJ3yUSJNue8seEIf5Spbc+/D5AKv6xJyLmvaSY+grHy2hkrCQ4zGqzuMiiArHr74759CCqSQqWOPMOhX5eMLo730uV1Z8LtaUybvN6n0AZbOt0i3rAzs1FEek5l6Fc/msTIUJSKRV70njG0cGnk6IHP+G0sNlfPi5yUH6mG0LSUutQEabyIORkFR7qwpwxTwMFuDYZq80wDbePG5/2TEQ7Y8uP0+RGHHCexGs99Ngps4xkJiBVGykt8jHcmfewchymJZbEtMAjB5g+RXtdO0S1BOygHBdrECz9EDyiYTZhJjM7QUl8CtLJTq+9cLL5T+sU5bVXtO1V7AMJUKF+0aYHUN9dg1BCuJuZoWN2z90= ceph-a81cffe6-700c-11ef-bcea-c7b262605968 2024-09-11T07:12:06.661 DEBUG:teuthology.orchestra.run.smithi196:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCq057JKHuLEMKGMKM1VFEthlHBNprIh7HO1VP02lOx8wzsmsB2fswV00dzDSKlqJ9L2tqDQ5SKQBtBNgr3dPn+UArG3qEMDMItGTV+Wg563E6TTEgXlHtwl9pawKCkIK75GQXUJcQI68vPpbeR2i2HF1HJ3yUSJNue8seEIf5Spbc+/D5AKv6xJyLmvaSY+grHy2hkrCQ4zGqzuMiiArHr74759CCqSQqWOPMOhX5eMLo730uV1Z8LtaUybvN6n0AZbOt0i3rAzs1FEek5l6Fc/msTIUJSKRV70njG0cGnk6IHP+G0sNlfPi5yUH6mG0LSUutQEabyIORkFR7qwpwxTwMFuDYZq80wDbePG5/2TEQ7Y8uP0+RGHHCexGs99Ngps4xkJiBVGykt8jHcmfewchymJZbEtMAjB5g+RXtdO0S1BOygHBdrECz9EDyiYTZhJjM7QUl8CtLJTq+9cLL5T+sU5bVXtO1V7AMJUKF+0aYHUN9dg1BCuJuZoWN2z90= ceph-a81cffe6-700c-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-11T07:12:06.781 INFO:teuthology.orchestra.run.smithi196.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCq057JKHuLEMKGMKM1VFEthlHBNprIh7HO1VP02lOx8wzsmsB2fswV00dzDSKlqJ9L2tqDQ5SKQBtBNgr3dPn+UArG3qEMDMItGTV+Wg563E6TTEgXlHtwl9pawKCkIK75GQXUJcQI68vPpbeR2i2HF1HJ3yUSJNue8seEIf5Spbc+/D5AKv6xJyLmvaSY+grHy2hkrCQ4zGqzuMiiArHr74759CCqSQqWOPMOhX5eMLo730uV1Z8LtaUybvN6n0AZbOt0i3rAzs1FEek5l6Fc/msTIUJSKRV70njG0cGnk6IHP+G0sNlfPi5yUH6mG0LSUutQEabyIORkFR7qwpwxTwMFuDYZq80wDbePG5/2TEQ7Y8uP0+RGHHCexGs99Ngps4xkJiBVGykt8jHcmfewchymJZbEtMAjB5g+RXtdO0S1BOygHBdrECz9EDyiYTZhJjM7QUl8CtLJTq+9cLL5T+sU5bVXtO1V7AMJUKF+0aYHUN9dg1BCuJuZoWN2z90= ceph-a81cffe6-700c-11ef-bcea-c7b262605968 2024-09-11T07:12:06.794 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a81cffe6-700c-11ef-bcea-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-11T07:12:06.887 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:06 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/3958065341' entity='client.admin' 2024-09-11T07:12:07.054 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:12:09.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:09 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:12:09.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:09 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:12:09.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:09 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi071", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:12:09.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:09 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:12:09.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:09 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:12:09.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:09 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:12:09.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:09 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/834958488' entity='client.admin' 2024-09-11T07:12:09.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:09 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:12:09.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:09 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:12:10.173 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-11T07:12:10.173 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a81cffe6-700c-11ef-bcea-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-11T07:12:10.747 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:12:13.169 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi196 2024-09-11T07:12:13.170 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-11T07:12:13.170 DEBUG:teuthology.orchestra.run.smithi196:> dd of=/etc/ceph/ceph.conf 2024-09-11T07:12:13.189 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-11T07:12:13.189 DEBUG:teuthology.orchestra.run.smithi196:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-11T07:12:13.247 INFO:tasks.cephadm:Adding host smithi196 to orchestrator... 2024-09-11T07:12:13.247 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a81cffe6-700c-11ef-bcea-c7b262605968 -- ceph orch host add smithi196 2024-09-11T07:12:13.489 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:12:13.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:13 smithi071 ceph-mon[27013]: from='client.14174 v1:172.21.15.71:0/3874399237' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:12:13.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:13 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:12:13.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:13 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:12:13.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:13 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:12:13.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:13 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:12:13.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:13 smithi071 ceph-mon[27013]: Updating smithi071:/etc/ceph/ceph.conf 2024-09-11T07:12:13.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:13 smithi071 ceph-mon[27013]: Updating smithi071:/var/lib/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T07:12:13.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:13 smithi071 ceph-mon[27013]: Updating smithi071:/etc/ceph/ceph.client.admin.keyring 2024-09-11T07:12:13.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:13 smithi071 ceph-mon[27013]: Updating smithi071:/var/lib/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T07:12:13.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:13 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:12:13.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:13 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:12:13.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:13 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:12:16.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:16 smithi071 ceph-mon[27013]: from='client.14176 v1:172.21.15.71:0/3186029323' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi196", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:12:17.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:17 smithi071 ceph-mon[27013]: Deploying cephadm binary to smithi196 2024-09-11T07:12:17.950 INFO:teuthology.orchestra.run.smithi071.stdout:Added host 'smithi196' with addr '172.21.15.196' 2024-09-11T07:12:18.543 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a81cffe6-700c-11ef-bcea-c7b262605968 -- ceph orch host ls --format=json 2024-09-11T07:12:18.778 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:12:19.178 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:18 smithi071 ceph-mon[27013]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:12:19.178 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:18 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:12:19.178 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:18 smithi071 ceph-mon[27013]: Added host smithi196 2024-09-11T07:12:19.178 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:18 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:12:19.179 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:18 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:12:19.179 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:18 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:12:20.509 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T07:12:20.509 INFO:teuthology.orchestra.run.smithi071.stdout:[{"addr": "172.21.15.71", "hostname": "smithi071", "labels": [], "status": ""}, {"addr": "172.21.15.196", "hostname": "smithi196", "labels": [], "status": ""}] 2024-09-11T07:12:20.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:20 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:12:20.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:20 smithi071 ceph-mon[27013]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:12:21.079 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-11T07:12:21.079 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a81cffe6-700c-11ef-bcea-c7b262605968 -- ceph osd crush tunables default 2024-09-11T07:12:21.313 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:12:21.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:21 smithi071 ceph-mon[27013]: from='client.14178 v1:172.21.15.71:0/403190124' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T07:12:23.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:23 smithi071 ceph-mon[27013]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:12:23.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:23 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:12:23.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:23 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:12:23.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:23 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:12:23.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:23 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:12:23.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:23 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi196", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:12:23.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:23 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:12:23.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:23 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:12:23.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:23 smithi071 ceph-mon[27013]: Updating smithi196:/etc/ceph/ceph.conf 2024-09-11T07:12:23.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:23 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:12:23.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:23 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:12:23.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:23 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:12:23.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:23 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/2939078610' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-11T07:12:23.952 INFO:teuthology.orchestra.run.smithi071.stderr:adjusted tunables profile to default 2024-09-11T07:12:24.278 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:24 smithi071 ceph-mon[27013]: Updating smithi196:/var/lib/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T07:12:24.278 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:24 smithi071 ceph-mon[27013]: Updating smithi196:/etc/ceph/ceph.client.admin.keyring 2024-09-11T07:12:24.278 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:24 smithi071 ceph-mon[27013]: Updating smithi196:/var/lib/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T07:12:24.278 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:24 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/2939078610' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-11T07:12:24.278 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:24 smithi071 ceph-mon[27013]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T07:12:24.590 INFO:tasks.cephadm:Adding mon.a on smithi071 2024-09-11T07:12:24.590 INFO:tasks.cephadm:Adding mon.c on smithi071 2024-09-11T07:12:24.591 INFO:tasks.cephadm:Adding mon.b on smithi196 2024-09-11T07:12:24.591 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a81cffe6-700c-11ef-bcea-c7b262605968 -- ceph orch apply mon '3;smithi071:[v1:172.21.15.71:6789]=a;smithi071:[v1:172.21.15.71:6790]=c;smithi196:[v1:172.21.15.196:6789]=b' 2024-09-11T07:12:24.833 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T07:12:25.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:25 smithi071 ceph-mon[27013]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:12:26.496 INFO:teuthology.orchestra.run.smithi196.stdout:Scheduled mon update... 2024-09-11T07:12:27.143 DEBUG:teuthology.orchestra.run.smithi071:mon.c> sudo journalctl -f -n 0 -u ceph-a81cffe6-700c-11ef-bcea-c7b262605968@mon.c.service 2024-09-11T07:12:27.146 DEBUG:teuthology.orchestra.run.smithi196:mon.b> sudo journalctl -f -n 0 -u ceph-a81cffe6-700c-11ef-bcea-c7b262605968@mon.b.service 2024-09-11T07:12:27.148 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-11T07:12:27.148 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a81cffe6-700c-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-11T07:12:27.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:27 smithi071 ceph-mon[27013]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:12:27.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:27 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:12:27.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:27 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:12:27.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:27 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:12:27.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:27 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:12:27.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:27 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:12:27.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:27 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T07:12:27.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:27 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:12:27.792 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T07:12:28.490 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:28 smithi071 ceph-mon[27013]: from='client.14182 v1:172.21.15.196:0/1836429628' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi071:[v1:172.21.15.71:6789]=a;smithi071:[v1:172.21.15.71:6790]=c;smithi196:[v1:172.21.15.196:6789]=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:12:28.491 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:28 smithi071 ceph-mon[27013]: Saving service mon spec with placement smithi071:[v1:172.21.15.71:6789]=a;smithi071:[v1:172.21.15.71:6790]=c;smithi196:[v1:172.21.15.196:6789]=b;count:3 2024-09-11T07:12:28.491 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:28 smithi071 ceph-mon[27013]: Deploying daemon mon.b on smithi196 2024-09-11T07:12:29.504 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:29 smithi071 ceph-mon[27013]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:12:31.021 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-11T07:12:31.022 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":1,"fsid":"a81cffe6-700c-11ef-bcea-c7b262605968","modified":"2024-09-11T07:11:05.750744Z","created":"2024-09-11T07:11:05.750744Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.71:6789","nonce":0}]},"addr":"172.21.15.71:6789/0","public_addr":"172.21.15.71:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-11T07:12:31.022 INFO:teuthology.orchestra.run.smithi196.stderr:dumped monmap epoch 1 2024-09-11T07:12:31.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:31 smithi071 ceph-mon[27013]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:12:31.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:31 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.196:0/255569525' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T07:12:31.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:31 smithi196 systemd[1]: Started Ceph mon.b for a81cffe6-700c-11ef-bcea-c7b262605968. 2024-09-11T07:12:33.065 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-11T07:12:33.065 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a81cffe6-700c-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-11T07:12:33.323 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:12:36.776 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-11T07:12:36.776 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":2,"fsid":"a81cffe6-700c-11ef-bcea-c7b262605968","modified":"2024-09-11T07:12:31.526889Z","created":"2024-09-11T07:11:05.750744Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.71:6789","nonce":0}]},"addr":"172.21.15.71:6789/0","public_addr":"172.21.15.71:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.196:6789","nonce":0}]},"addr":"172.21.15.196:6789/0","public_addr":"172.21.15.196:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-11T07:12:36.776 INFO:teuthology.orchestra.run.smithi196.stderr:dumped monmap epoch 2 2024-09-11T07:12:36.972 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:36 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T07:12:36.973 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:36 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:12:36.973 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:36 smithi071 ceph-mon[27013]: mon.a calling monitor election 2024-09-11T07:12:36.973 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:36 smithi071 ceph-mon[27013]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:12:36.973 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:36 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:12:36.973 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:36 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:12:36.973 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:36 smithi071 ceph-mon[27013]: mon.b calling monitor election 2024-09-11T07:12:36.973 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:36 smithi071 ceph-mon[27013]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:12:36.973 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:36 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:12:36.973 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:36 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:12:36.973 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:36 smithi071 ceph-mon[27013]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:12:36.973 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:36 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:12:36.973 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:36 smithi071 ceph-mon[27013]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-09-11T07:12:36.973 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:36 smithi071 ceph-mon[27013]: monmap e2: 2 mons at {a=v1:172.21.15.71:6789/0,b=v1:172.21.15.196:6789/0} removed_ranks: {} disallowed_leaders: {} 2024-09-11T07:12:36.974 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:36 smithi071 ceph-mon[27013]: fsmap 2024-09-11T07:12:36.974 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:36 smithi071 ceph-mon[27013]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T07:12:36.974 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:36 smithi071 ceph-mon[27013]: mgrmap e13: y(active, since 39s) 2024-09-11T07:12:36.974 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:36 smithi071 ceph-mon[27013]: overall HEALTH_OK 2024-09-11T07:12:36.974 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:36 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:12:36.974 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:36 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:12:36.974 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:36 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:12:36.974 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:36 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T07:12:36.974 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:36 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:12:37.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:36 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T07:12:37.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:36 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:12:37.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:36 smithi196 ceph-mon[30773]: mon.a calling monitor election 2024-09-11T07:12:37.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:36 smithi196 ceph-mon[30773]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:12:37.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:36 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:12:37.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:36 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:12:37.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:36 smithi196 ceph-mon[30773]: mon.b calling monitor election 2024-09-11T07:12:37.036 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:36 smithi196 ceph-mon[30773]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:12:37.036 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:36 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:12:37.036 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:36 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:12:37.036 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:36 smithi196 ceph-mon[30773]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:12:37.036 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:36 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:12:37.036 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:36 smithi196 ceph-mon[30773]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-09-11T07:12:37.036 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:36 smithi196 ceph-mon[30773]: monmap e2: 2 mons at {a=v1:172.21.15.71:6789/0,b=v1:172.21.15.196:6789/0} removed_ranks: {} disallowed_leaders: {} 2024-09-11T07:12:37.036 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:36 smithi196 ceph-mon[30773]: fsmap 2024-09-11T07:12:37.036 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:36 smithi196 ceph-mon[30773]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T07:12:37.036 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:36 smithi196 ceph-mon[30773]: mgrmap e13: y(active, since 39s) 2024-09-11T07:12:37.036 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:36 smithi196 ceph-mon[30773]: overall HEALTH_OK 2024-09-11T07:12:37.036 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:36 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:12:37.036 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:36 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:12:37.037 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:36 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:12:37.037 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:36 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T07:12:37.037 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:36 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:12:37.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:37 smithi196 ceph-mon[30773]: Deploying daemon mon.c on smithi071 2024-09-11T07:12:37.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:37 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.196:0/3911402607' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T07:12:37.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:37 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:12:37.884 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:37 smithi071 ceph-mon[27013]: Deploying daemon mon.c on smithi071 2024-09-11T07:12:37.884 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:37 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.196:0/3911402607' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T07:12:37.884 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:37 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:12:38.377 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-11T07:12:38.377 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a81cffe6-700c-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-11T07:12:38.636 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:12:38.755 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:38 smithi196 ceph-mon[30773]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:12:38.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:38 smithi071 ceph-mon[27013]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:12:40.218 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:12:39 smithi071 systemd[1]: Started Ceph mon.c for a81cffe6-700c-11ef-bcea-c7b262605968. 2024-09-11T07:12:45.160 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-09-11T07:12:45.161 INFO:teuthology.orchestra.run.smithi196.stdout:{"epoch":3,"fsid":"a81cffe6-700c-11ef-bcea-c7b262605968","modified":"2024-09-11T07:12:39.889967Z","created":"2024-09-11T07:11:05.750744Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.71:6789","nonce":0}]},"addr":"172.21.15.71:6789/0","public_addr":"172.21.15.71:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.196:6789","nonce":0}]},"addr":"172.21.15.196:6789/0","public_addr":"172.21.15.196:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.71:6790","nonce":0}]},"addr":"172.21.15.71:6790/0","public_addr":"172.21.15.71:6790/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-09-11T07:12:45.161 INFO:teuthology.orchestra.run.smithi196.stderr:dumped monmap epoch 3 2024-09-11T07:12:45.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:44 smithi196 ceph-mon[30773]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:12:45.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:44 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T07:12:45.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:44 smithi196 ceph-mon[30773]: mon.a calling monitor election 2024-09-11T07:12:45.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:44 smithi196 ceph-mon[30773]: mon.b calling monitor election 2024-09-11T07:12:45.286 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:44 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:12:45.286 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:44 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:12:45.286 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:44 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:12:45.286 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:44 smithi196 ceph-mon[30773]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:12:45.286 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:44 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:12:45.286 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:44 smithi196 ceph-mon[30773]: mon.c calling monitor election 2024-09-11T07:12:45.286 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:44 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:12:45.286 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:44 smithi196 ceph-mon[30773]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:12:45.286 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:44 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:12:45.286 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:44 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:12:45.286 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:44 smithi196 ceph-mon[30773]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-09-11T07:12:45.286 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:44 smithi196 ceph-mon[30773]: monmap e3: 3 mons at {a=v1:172.21.15.71:6789/0,b=v1:172.21.15.196:6789/0,c=v1:172.21.15.71:6790/0} removed_ranks: {} disallowed_leaders: {} 2024-09-11T07:12:45.287 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:44 smithi196 ceph-mon[30773]: fsmap 2024-09-11T07:12:45.287 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:44 smithi196 ceph-mon[30773]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T07:12:45.287 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:44 smithi196 ceph-mon[30773]: mgrmap e13: y(active, since 47s) 2024-09-11T07:12:45.287 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:44 smithi196 ceph-mon[30773]: overall HEALTH_OK 2024-09-11T07:12:45.287 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:44 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:12:45.287 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:44 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:12:45.287 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:44 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:12:45.287 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:44 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:12:45.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:44 smithi071 ceph-mon[27013]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:12:45.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:44 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T07:12:45.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:44 smithi071 ceph-mon[27013]: mon.a calling monitor election 2024-09-11T07:12:45.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:44 smithi071 ceph-mon[27013]: mon.b calling monitor election 2024-09-11T07:12:45.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:44 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:12:45.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:44 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:12:45.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:44 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:12:45.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:44 smithi071 ceph-mon[27013]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:12:45.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:44 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:12:45.341 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:44 smithi071 ceph-mon[27013]: mon.c calling monitor election 2024-09-11T07:12:45.341 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:44 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:12:45.341 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:44 smithi071 ceph-mon[27013]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:12:45.341 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:44 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:12:45.341 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:44 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:12:45.341 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:44 smithi071 ceph-mon[27013]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-09-11T07:12:45.341 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:44 smithi071 ceph-mon[27013]: monmap e3: 3 mons at {a=v1:172.21.15.71:6789/0,b=v1:172.21.15.196:6789/0,c=v1:172.21.15.71:6790/0} removed_ranks: {} disallowed_leaders: {} 2024-09-11T07:12:45.341 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:44 smithi071 ceph-mon[27013]: fsmap 2024-09-11T07:12:45.341 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:44 smithi071 ceph-mon[27013]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T07:12:45.341 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:44 smithi071 ceph-mon[27013]: mgrmap e13: y(active, since 47s) 2024-09-11T07:12:45.341 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:44 smithi071 ceph-mon[27013]: overall HEALTH_OK 2024-09-11T07:12:45.341 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:44 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:12:45.342 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:44 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:12:45.342 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:44 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:12:45.342 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:44 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:12:45.342 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:12:44 smithi071 ceph-mon[37727]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:12:45.342 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:12:44 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T07:12:45.342 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:12:44 smithi071 ceph-mon[37727]: mon.a calling monitor election 2024-09-11T07:12:45.342 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:12:44 smithi071 ceph-mon[37727]: mon.b calling monitor election 2024-09-11T07:12:45.342 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:12:44 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:12:45.342 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:12:44 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:12:45.343 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:12:44 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:12:45.343 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:12:44 smithi071 ceph-mon[37727]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:12:45.343 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:12:44 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:12:45.343 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:12:44 smithi071 ceph-mon[37727]: mon.c calling monitor election 2024-09-11T07:12:45.343 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:12:44 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:12:45.343 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:12:44 smithi071 ceph-mon[37727]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:12:45.343 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:12:44 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:12:45.343 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:12:44 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:12:45.343 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:12:44 smithi071 ceph-mon[37727]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-09-11T07:12:45.343 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:12:44 smithi071 ceph-mon[37727]: monmap e3: 3 mons at {a=v1:172.21.15.71:6789/0,b=v1:172.21.15.196:6789/0,c=v1:172.21.15.71:6790/0} removed_ranks: {} disallowed_leaders: {} 2024-09-11T07:12:45.344 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:12:44 smithi071 ceph-mon[37727]: fsmap 2024-09-11T07:12:45.344 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:12:44 smithi071 ceph-mon[37727]: osdmap e4: 0 total, 0 up, 0 in 2024-09-11T07:12:45.344 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:12:44 smithi071 ceph-mon[37727]: mgrmap e13: y(active, since 47s) 2024-09-11T07:12:45.344 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:12:44 smithi071 ceph-mon[37727]: overall HEALTH_OK 2024-09-11T07:12:45.344 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:12:44 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:12:45.344 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:12:44 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:12:45.344 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:12:44 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:12:45.344 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:12:44 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:12:45.788 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-11T07:12:45.789 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a81cffe6-700c-11ef-bcea-c7b262605968 -- ceph config generate-minimal-conf 2024-09-11T07:12:46.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:45 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:12:46.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:45 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.196:0/3933512080' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T07:12:46.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:45 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:12:46.057 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:12:46.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:45 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:12:46.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:45 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.196:0/3933512080' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T07:12:46.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:45 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:12:46.089 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:12:45 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:12:46.089 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:12:45 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.196:0/3933512080' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T07:12:46.089 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:12:45 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:12:47.089 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:12:47 smithi071 ceph-mon[37727]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:12:47.089 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:12:47 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:12:47.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:12:47 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:12:47.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:12:47 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:12:47.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:12:47 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:12:47.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:12:47 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:12:47.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:12:47 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:12:47.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:12:47 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:12:47.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:12:47 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:12:47.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:12:47 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:12:47.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:12:47 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:12:47.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:12:47 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:12:47.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:12:47 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T07:12:47.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:12:47 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T07:12:47.091 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:12:47 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:12:47.091 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:47 smithi071 ceph-mon[27013]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:12:47.091 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:47 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:12:47.091 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:47 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:12:47.091 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:47 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:12:47.091 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:47 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:12:47.091 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:47 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:12:47.091 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:47 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:12:47.091 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:47 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:12:47.091 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:47 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:12:47.091 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:47 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:12:47.092 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:47 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:12:47.092 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:47 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:12:47.092 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:47 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T07:12:47.092 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:47 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T07:12:47.092 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:47 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:12:47.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:47 smithi196 ceph-mon[30773]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:12:47.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:47 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:12:47.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:47 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:12:47.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:47 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:12:47.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:47 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:12:47.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:47 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:12:47.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:47 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:12:47.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:47 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:12:47.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:47 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:12:47.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:47 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:12:47.536 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:47 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:12:47.536 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:47 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:12:47.536 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:47 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T07:12:47.536 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:47 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T07:12:47.536 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:47 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:12:47.798 INFO:teuthology.orchestra.run.smithi071.stdout:# minimal ceph.conf for a81cffe6-700c-11ef-bcea-c7b262605968 2024-09-11T07:12:47.799 INFO:teuthology.orchestra.run.smithi071.stdout:[global] 2024-09-11T07:12:47.799 INFO:teuthology.orchestra.run.smithi071.stdout: fsid = a81cffe6-700c-11ef-bcea-c7b262605968 2024-09-11T07:12:47.799 INFO:teuthology.orchestra.run.smithi071.stdout: mon_host = 172.21.15.71:6789/0 172.21.15.196:6789/0 v1:172.21.15.71:6790/0 2024-09-11T07:12:48.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:48 smithi071 ceph-mon[27013]: Updating smithi071:/etc/ceph/ceph.conf 2024-09-11T07:12:48.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:48 smithi071 ceph-mon[27013]: Updating smithi196:/etc/ceph/ceph.conf 2024-09-11T07:12:48.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:48 smithi071 ceph-mon[27013]: Updating smithi196:/var/lib/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T07:12:48.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:48 smithi071 ceph-mon[27013]: Updating smithi071:/var/lib/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T07:12:48.341 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:48 smithi071 ceph-mon[27013]: Reconfiguring mon.a (unknown last config time)... 2024-09-11T07:12:48.341 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:48 smithi071 ceph-mon[27013]: Reconfiguring daemon mon.a on smithi071 2024-09-11T07:12:48.341 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:48 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:12:48.341 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:48 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/1502917934' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:12:48.341 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:12:48 smithi071 ceph-mon[37727]: Updating smithi071:/etc/ceph/ceph.conf 2024-09-11T07:12:48.341 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:12:48 smithi071 ceph-mon[37727]: Updating smithi196:/etc/ceph/ceph.conf 2024-09-11T07:12:48.341 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:12:48 smithi071 ceph-mon[37727]: Updating smithi196:/var/lib/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T07:12:48.341 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:12:48 smithi071 ceph-mon[37727]: Updating smithi071:/var/lib/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T07:12:48.341 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:12:48 smithi071 ceph-mon[37727]: Reconfiguring mon.a (unknown last config time)... 2024-09-11T07:12:48.341 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:12:48 smithi071 ceph-mon[37727]: Reconfiguring daemon mon.a on smithi071 2024-09-11T07:12:48.341 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:12:48 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:12:48.342 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:12:48 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/1502917934' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:12:48.373 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-11T07:12:48.373 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-11T07:12:48.374 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/etc/ceph/ceph.conf 2024-09-11T07:12:48.400 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-11T07:12:48.400 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-11T07:12:48.465 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-11T07:12:48.465 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/etc/ceph/ceph.conf 2024-09-11T07:12:48.491 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:48 smithi196 ceph-mon[30773]: Updating smithi071:/etc/ceph/ceph.conf 2024-09-11T07:12:48.491 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:48 smithi196 ceph-mon[30773]: Updating smithi196:/etc/ceph/ceph.conf 2024-09-11T07:12:48.491 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:48 smithi196 ceph-mon[30773]: Updating smithi196:/var/lib/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T07:12:48.491 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:48 smithi196 ceph-mon[30773]: Updating smithi071:/var/lib/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T07:12:48.491 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:48 smithi196 ceph-mon[30773]: Reconfiguring mon.a (unknown last config time)... 2024-09-11T07:12:48.492 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:48 smithi196 ceph-mon[30773]: Reconfiguring daemon mon.a on smithi071 2024-09-11T07:12:48.492 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:48 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:12:48.492 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:48 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/1502917934' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:12:48.496 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-11T07:12:48.496 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-11T07:12:48.563 INFO:tasks.cephadm:Adding mgr.y on smithi071 2024-09-11T07:12:48.563 INFO:tasks.cephadm:Adding mgr.x on smithi196 2024-09-11T07:12:48.563 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a81cffe6-700c-11ef-bcea-c7b262605968 -- ceph orch apply mgr '2;smithi071=y;smithi196=x' 2024-09-11T07:12:48.861 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:12:49.213 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:49 smithi071 ceph-mon[27013]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:12:49.214 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:12:49 smithi071 ceph-mon[37727]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:12:49.241 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:49 smithi196 ceph-mon[30773]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:12:50.509 INFO:teuthology.orchestra.run.smithi196.stdout:Scheduled mgr update... 2024-09-11T07:12:50.509 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:50 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:12:50.509 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:50 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:12:50.509 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:50 smithi196 ceph-mon[30773]: Reconfiguring mon.c (monmap changed)... 2024-09-11T07:12:50.509 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:50 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T07:12:50.509 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:50 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T07:12:50.509 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:50 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:12:50.509 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:50 smithi196 ceph-mon[30773]: Reconfiguring daemon mon.c on smithi071 2024-09-11T07:12:50.510 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:50 smithi196 ceph-mon[30773]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:12:50.520 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:50 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:12:50.520 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:50 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:12:50.520 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:50 smithi071 ceph-mon[27013]: Reconfiguring mon.c (monmap changed)... 2024-09-11T07:12:50.520 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:50 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T07:12:50.520 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:50 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T07:12:50.520 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:50 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:12:50.520 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:50 smithi071 ceph-mon[27013]: Reconfiguring daemon mon.c on smithi071 2024-09-11T07:12:50.521 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:50 smithi071 ceph-mon[27013]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:12:50.521 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:12:50 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:12:50.521 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:12:50 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:12:50.521 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:12:50 smithi071 ceph-mon[37727]: Reconfiguring mon.c (monmap changed)... 2024-09-11T07:12:50.521 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:12:50 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T07:12:50.521 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:12:50 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T07:12:50.521 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:12:50 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:12:50.521 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:12:50 smithi071 ceph-mon[37727]: Reconfiguring daemon mon.c on smithi071 2024-09-11T07:12:50.521 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:12:50 smithi071 ceph-mon[37727]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:12:51.153 DEBUG:teuthology.orchestra.run.smithi196:mgr.x> sudo journalctl -f -n 0 -u ceph-a81cffe6-700c-11ef-bcea-c7b262605968@mgr.x.service 2024-09-11T07:12:51.155 INFO:tasks.cephadm:Deploying OSDs... 2024-09-11T07:12:51.155 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-11T07:12:51.156 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/scratch_devs of=/dev/stdout 2024-09-11T07:12:51.175 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-11T07:12:51.175 DEBUG:teuthology.orchestra.run.smithi071:> stat /dev/vg_nvme/lv_1 2024-09-11T07:12:51.232 INFO:teuthology.orchestra.run.smithi071.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-11T07:12:51.232 INFO:teuthology.orchestra.run.smithi071.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T07:12:51.233 INFO:teuthology.orchestra.run.smithi071.stdout:Device: 5h/5d Inode: 988 Links: 1 2024-09-11T07:12:51.233 INFO:teuthology.orchestra.run.smithi071.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T07:12:51.233 INFO:teuthology.orchestra.run.smithi071.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T07:12:51.233 INFO:teuthology.orchestra.run.smithi071.stdout:Access: 2024-09-11 07:12:39.315100592 +0000 2024-09-11T07:12:51.233 INFO:teuthology.orchestra.run.smithi071.stdout:Modify: 2024-09-11 07:06:55.165392487 +0000 2024-09-11T07:12:51.233 INFO:teuthology.orchestra.run.smithi071.stdout:Change: 2024-09-11 07:06:55.165392487 +0000 2024-09-11T07:12:51.233 INFO:teuthology.orchestra.run.smithi071.stdout: Birth: 2024-09-11 07:06:55.165392487 +0000 2024-09-11T07:12:51.233 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-11T07:12:51.300 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records in 2024-09-11T07:12:51.300 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records out 2024-09-11T07:12:51.300 INFO:teuthology.orchestra.run.smithi071.stderr:512 bytes copied, 0.000271696 s, 1.9 MB/s 2024-09-11T07:12:51.301 DEBUG:teuthology.orchestra.run.smithi071:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-11T07:12:51.359 DEBUG:teuthology.orchestra.run.smithi071:> stat /dev/vg_nvme/lv_2 2024-09-11T07:12:51.415 INFO:teuthology.orchestra.run.smithi071.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-11T07:12:51.415 INFO:teuthology.orchestra.run.smithi071.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T07:12:51.415 INFO:teuthology.orchestra.run.smithi071.stdout:Device: 5h/5d Inode: 983 Links: 1 2024-09-11T07:12:51.415 INFO:teuthology.orchestra.run.smithi071.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T07:12:51.416 INFO:teuthology.orchestra.run.smithi071.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T07:12:51.416 INFO:teuthology.orchestra.run.smithi071.stdout:Access: 2024-09-11 07:12:39.315100592 +0000 2024-09-11T07:12:51.416 INFO:teuthology.orchestra.run.smithi071.stdout:Modify: 2024-09-11 07:06:55.165392487 +0000 2024-09-11T07:12:51.416 INFO:teuthology.orchestra.run.smithi071.stdout:Change: 2024-09-11 07:06:55.165392487 +0000 2024-09-11T07:12:51.416 INFO:teuthology.orchestra.run.smithi071.stdout: Birth: 2024-09-11 07:06:55.165392487 +0000 2024-09-11T07:12:51.416 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-11T07:12:51.481 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records in 2024-09-11T07:12:51.481 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records out 2024-09-11T07:12:51.481 INFO:teuthology.orchestra.run.smithi071.stderr:512 bytes copied, 0.000220551 s, 2.3 MB/s 2024-09-11T07:12:51.482 DEBUG:teuthology.orchestra.run.smithi071:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-11T07:12:51.539 DEBUG:teuthology.orchestra.run.smithi071:> stat /dev/vg_nvme/lv_3 2024-09-11T07:12:51.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:51 smithi071 ceph-mon[27013]: from='client.24106 v1:172.21.15.196:0/2104364324' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi071=y;smithi196=x", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:12:51.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:51 smithi071 ceph-mon[27013]: Saving service mgr spec with placement smithi071=y;smithi196=x;count:2 2024-09-11T07:12:51.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:51 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:12:51.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:51 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:12:51.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:51 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:12:51.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:51 smithi071 ceph-mon[27013]: Reconfiguring mon.b (monmap changed)... 2024-09-11T07:12:51.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:51 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T07:12:51.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:51 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T07:12:51.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:51 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:12:51.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:51 smithi071 ceph-mon[27013]: Reconfiguring daemon mon.b on smithi196 2024-09-11T07:12:51.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:12:51 smithi071 ceph-mon[37727]: from='client.24106 v1:172.21.15.196:0/2104364324' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi071=y;smithi196=x", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:12:51.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:12:51 smithi071 ceph-mon[37727]: Saving service mgr spec with placement smithi071=y;smithi196=x;count:2 2024-09-11T07:12:51.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:12:51 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:12:51.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:12:51 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:12:51.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:12:51 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:12:51.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:12:51 smithi071 ceph-mon[37727]: Reconfiguring mon.b (monmap changed)... 2024-09-11T07:12:51.591 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:12:51 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T07:12:51.591 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:12:51 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T07:12:51.591 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:12:51 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:12:51.591 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:12:51 smithi071 ceph-mon[37727]: Reconfiguring daemon mon.b on smithi196 2024-09-11T07:12:51.595 INFO:teuthology.orchestra.run.smithi071.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-11T07:12:51.595 INFO:teuthology.orchestra.run.smithi071.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T07:12:51.595 INFO:teuthology.orchestra.run.smithi071.stdout:Device: 5h/5d Inode: 972 Links: 1 2024-09-11T07:12:51.595 INFO:teuthology.orchestra.run.smithi071.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T07:12:51.595 INFO:teuthology.orchestra.run.smithi071.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T07:12:51.595 INFO:teuthology.orchestra.run.smithi071.stdout:Access: 2024-09-11 07:12:39.316100573 +0000 2024-09-11T07:12:51.595 INFO:teuthology.orchestra.run.smithi071.stdout:Modify: 2024-09-11 07:06:55.164392506 +0000 2024-09-11T07:12:51.595 INFO:teuthology.orchestra.run.smithi071.stdout:Change: 2024-09-11 07:06:55.164392506 +0000 2024-09-11T07:12:51.595 INFO:teuthology.orchestra.run.smithi071.stdout: Birth: 2024-09-11 07:06:55.164392506 +0000 2024-09-11T07:12:51.596 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-11T07:12:51.632 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:51 smithi196 ceph-mon[30773]: from='client.24106 v1:172.21.15.196:0/2104364324' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi071=y;smithi196=x", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:12:51.632 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:51 smithi196 ceph-mon[30773]: Saving service mgr spec with placement smithi071=y;smithi196=x;count:2 2024-09-11T07:12:51.632 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:51 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:12:51.632 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:51 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:12:51.633 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:51 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:12:51.633 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:51 smithi196 ceph-mon[30773]: Reconfiguring mon.b (monmap changed)... 2024-09-11T07:12:51.633 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:51 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-11T07:12:51.633 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:51 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-11T07:12:51.633 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:51 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:12:51.633 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:51 smithi196 ceph-mon[30773]: Reconfiguring daemon mon.b on smithi196 2024-09-11T07:12:51.659 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records in 2024-09-11T07:12:51.659 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records out 2024-09-11T07:12:51.659 INFO:teuthology.orchestra.run.smithi071.stderr:512 bytes copied, 0.000238348 s, 2.1 MB/s 2024-09-11T07:12:51.660 DEBUG:teuthology.orchestra.run.smithi071:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-11T07:12:51.717 DEBUG:teuthology.orchestra.run.smithi071:> stat /dev/vg_nvme/lv_4 2024-09-11T07:12:51.773 INFO:teuthology.orchestra.run.smithi071.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-11T07:12:51.774 INFO:teuthology.orchestra.run.smithi071.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T07:12:51.774 INFO:teuthology.orchestra.run.smithi071.stdout:Device: 5h/5d Inode: 976 Links: 1 2024-09-11T07:12:51.774 INFO:teuthology.orchestra.run.smithi071.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T07:12:51.774 INFO:teuthology.orchestra.run.smithi071.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T07:12:51.774 INFO:teuthology.orchestra.run.smithi071.stdout:Access: 2024-09-11 07:12:39.316100573 +0000 2024-09-11T07:12:51.774 INFO:teuthology.orchestra.run.smithi071.stdout:Modify: 2024-09-11 07:06:55.164392506 +0000 2024-09-11T07:12:51.774 INFO:teuthology.orchestra.run.smithi071.stdout:Change: 2024-09-11 07:06:55.164392506 +0000 2024-09-11T07:12:51.774 INFO:teuthology.orchestra.run.smithi071.stdout: Birth: 2024-09-11 07:06:55.164392506 +0000 2024-09-11T07:12:51.774 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-11T07:12:51.837 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records in 2024-09-11T07:12:51.837 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records out 2024-09-11T07:12:51.837 INFO:teuthology.orchestra.run.smithi071.stderr:512 bytes copied, 0.000198783 s, 2.6 MB/s 2024-09-11T07:12:51.838 DEBUG:teuthology.orchestra.run.smithi071:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-11T07:12:51.895 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-11T07:12:51.895 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/scratch_devs of=/dev/stdout 2024-09-11T07:12:51.911 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-11T07:12:51.911 DEBUG:teuthology.orchestra.run.smithi196:> stat /dev/vg_nvme/lv_1 2024-09-11T07:12:51.967 INFO:teuthology.orchestra.run.smithi196.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-11T07:12:51.967 INFO:teuthology.orchestra.run.smithi196.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T07:12:51.967 INFO:teuthology.orchestra.run.smithi196.stdout:Device: 5h/5d Inode: 971 Links: 1 2024-09-11T07:12:51.967 INFO:teuthology.orchestra.run.smithi196.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T07:12:51.967 INFO:teuthology.orchestra.run.smithi196.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T07:12:51.967 INFO:teuthology.orchestra.run.smithi196.stdout:Access: 2024-09-11 07:12:30.509254934 +0000 2024-09-11T07:12:51.967 INFO:teuthology.orchestra.run.smithi196.stdout:Modify: 2024-09-11 07:06:56.611515274 +0000 2024-09-11T07:12:51.967 INFO:teuthology.orchestra.run.smithi196.stdout:Change: 2024-09-11 07:06:56.611515274 +0000 2024-09-11T07:12:51.967 INFO:teuthology.orchestra.run.smithi196.stdout: Birth: 2024-09-11 07:06:56.611515274 +0000 2024-09-11T07:12:51.967 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-11T07:12:52.031 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records in 2024-09-11T07:12:52.031 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records out 2024-09-11T07:12:52.031 INFO:teuthology.orchestra.run.smithi196.stderr:512 bytes copied, 0.000248684 s, 2.1 MB/s 2024-09-11T07:12:52.033 DEBUG:teuthology.orchestra.run.smithi196:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-11T07:12:52.090 DEBUG:teuthology.orchestra.run.smithi196:> stat /dev/vg_nvme/lv_2 2024-09-11T07:12:52.147 INFO:teuthology.orchestra.run.smithi196.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-11T07:12:52.147 INFO:teuthology.orchestra.run.smithi196.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T07:12:52.147 INFO:teuthology.orchestra.run.smithi196.stdout:Device: 5h/5d Inode: 979 Links: 1 2024-09-11T07:12:52.148 INFO:teuthology.orchestra.run.smithi196.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T07:12:52.148 INFO:teuthology.orchestra.run.smithi196.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T07:12:52.148 INFO:teuthology.orchestra.run.smithi196.stdout:Access: 2024-09-11 07:12:30.510254912 +0000 2024-09-11T07:12:52.148 INFO:teuthology.orchestra.run.smithi196.stdout:Modify: 2024-09-11 07:06:56.613515230 +0000 2024-09-11T07:12:52.148 INFO:teuthology.orchestra.run.smithi196.stdout:Change: 2024-09-11 07:06:56.613515230 +0000 2024-09-11T07:12:52.148 INFO:teuthology.orchestra.run.smithi196.stdout: Birth: 2024-09-11 07:06:56.613515230 +0000 2024-09-11T07:12:52.148 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-11T07:12:52.211 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records in 2024-09-11T07:12:52.211 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records out 2024-09-11T07:12:52.212 INFO:teuthology.orchestra.run.smithi196.stderr:512 bytes copied, 0.000195218 s, 2.6 MB/s 2024-09-11T07:12:52.213 DEBUG:teuthology.orchestra.run.smithi196:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-11T07:12:52.269 DEBUG:teuthology.orchestra.run.smithi196:> stat /dev/vg_nvme/lv_3 2024-09-11T07:12:52.324 INFO:teuthology.orchestra.run.smithi196.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-11T07:12:52.324 INFO:teuthology.orchestra.run.smithi196.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T07:12:52.324 INFO:teuthology.orchestra.run.smithi196.stdout:Device: 5h/5d Inode: 984 Links: 1 2024-09-11T07:12:52.324 INFO:teuthology.orchestra.run.smithi196.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T07:12:52.324 INFO:teuthology.orchestra.run.smithi196.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T07:12:52.324 INFO:teuthology.orchestra.run.smithi196.stdout:Access: 2024-09-11 07:12:30.510254912 +0000 2024-09-11T07:12:52.324 INFO:teuthology.orchestra.run.smithi196.stdout:Modify: 2024-09-11 07:06:56.614515209 +0000 2024-09-11T07:12:52.325 INFO:teuthology.orchestra.run.smithi196.stdout:Change: 2024-09-11 07:06:56.614515209 +0000 2024-09-11T07:12:52.325 INFO:teuthology.orchestra.run.smithi196.stdout: Birth: 2024-09-11 07:06:56.614515209 +0000 2024-09-11T07:12:52.325 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-11T07:12:52.387 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records in 2024-09-11T07:12:52.387 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records out 2024-09-11T07:12:52.387 INFO:teuthology.orchestra.run.smithi196.stderr:512 bytes copied, 0.000230459 s, 2.2 MB/s 2024-09-11T07:12:52.388 DEBUG:teuthology.orchestra.run.smithi196:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-11T07:12:52.445 DEBUG:teuthology.orchestra.run.smithi196:> stat /dev/vg_nvme/lv_4 2024-09-11T07:12:52.500 INFO:teuthology.orchestra.run.smithi196.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-11T07:12:52.500 INFO:teuthology.orchestra.run.smithi196.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T07:12:52.500 INFO:teuthology.orchestra.run.smithi196.stdout:Device: 5h/5d Inode: 968 Links: 1 2024-09-11T07:12:52.500 INFO:teuthology.orchestra.run.smithi196.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T07:12:52.500 INFO:teuthology.orchestra.run.smithi196.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T07:12:52.500 INFO:teuthology.orchestra.run.smithi196.stdout:Access: 2024-09-11 07:12:30.510254912 +0000 2024-09-11T07:12:52.501 INFO:teuthology.orchestra.run.smithi196.stdout:Modify: 2024-09-11 07:06:56.611515274 +0000 2024-09-11T07:12:52.501 INFO:teuthology.orchestra.run.smithi196.stdout:Change: 2024-09-11 07:06:56.611515274 +0000 2024-09-11T07:12:52.501 INFO:teuthology.orchestra.run.smithi196.stdout: Birth: 2024-09-11 07:06:56.611515274 +0000 2024-09-11T07:12:52.501 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-11T07:12:52.563 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records in 2024-09-11T07:12:52.564 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records out 2024-09-11T07:12:52.564 INFO:teuthology.orchestra.run.smithi196.stderr:512 bytes copied, 0.00021751 s, 2.4 MB/s 2024-09-11T07:12:52.565 DEBUG:teuthology.orchestra.run.smithi196:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-11T07:12:52.621 INFO:tasks.cephadm:Deploying osd.0 on smithi071 with /dev/vg_nvme/lv_4... 2024-09-11T07:12:52.621 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a81cffe6-700c-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-09-11T07:12:52.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:52 smithi196 ceph-mon[30773]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:12:52.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:52 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:12:52.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:52 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:12:52.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:52 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:12:52.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:52 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:12:52.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:52 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:12:52.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:52 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:12:52.786 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:52 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T07:12:52.786 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:52 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-11T07:12:52.786 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:52 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T07:12:52.786 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:52 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:12:52.839 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:12:52 smithi071 ceph-mon[37727]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:12:52.839 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:12:52 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:12:52.839 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:12:52 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:12:52.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:12:52 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:12:52.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:12:52 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:12:52.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:12:52 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:12:52.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:12:52 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:12:52.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:12:52 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T07:12:52.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:12:52 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-11T07:12:52.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:12:52 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T07:12:52.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:12:52 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:12:52.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:52 smithi071 ceph-mon[27013]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:12:52.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:52 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:12:52.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:52 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:12:52.841 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:52 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:12:52.841 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:52 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:12:52.841 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:52 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:12:52.841 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:52 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:12:52.841 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:52 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T07:12:52.841 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:52 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-11T07:12:52.841 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:52 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T07:12:52.841 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:52 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:12:52.856 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:12:53.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:53 smithi071 ceph-mon[27013]: Deploying daemon mgr.x on smithi196 2024-09-11T07:12:53.589 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:12:53 smithi071 ceph-mon[37727]: Deploying daemon mgr.x on smithi196 2024-09-11T07:12:53.784 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:53 smithi196 ceph-mon[30773]: Deploying daemon mgr.x on smithi196 2024-09-11T07:12:54.521 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 07:12:54 smithi196 podman[32594]: 2024-09-11 07:12:54.425270126 +0000 UTC m=+0.118873844 container create c748882f8ce1262fc2b136638be28edd90f0800a5428d82213bdffa8a08c4c40 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-x, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731) 2024-09-11T07:12:54.548 INFO:teuthology.orchestra.run.smithi071.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-09-11T07:12:54.629 INFO:teuthology.orchestra.run.smithi071.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-09-11T07:12:54.642 INFO:teuthology.orchestra.run.smithi071.stderr: stderr: 10+0 records in 2024-09-11T07:12:54.642 INFO:teuthology.orchestra.run.smithi071.stderr:10+0 records out 2024-09-11T07:12:54.642 INFO:teuthology.orchestra.run.smithi071.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113452 s, 924 MB/s 2024-09-11T07:12:54.643 INFO:teuthology.orchestra.run.smithi071.stderr:--> Zapping successful for: 2024-09-11T07:12:54.784 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:54 smithi196 ceph-mon[30773]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:12:54.785 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 07:12:54 smithi196 podman[32594]: 2024-09-11 07:12:54.627663295 +0000 UTC m=+0.321266967 container init c748882f8ce1262fc2b136638be28edd90f0800a5428d82213bdffa8a08c4c40 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-x, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T07:12:54.785 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 07:12:54 smithi196 podman[32594]: 2024-09-11 07:12:54.631275899 +0000 UTC m=+0.324879573 container start c748882f8ce1262fc2b136638be28edd90f0800a5428d82213bdffa8a08c4c40 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-x, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-09-11T07:12:54.785 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 07:12:54 smithi196 bash[32594]: c748882f8ce1262fc2b136638be28edd90f0800a5428d82213bdffa8a08c4c40 2024-09-11T07:12:54.785 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 07:12:54 smithi196 systemd[1]: Started Ceph mgr.x for a81cffe6-700c-11ef-bcea-c7b262605968. 2024-09-11T07:12:54.785 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 07:12:54 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-x[32609]: 2024-09-11T07:12:54.741+0000 7f359a2a4ec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-11T07:12:54.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:54 smithi071 ceph-mon[27013]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:12:54.839 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:12:54 smithi071 ceph-mon[37727]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:12:55.148 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 07:12:54 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-x[32609]: 2024-09-11T07:12:54.800+0000 7f359a2a4ec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-11T07:12:55.148 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 07:12:54 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-x[32609]: 2024-09-11T07:12:54.867+0000 7f359a2a4ec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-11T07:12:55.148 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 07:12:54 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-x[32609]: 2024-09-11T07:12:54.996+0000 7f359a2a4ec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-11T07:12:55.148 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 07:12:55 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-x[32609]: 2024-09-11T07:12:55.145+0000 7f359a2a4ec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-11T07:12:55.198 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a81cffe6-700c-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi071:vg_nvme/lv_4 2024-09-11T07:12:55.435 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:12:55.784 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 07:12:55 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-x[32609]: 2024-09-11T07:12:55.415+0000 7f359a2a4ec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-11T07:12:55.785 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 07:12:55 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-x[32609]: 2024-09-11T07:12:55.478+0000 7f359a2a4ec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-11T07:12:55.785 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 07:12:55 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-x[32609]: 2024-09-11T07:12:55.644+0000 7f359a2a4ec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-11T07:12:55.841 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:55 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:12:55.841 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:55 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:12:55.842 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:55 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:12:55.842 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:55 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:12:55.842 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:55 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:12:55.842 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:12:55 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:12:55.842 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:12:55 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:12:55.842 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:12:55 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:12:55.842 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:12:55 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:12:55.842 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:12:55 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:12:56.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:55 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:12:56.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:55 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:12:56.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:55 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:12:56.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:55 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:12:56.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:55 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:12:56.784 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 07:12:56 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-x[32609]: 2024-09-11T07:12:56.500+0000 7f359a2a4ec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-11T07:12:56.785 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 07:12:56 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-x[32609]: 2024-09-11T07:12:56.565+0000 7f359a2a4ec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-11T07:12:57.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:57 smithi071 ceph-mon[27013]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:12:57.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:57 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:12:57.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:57 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:12:57.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:57 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:12:57.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:57 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:12:57.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:57 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:12:57.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:57 smithi071 ceph-mon[27013]: Reconfiguring mgr.y (unknown last config time)... 2024-09-11T07:12:57.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:57 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T07:12:57.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:57 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T07:12:57.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:57 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:12:57.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:57 smithi071 ceph-mon[27013]: Reconfiguring daemon mgr.y on smithi071 2024-09-11T07:12:57.341 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:12:57 smithi071 ceph-mon[37727]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:12:57.341 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:12:57 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:12:57.341 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:12:57 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:12:57.341 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:12:57 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:12:57.341 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:12:57 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:12:57.341 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:12:57 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:12:57.341 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:12:57 smithi071 ceph-mon[37727]: Reconfiguring mgr.y (unknown last config time)... 2024-09-11T07:12:57.341 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:12:57 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T07:12:57.341 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:12:57 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T07:12:57.341 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:12:57 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:12:57.342 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:12:57 smithi071 ceph-mon[37727]: Reconfiguring daemon mgr.y on smithi071 2024-09-11T07:12:57.383 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 07:12:57 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-x[32609]: 2024-09-11T07:12:57.271+0000 7f359a2a4ec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-11T07:12:57.383 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:57 smithi196 ceph-mon[30773]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:12:57.384 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:57 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:12:57.384 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:57 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:12:57.384 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:57 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:12:57.384 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:57 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:12:57.384 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:57 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:12:57.384 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:57 smithi196 ceph-mon[30773]: Reconfiguring mgr.y (unknown last config time)... 2024-09-11T07:12:57.384 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:57 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-11T07:12:57.384 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:57 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-11T07:12:57.384 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:57 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:12:57.384 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:57 smithi196 ceph-mon[30773]: Reconfiguring daemon mgr.y on smithi071 2024-09-11T07:12:57.692 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 07:12:57 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-x[32609]: 2024-09-11T07:12:57.381+0000 7f359a2a4ec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-11T07:12:57.692 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 07:12:57 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-x[32609]: 2024-09-11T07:12:57.504+0000 7f359a2a4ec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-11T07:12:57.692 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 07:12:57 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-x[32609]: 2024-09-11T07:12:57.564+0000 7f359a2a4ec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-11T07:12:57.692 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 07:12:57 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-x[32609]: 2024-09-11T07:12:57.689+0000 7f359a2a4ec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-11T07:12:57.946 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 07:12:57 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-x[32609]: 2024-09-11T07:12:57.755+0000 7f359a2a4ec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-11T07:12:57.946 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 07:12:57 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-x[32609]: 2024-09-11T07:12:57.812+0000 7f359a2a4ec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-11T07:12:57.946 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 07:12:57 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-x[32609]: 2024-09-11T07:12:57.944+0000 7f359a2a4ec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-11T07:12:58.208 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 07:12:58 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-x[32609]: 2024-09-11T07:12:58.006+0000 7f359a2a4ec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-11T07:12:58.208 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 07:12:58 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-x[32609]: 2024-09-11T07:12:58.070+0000 7f359a2a4ec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-11T07:12:58.209 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 07:12:58 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-x[32609]: 2024-09-11T07:12:58.149+0000 7f359a2a4ec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-11T07:12:58.209 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 07:12:58 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-x[32609]: 2024-09-11T07:12:58.206+0000 7f359a2a4ec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-11T07:12:58.534 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 07:12:58 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-x[32609]: 2024-09-11T07:12:58.263+0000 7f359a2a4ec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-11T07:12:58.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:58 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:12:58.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:58 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:12:58.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:58 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:12:58.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:58 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:12:58.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:58 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:12:58.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:58 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:12:58.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:58 smithi196 ceph-mon[30773]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:12:58.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:58 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:12:58.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:58 smithi196 ceph-mon[30773]: from='client.14214 v1:172.21.15.71:0/79485226' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:12:58.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:58 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T07:12:58.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:58 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T07:12:58.536 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:58 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:12:58.599 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:12:58 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:12:58.599 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:12:58 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:12:58.599 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:12:58 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:12:58.599 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:12:58 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:12:58.599 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:12:58 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:12:58.599 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:12:58 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:12:58.599 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:12:58 smithi071 ceph-mon[37727]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:12:58.600 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:12:58 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:12:58.600 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:12:58 smithi071 ceph-mon[37727]: from='client.14214 v1:172.21.15.71:0/79485226' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:12:58.600 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:12:58 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T07:12:58.600 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:12:58 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T07:12:58.600 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:12:58 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:12:58.600 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:58 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:12:58.600 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:58 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:12:58.600 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:58 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:12:58.600 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:58 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:12:58.600 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:58 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:12:58.600 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:58 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:12:58.601 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:58 smithi071 ceph-mon[27013]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:12:58.601 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:58 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:12:58.601 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:58 smithi071 ceph-mon[27013]: from='client.14214 v1:172.21.15.71:0/79485226' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:12:58.601 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:58 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T07:12:58.601 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:58 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T07:12:58.601 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:58 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:12:59.285 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 07:12:59 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-x[32609]: 2024-09-11T07:12:59.005+0000 7f359a2a4ec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-11T07:12:59.655 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:59 smithi071 ceph-mon[27013]: Standby manager daemon x started 2024-09-11T07:12:59.655 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:59 smithi071 ceph-mon[27013]: from='mgr.? v1:172.21.15.196:0/4268579044' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-11T07:12:59.655 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:59 smithi071 ceph-mon[27013]: from='mgr.? v1:172.21.15.196:0/4268579044' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T07:12:59.655 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:59 smithi071 ceph-mon[27013]: from='mgr.? v1:172.21.15.196:0/4268579044' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-11T07:12:59.655 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:12:59 smithi071 ceph-mon[27013]: from='mgr.? v1:172.21.15.196:0/4268579044' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T07:12:59.655 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:12:59 smithi071 ceph-mon[37727]: Standby manager daemon x started 2024-09-11T07:12:59.655 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:12:59 smithi071 ceph-mon[37727]: from='mgr.? v1:172.21.15.196:0/4268579044' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-11T07:12:59.655 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:12:59 smithi071 ceph-mon[37727]: from='mgr.? v1:172.21.15.196:0/4268579044' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T07:12:59.655 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:12:59 smithi071 ceph-mon[37727]: from='mgr.? v1:172.21.15.196:0/4268579044' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-11T07:12:59.656 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:12:59 smithi071 ceph-mon[37727]: from='mgr.? v1:172.21.15.196:0/4268579044' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T07:12:59.784 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:59 smithi196 ceph-mon[30773]: Standby manager daemon x started 2024-09-11T07:12:59.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:59 smithi196 ceph-mon[30773]: from='mgr.? v1:172.21.15.196:0/4268579044' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-11T07:12:59.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:59 smithi196 ceph-mon[30773]: from='mgr.? v1:172.21.15.196:0/4268579044' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T07:12:59.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:59 smithi196 ceph-mon[30773]: from='mgr.? v1:172.21.15.196:0/4268579044' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-11T07:12:59.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:12:59 smithi196 ceph-mon[30773]: from='mgr.? v1:172.21.15.196:0/4268579044' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T07:13:00.675 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:00 smithi071 ceph-mon[37727]: mgrmap e14: y(active, since 61s), standbys: x 2024-09-11T07:13:00.675 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:00 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-11T07:13:00.675 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:00 smithi071 ceph-mon[37727]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:13:00.675 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:00 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/4064191042' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "10eebb2a-001d-4106-b37b-3af068980575"}]: dispatch 2024-09-11T07:13:00.675 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:00 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/4064191042' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "10eebb2a-001d-4106-b37b-3af068980575"}]': finished 2024-09-11T07:13:00.675 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:00 smithi071 ceph-mon[37727]: osdmap e5: 1 total, 0 up, 1 in 2024-09-11T07:13:00.675 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:00 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T07:13:00.676 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:00 smithi071 ceph-mon[27013]: mgrmap e14: y(active, since 61s), standbys: x 2024-09-11T07:13:00.676 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:00 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-11T07:13:00.676 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:00 smithi071 ceph-mon[27013]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:13:00.676 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:00 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/4064191042' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "10eebb2a-001d-4106-b37b-3af068980575"}]: dispatch 2024-09-11T07:13:00.676 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:00 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/4064191042' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "10eebb2a-001d-4106-b37b-3af068980575"}]': finished 2024-09-11T07:13:00.676 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:00 smithi071 ceph-mon[27013]: osdmap e5: 1 total, 0 up, 1 in 2024-09-11T07:13:00.676 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:00 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T07:13:00.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:00 smithi196 ceph-mon[30773]: mgrmap e14: y(active, since 61s), standbys: x 2024-09-11T07:13:00.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:00 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-11T07:13:00.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:00 smithi196 ceph-mon[30773]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:13:00.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:00 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/4064191042' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "10eebb2a-001d-4106-b37b-3af068980575"}]: dispatch 2024-09-11T07:13:00.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:00 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/4064191042' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "10eebb2a-001d-4106-b37b-3af068980575"}]': finished 2024-09-11T07:13:00.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:00 smithi196 ceph-mon[30773]: osdmap e5: 1 total, 0 up, 1 in 2024-09-11T07:13:00.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:00 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T07:13:01.784 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:01 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/4282976904' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T07:13:01.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:01 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/4282976904' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T07:13:01.839 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:01 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/4282976904' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T07:13:02.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:02 smithi196 ceph-mon[30773]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:13:02.799 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:02 smithi071 ceph-mon[27013]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:13:02.799 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:02 smithi071 ceph-mon[37727]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:13:04.792 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:04 smithi071 ceph-mon[27013]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:13:04.792 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:04 smithi071 ceph-mon[37727]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:13:05.034 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:04 smithi196 ceph-mon[30773]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:13:06.785 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:06 smithi071 ceph-mon[27013]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:13:06.785 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:06 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-11T07:13:06.786 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:06 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:13:06.786 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:06 smithi071 ceph-mon[27013]: Deploying daemon osd.0 on smithi071 2024-09-11T07:13:06.786 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:06 smithi071 ceph-mon[37727]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:13:06.786 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:06 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-11T07:13:06.786 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:06 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:13:06.786 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:06 smithi071 ceph-mon[37727]: Deploying daemon osd.0 on smithi071 2024-09-11T07:13:07.034 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:06 smithi196 ceph-mon[30773]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:13:07.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:06 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-11T07:13:07.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:06 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:13:07.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:06 smithi196 ceph-mon[30773]: Deploying daemon osd.0 on smithi071 2024-09-11T07:13:08.778 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:08 smithi071 ceph-mon[27013]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:13:08.778 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:08 smithi071 ceph-mon[37727]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:13:09.034 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:08 smithi196 ceph-mon[30773]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:13:10.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:10 smithi071 ceph-mon[27013]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:13:10.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:10 smithi071 ceph-mon[37727]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:13:11.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:10 smithi196 ceph-mon[30773]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:13:11.839 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:11 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:13:11.839 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:11 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:13:11.839 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:11 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:13:11.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:11 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:13:11.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:11 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:13:11.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:11 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:13:12.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:11 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:13:12.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:11 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:13:12.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:11 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:13:13.158 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:12 smithi071 ceph-mon[27013]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:13:13.159 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:12 smithi071 ceph-mon[37727]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:13:13.284 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:12 smithi196 ceph-mon[30773]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:13:14.159 INFO:teuthology.orchestra.run.smithi071.stdout:Created osd(s) 0 on host 'smithi071' 2024-09-11T07:13:14.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:14 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:13:14.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:14 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:13:14.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:14 smithi196 ceph-mon[30773]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:13:14.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:14 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:13:14.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:14 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:13:14.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:14 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:13:14.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:14 smithi196 ceph-mon[30773]: from='osd.0 v1:172.21.15.71:6801/88530034' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-11T07:13:14.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:14 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:13:14.536 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:14 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:13:14.536 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:14 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:13:14.589 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:14 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:13:14.589 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:14 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:13:14.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:14 smithi071 ceph-mon[37727]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:13:14.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:14 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:13:14.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:14 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:13:14.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:14 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:13:14.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:14 smithi071 ceph-mon[37727]: from='osd.0 v1:172.21.15.71:6801/88530034' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-11T07:13:14.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:14 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:13:14.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:14 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:13:14.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:14 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:13:14.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:14 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:13:14.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:14 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:13:14.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:14 smithi071 ceph-mon[27013]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:13:14.591 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:14 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:13:14.591 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:14 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:13:14.591 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:14 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:13:14.591 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:14 smithi071 ceph-mon[27013]: from='osd.0 v1:172.21.15.71:6801/88530034' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-11T07:13:14.591 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:14 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:13:14.591 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:14 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:13:14.591 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:14 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:13:14.771 DEBUG:teuthology.orchestra.run.smithi071:osd.0> sudo journalctl -f -n 0 -u ceph-a81cffe6-700c-11ef-bcea-c7b262605968@osd.0.service 2024-09-11T07:13:14.773 INFO:tasks.cephadm:Deploying osd.1 on smithi071 with /dev/vg_nvme/lv_3... 2024-09-11T07:13:14.773 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a81cffe6-700c-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-09-11T07:13:15.715 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:13:15.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:15 smithi071 ceph-mon[27013]: from='osd.0 v1:172.21.15.71:6801/88530034' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-11T07:13:15.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:15 smithi071 ceph-mon[27013]: osdmap e6: 1 total, 0 up, 1 in 2024-09-11T07:13:15.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:15 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T07:13:15.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:15 smithi071 ceph-mon[27013]: from='osd.0 v1:172.21.15.71:6801/88530034' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-09-11T07:13:15.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:15 smithi071 ceph-mon[37727]: from='osd.0 v1:172.21.15.71:6801/88530034' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-11T07:13:15.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:15 smithi071 ceph-mon[37727]: osdmap e6: 1 total, 0 up, 1 in 2024-09-11T07:13:15.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:15 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T07:13:15.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:15 smithi071 ceph-mon[37727]: from='osd.0 v1:172.21.15.71:6801/88530034' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-09-11T07:13:15.840 INFO:journalctl@ceph.osd.0.smithi071.stdout:Sep 11 07:13:15 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-0[41593]: 2024-09-11T07:13:15.566+0000 7f299e0b7640 -1 osd.0 0 waiting for initial osdmap 2024-09-11T07:13:15.840 INFO:journalctl@ceph.osd.0.smithi071.stdout:Sep 11 07:13:15 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-0[41593]: 2024-09-11T07:13:15.569+0000 7f29976eb640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T07:13:16.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:15 smithi196 ceph-mon[30773]: from='osd.0 v1:172.21.15.71:6801/88530034' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-11T07:13:16.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:15 smithi196 ceph-mon[30773]: osdmap e6: 1 total, 0 up, 1 in 2024-09-11T07:13:16.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:15 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T07:13:16.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:15 smithi196 ceph-mon[30773]: from='osd.0 v1:172.21.15.71:6801/88530034' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-09-11T07:13:16.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:16 smithi071 ceph-mon[37727]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:13:16.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:16 smithi071 ceph-mon[37727]: from='osd.0 v1:172.21.15.71:6801/88530034' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-09-11T07:13:16.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:16 smithi071 ceph-mon[37727]: osdmap e7: 1 total, 0 up, 1 in 2024-09-11T07:13:16.841 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:16 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T07:13:16.841 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:16 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T07:13:16.841 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:16 smithi071 ceph-mon[27013]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:13:16.841 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:16 smithi071 ceph-mon[27013]: from='osd.0 v1:172.21.15.71:6801/88530034' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-09-11T07:13:16.842 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:16 smithi071 ceph-mon[27013]: osdmap e7: 1 total, 0 up, 1 in 2024-09-11T07:13:16.842 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:16 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T07:13:16.842 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:16 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T07:13:17.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:16 smithi196 ceph-mon[30773]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:13:17.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:16 smithi196 ceph-mon[30773]: from='osd.0 v1:172.21.15.71:6801/88530034' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-09-11T07:13:17.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:16 smithi196 ceph-mon[30773]: osdmap e7: 1 total, 0 up, 1 in 2024-09-11T07:13:17.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:16 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T07:13:17.036 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:16 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T07:13:17.634 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:17 smithi071 ceph-mon[27013]: purged_snaps scrub starts 2024-09-11T07:13:17.635 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:17 smithi071 ceph-mon[27013]: purged_snaps scrub ok 2024-09-11T07:13:17.635 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:17 smithi071 ceph-mon[27013]: osd.0 v1:172.21.15.71:6801/88530034 boot 2024-09-11T07:13:17.635 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:17 smithi071 ceph-mon[27013]: osdmap e8: 1 total, 1 up, 1 in 2024-09-11T07:13:17.635 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:17 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T07:13:17.635 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:17 smithi071 ceph-mon[37727]: purged_snaps scrub starts 2024-09-11T07:13:17.635 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:17 smithi071 ceph-mon[37727]: purged_snaps scrub ok 2024-09-11T07:13:17.635 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:17 smithi071 ceph-mon[37727]: osd.0 v1:172.21.15.71:6801/88530034 boot 2024-09-11T07:13:17.635 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:17 smithi071 ceph-mon[37727]: osdmap e8: 1 total, 1 up, 1 in 2024-09-11T07:13:17.635 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:17 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T07:13:17.982 INFO:teuthology.orchestra.run.smithi071.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-09-11T07:13:18.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:17 smithi196 ceph-mon[30773]: purged_snaps scrub starts 2024-09-11T07:13:18.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:17 smithi196 ceph-mon[30773]: purged_snaps scrub ok 2024-09-11T07:13:18.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:17 smithi196 ceph-mon[30773]: osd.0 v1:172.21.15.71:6801/88530034 boot 2024-09-11T07:13:18.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:17 smithi196 ceph-mon[30773]: osdmap e8: 1 total, 1 up, 1 in 2024-09-11T07:13:18.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:17 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T07:13:18.069 INFO:teuthology.orchestra.run.smithi071.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-09-11T07:13:18.085 INFO:teuthology.orchestra.run.smithi071.stderr: stderr: 10+0 records in 2024-09-11T07:13:18.085 INFO:teuthology.orchestra.run.smithi071.stderr:10+0 records out 2024-09-11T07:13:18.085 INFO:teuthology.orchestra.run.smithi071.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.014353 s, 731 MB/s 2024-09-11T07:13:18.086 INFO:teuthology.orchestra.run.smithi071.stderr:--> Zapping successful for: 2024-09-11T07:13:18.774 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:18 smithi071 ceph-mon[27013]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:13:18.774 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:18 smithi071 ceph-mon[27013]: osdmap e9: 1 total, 1 up, 1 in 2024-09-11T07:13:18.774 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:18 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:13:18.774 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:18 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:13:18.774 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:18 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:13:18.774 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:18 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:13:18.774 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:18 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:13:18.774 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:18 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:13:18.774 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:18 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:13:18.775 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:18 smithi071 ceph-mon[37727]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:13:18.775 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:18 smithi071 ceph-mon[37727]: osdmap e9: 1 total, 1 up, 1 in 2024-09-11T07:13:18.775 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:18 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:13:18.775 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:18 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:13:18.775 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:18 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:13:18.775 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:18 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:13:18.775 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:18 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:13:18.775 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:18 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:13:18.775 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:18 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:13:19.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:18 smithi196 ceph-mon[30773]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-11T07:13:19.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:18 smithi196 ceph-mon[30773]: osdmap e9: 1 total, 1 up, 1 in 2024-09-11T07:13:19.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:18 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:13:19.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:18 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:13:19.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:18 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:13:19.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:18 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:13:19.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:18 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:13:19.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:18 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:13:19.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:18 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:13:19.066 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a81cffe6-700c-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi071:vg_nvme/lv_3 2024-09-11T07:13:19.304 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:13:19.675 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:19 smithi071 ceph-mon[27013]: Detected new or changed devices on smithi071 2024-09-11T07:13:19.676 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:19 smithi071 ceph-mon[27013]: Adjusting osd_memory_target on smithi071 to 16064M 2024-09-11T07:13:19.676 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:19 smithi071 ceph-mon[37727]: Detected new or changed devices on smithi071 2024-09-11T07:13:19.676 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:19 smithi071 ceph-mon[37727]: Adjusting osd_memory_target on smithi071 to 16064M 2024-09-11T07:13:20.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:19 smithi196 ceph-mon[30773]: Detected new or changed devices on smithi071 2024-09-11T07:13:20.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:19 smithi196 ceph-mon[30773]: Adjusting osd_memory_target on smithi071 to 16064M 2024-09-11T07:13:20.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:20 smithi071 ceph-mon[27013]: pgmap v40: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:13:20.839 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:20 smithi071 ceph-mon[37727]: pgmap v40: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:13:21.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:20 smithi196 ceph-mon[30773]: pgmap v40: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:13:21.991 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:21 smithi071 ceph-mon[37727]: from='client.34139 v1:172.21.15.71:0/1399493300' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:13:21.991 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:21 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T07:13:21.991 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:21 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T07:13:21.992 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:21 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:13:21.992 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:21 smithi071 ceph-mon[27013]: from='client.34139 v1:172.21.15.71:0/1399493300' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:13:21.992 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:21 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T07:13:21.992 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:21 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T07:13:21.992 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:21 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:13:22.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:21 smithi196 ceph-mon[30773]: from='client.34139 v1:172.21.15.71:0/1399493300' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:13:22.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:21 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T07:13:22.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:21 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T07:13:22.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:21 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:13:22.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:22 smithi071 ceph-mon[27013]: pgmap v41: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:13:22.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:22 smithi071 ceph-mon[37727]: pgmap v41: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:13:23.034 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:22 smithi196 ceph-mon[30773]: pgmap v41: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:13:23.757 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:23 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/3736815696' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c46d1c53-e519-4ac0-b557-91ee98f659fe"}]: dispatch 2024-09-11T07:13:23.757 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:23 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/3736815696' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c46d1c53-e519-4ac0-b557-91ee98f659fe"}]': finished 2024-09-11T07:13:23.758 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:23 smithi071 ceph-mon[27013]: osdmap e10: 2 total, 1 up, 2 in 2024-09-11T07:13:23.758 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:23 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T07:13:23.758 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:23 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/3736815696' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c46d1c53-e519-4ac0-b557-91ee98f659fe"}]: dispatch 2024-09-11T07:13:23.758 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:23 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/3736815696' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c46d1c53-e519-4ac0-b557-91ee98f659fe"}]': finished 2024-09-11T07:13:23.758 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:23 smithi071 ceph-mon[37727]: osdmap e10: 2 total, 1 up, 2 in 2024-09-11T07:13:23.758 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:23 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T07:13:24.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:23 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/3736815696' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c46d1c53-e519-4ac0-b557-91ee98f659fe"}]: dispatch 2024-09-11T07:13:24.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:23 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/3736815696' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c46d1c53-e519-4ac0-b557-91ee98f659fe"}]': finished 2024-09-11T07:13:24.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:23 smithi196 ceph-mon[30773]: osdmap e10: 2 total, 1 up, 2 in 2024-09-11T07:13:24.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:23 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T07:13:25.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:24 smithi196 ceph-mon[30773]: pgmap v43: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:13:25.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:24 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/2615118640' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T07:13:25.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:24 smithi071 ceph-mon[27013]: pgmap v43: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:13:25.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:24 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/2615118640' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T07:13:25.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:24 smithi071 ceph-mon[37727]: pgmap v43: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:13:25.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:24 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/2615118640' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T07:13:26.943 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:26 smithi071 ceph-mon[27013]: pgmap v44: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:13:26.943 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:26 smithi071 ceph-mon[37727]: pgmap v44: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:13:27.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:26 smithi196 ceph-mon[30773]: pgmap v44: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:13:29.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:28 smithi196 ceph-mon[30773]: pgmap v45: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:13:29.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:28 smithi071 ceph-mon[27013]: pgmap v45: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:13:29.089 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:28 smithi071 ceph-mon[37727]: pgmap v45: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:13:29.684 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:29 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-11T07:13:29.684 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:29 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:13:29.684 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:29 smithi071 ceph-mon[27013]: Deploying daemon osd.1 on smithi071 2024-09-11T07:13:29.685 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:29 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-11T07:13:29.685 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:29 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:13:29.685 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:29 smithi071 ceph-mon[37727]: Deploying daemon osd.1 on smithi071 2024-09-11T07:13:30.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:29 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-11T07:13:30.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:29 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:13:30.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:29 smithi196 ceph-mon[30773]: Deploying daemon osd.1 on smithi071 2024-09-11T07:13:30.693 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:30 smithi071 ceph-mon[27013]: pgmap v46: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:13:30.693 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:30 smithi071 ceph-mon[37727]: pgmap v46: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:13:31.036 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:30 smithi196 ceph-mon[30773]: pgmap v46: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:13:32.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:32 smithi071 ceph-mon[27013]: pgmap v47: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:13:32.839 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:32 smithi071 ceph-mon[37727]: pgmap v47: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:13:33.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:32 smithi196 ceph-mon[30773]: pgmap v47: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:13:34.710 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:34 smithi071 ceph-mon[27013]: pgmap v48: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:13:34.710 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:34 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:13:34.711 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:34 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:13:34.711 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:34 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:13:34.711 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:34 smithi071 ceph-mon[37727]: pgmap v48: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:13:34.711 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:34 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:13:34.711 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:34 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:13:34.711 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:34 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:13:35.034 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:34 smithi196 ceph-mon[30773]: pgmap v48: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:13:35.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:34 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:13:35.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:34 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:13:35.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:34 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:13:36.732 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:36 smithi071 ceph-mon[27013]: pgmap v49: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:13:36.732 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:36 smithi071 ceph-mon[37727]: pgmap v49: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:13:37.034 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:36 smithi196 ceph-mon[30773]: pgmap v49: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:13:37.442 INFO:teuthology.orchestra.run.smithi071.stdout:Created osd(s) 1 on host 'smithi071' 2024-09-11T07:13:38.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:37 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:13:38.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:37 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:13:38.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:37 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:13:38.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:37 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:13:38.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:37 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:13:38.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:37 smithi196 ceph-mon[30773]: from='osd.1 v1:172.21.15.71:6805/2000038894' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-11T07:13:38.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:37 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:13:38.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:37 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:13:38.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:37 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:13:38.037 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:37 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:13:38.037 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:37 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:13:38.037 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:37 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:13:38.037 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:37 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:13:38.037 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:37 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:13:38.037 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:37 smithi071 ceph-mon[27013]: from='osd.1 v1:172.21.15.71:6805/2000038894' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-11T07:13:38.037 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:37 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:13:38.037 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:37 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:13:38.037 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:37 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:13:38.038 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:37 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:13:38.038 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:37 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:13:38.038 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:37 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:13:38.038 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:37 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:13:38.038 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:37 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:13:38.038 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:37 smithi071 ceph-mon[37727]: from='osd.1 v1:172.21.15.71:6805/2000038894' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-11T07:13:38.038 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:37 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:13:38.038 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:37 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:13:38.038 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:37 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:13:38.061 DEBUG:teuthology.orchestra.run.smithi071:osd.1> sudo journalctl -f -n 0 -u ceph-a81cffe6-700c-11ef-bcea-c7b262605968@osd.1.service 2024-09-11T07:13:38.064 INFO:tasks.cephadm:Deploying osd.2 on smithi071 with /dev/vg_nvme/lv_2... 2024-09-11T07:13:38.064 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a81cffe6-700c-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-09-11T07:13:39.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:38 smithi196 ceph-mon[30773]: pgmap v50: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:13:39.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:38 smithi196 ceph-mon[30773]: from='osd.1 v1:172.21.15.71:6805/2000038894' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-11T07:13:39.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:38 smithi196 ceph-mon[30773]: osdmap e11: 2 total, 1 up, 2 in 2024-09-11T07:13:39.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:38 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T07:13:39.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:38 smithi196 ceph-mon[30773]: from='osd.1 v1:172.21.15.71:6805/2000038894' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-09-11T07:13:39.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:38 smithi071 ceph-mon[27013]: pgmap v50: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:13:39.090 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:38 smithi071 ceph-mon[27013]: from='osd.1 v1:172.21.15.71:6805/2000038894' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-11T07:13:39.090 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:38 smithi071 ceph-mon[27013]: osdmap e11: 2 total, 1 up, 2 in 2024-09-11T07:13:39.090 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:38 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T07:13:39.090 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:38 smithi071 ceph-mon[27013]: from='osd.1 v1:172.21.15.71:6805/2000038894' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-09-11T07:13:39.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:38 smithi071 ceph-mon[37727]: pgmap v50: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:13:39.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:38 smithi071 ceph-mon[37727]: from='osd.1 v1:172.21.15.71:6805/2000038894' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-11T07:13:39.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:38 smithi071 ceph-mon[37727]: osdmap e11: 2 total, 1 up, 2 in 2024-09-11T07:13:39.091 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:38 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T07:13:39.091 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:38 smithi071 ceph-mon[37727]: from='osd.1 v1:172.21.15.71:6805/2000038894' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-09-11T07:13:39.091 INFO:journalctl@ceph.osd.1.smithi071.stdout:Sep 11 07:13:38 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-1[47504]: 2024-09-11T07:13:38.692+0000 7fb83c772640 -1 osd.1 0 waiting for initial osdmap 2024-09-11T07:13:39.091 INFO:journalctl@ceph.osd.1.smithi071.stdout:Sep 11 07:13:38 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-1[47504]: 2024-09-11T07:13:38.698+0000 7fb835d84640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T07:13:39.648 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:13:40.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:39 smithi196 ceph-mon[30773]: from='osd.1 v1:172.21.15.71:6805/2000038894' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-09-11T07:13:40.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:39 smithi196 ceph-mon[30773]: osdmap e12: 2 total, 1 up, 2 in 2024-09-11T07:13:40.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:39 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T07:13:40.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:39 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T07:13:40.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:39 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T07:13:40.036 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:39 smithi071 ceph-mon[27013]: from='osd.1 v1:172.21.15.71:6805/2000038894' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-09-11T07:13:40.036 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:39 smithi071 ceph-mon[27013]: osdmap e12: 2 total, 1 up, 2 in 2024-09-11T07:13:40.036 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:39 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T07:13:40.036 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:39 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T07:13:40.036 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:39 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T07:13:40.037 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:39 smithi071 ceph-mon[37727]: from='osd.1 v1:172.21.15.71:6805/2000038894' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-09-11T07:13:40.037 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:39 smithi071 ceph-mon[37727]: osdmap e12: 2 total, 1 up, 2 in 2024-09-11T07:13:40.037 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:39 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T07:13:40.037 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:39 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T07:13:40.037 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:39 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T07:13:40.891 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:40 smithi071 ceph-mon[27013]: purged_snaps scrub starts 2024-09-11T07:13:40.891 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:40 smithi071 ceph-mon[27013]: purged_snaps scrub ok 2024-09-11T07:13:40.891 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:40 smithi071 ceph-mon[27013]: pgmap v53: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:13:40.891 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:40 smithi071 ceph-mon[27013]: osd.1 v1:172.21.15.71:6805/2000038894 boot 2024-09-11T07:13:40.891 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:40 smithi071 ceph-mon[27013]: osdmap e13: 2 total, 2 up, 2 in 2024-09-11T07:13:40.892 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:40 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T07:13:40.892 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:40 smithi071 ceph-mon[37727]: purged_snaps scrub starts 2024-09-11T07:13:40.892 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:40 smithi071 ceph-mon[37727]: purged_snaps scrub ok 2024-09-11T07:13:40.892 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:40 smithi071 ceph-mon[37727]: pgmap v53: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:13:40.892 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:40 smithi071 ceph-mon[37727]: osd.1 v1:172.21.15.71:6805/2000038894 boot 2024-09-11T07:13:40.892 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:40 smithi071 ceph-mon[37727]: osdmap e13: 2 total, 2 up, 2 in 2024-09-11T07:13:40.892 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:40 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T07:13:41.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:40 smithi196 ceph-mon[30773]: purged_snaps scrub starts 2024-09-11T07:13:41.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:40 smithi196 ceph-mon[30773]: purged_snaps scrub ok 2024-09-11T07:13:41.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:40 smithi196 ceph-mon[30773]: pgmap v53: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-11T07:13:41.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:40 smithi196 ceph-mon[30773]: osd.1 v1:172.21.15.71:6805/2000038894 boot 2024-09-11T07:13:41.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:40 smithi196 ceph-mon[30773]: osdmap e13: 2 total, 2 up, 2 in 2024-09-11T07:13:41.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:40 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T07:13:41.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:41 smithi071 ceph-mon[27013]: osdmap e14: 2 total, 2 up, 2 in 2024-09-11T07:13:41.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:41 smithi071 ceph-mon[27013]: Detected new or changed devices on smithi071 2024-09-11T07:13:41.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:41 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:13:41.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:41 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:13:41.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:41 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:13:41.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:41 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:13:41.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:41 smithi071 ceph-mon[27013]: Adjusting osd_memory_target on smithi071 to 8032M 2024-09-11T07:13:41.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:41 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:13:41.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:41 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:13:41.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:41 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:13:41.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:41 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:13:41.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:41 smithi071 ceph-mon[37727]: osdmap e14: 2 total, 2 up, 2 in 2024-09-11T07:13:41.841 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:41 smithi071 ceph-mon[37727]: Detected new or changed devices on smithi071 2024-09-11T07:13:41.841 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:41 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:13:41.841 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:41 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:13:41.841 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:41 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:13:41.841 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:41 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:13:41.841 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:41 smithi071 ceph-mon[37727]: Adjusting osd_memory_target on smithi071 to 8032M 2024-09-11T07:13:41.841 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:41 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:13:41.841 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:41 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:13:41.841 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:41 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:13:41.841 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:41 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:13:41.906 INFO:teuthology.orchestra.run.smithi071.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-09-11T07:13:42.000 INFO:teuthology.orchestra.run.smithi071.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-09-11T07:13:42.020 INFO:teuthology.orchestra.run.smithi071.stderr: stderr: 10+0 records in 2024-09-11T07:13:42.020 INFO:teuthology.orchestra.run.smithi071.stderr:10+0 records out 2024-09-11T07:13:42.020 INFO:teuthology.orchestra.run.smithi071.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.017816 s, 589 MB/s 2024-09-11T07:13:42.020 INFO:teuthology.orchestra.run.smithi071.stderr:--> Zapping successful for: 2024-09-11T07:13:42.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:41 smithi196 ceph-mon[30773]: osdmap e14: 2 total, 2 up, 2 in 2024-09-11T07:13:42.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:41 smithi196 ceph-mon[30773]: Detected new or changed devices on smithi071 2024-09-11T07:13:42.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:41 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:13:42.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:41 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:13:42.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:41 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:13:42.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:41 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:13:42.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:41 smithi196 ceph-mon[30773]: Adjusting osd_memory_target on smithi071 to 8032M 2024-09-11T07:13:42.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:41 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:13:42.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:41 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:13:42.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:41 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:13:42.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:41 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:13:42.684 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a81cffe6-700c-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi071:vg_nvme/lv_2 2024-09-11T07:13:42.925 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:13:43.046 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:42 smithi071 ceph-mon[27013]: pgmap v56: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:13:43.046 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:42 smithi071 ceph-mon[37727]: pgmap v56: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:13:43.284 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:42 smithi196 ceph-mon[30773]: pgmap v56: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:13:45.222 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:44 smithi071 ceph-mon[37727]: pgmap v57: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:13:45.222 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:44 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T07:13:45.223 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:44 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T07:13:45.223 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:44 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:13:45.223 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:44 smithi071 ceph-mon[27013]: pgmap v57: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:13:45.223 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:44 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T07:13:45.223 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:44 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T07:13:45.223 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:44 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:13:45.284 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:44 smithi196 ceph-mon[30773]: pgmap v57: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:13:45.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:44 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T07:13:45.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:44 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T07:13:45.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:44 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:13:46.066 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:45 smithi071 ceph-mon[27013]: from='client.14268 v1:172.21.15.71:0/3698908023' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:13:46.067 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:45 smithi071 ceph-mon[37727]: from='client.14268 v1:172.21.15.71:0/3698908023' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:13:46.284 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:45 smithi196 ceph-mon[30773]: from='client.14268 v1:172.21.15.71:0/3698908023' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:13:47.254 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:46 smithi071 ceph-mon[27013]: pgmap v58: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:13:47.254 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:46 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/2808425287' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a6a12661-7514-4887-9acf-a313f7055e30"}]: dispatch 2024-09-11T07:13:47.254 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:46 smithi071 ceph-mon[27013]: from='client.34172 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a6a12661-7514-4887-9acf-a313f7055e30"}]: dispatch 2024-09-11T07:13:47.254 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:46 smithi071 ceph-mon[27013]: from='client.34172 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a6a12661-7514-4887-9acf-a313f7055e30"}]': finished 2024-09-11T07:13:47.254 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:46 smithi071 ceph-mon[27013]: osdmap e15: 3 total, 2 up, 3 in 2024-09-11T07:13:47.255 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:46 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T07:13:47.255 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:46 smithi071 ceph-mon[37727]: pgmap v58: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:13:47.255 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:46 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/2808425287' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a6a12661-7514-4887-9acf-a313f7055e30"}]: dispatch 2024-09-11T07:13:47.255 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:46 smithi071 ceph-mon[37727]: from='client.34172 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a6a12661-7514-4887-9acf-a313f7055e30"}]: dispatch 2024-09-11T07:13:47.255 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:46 smithi071 ceph-mon[37727]: from='client.34172 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a6a12661-7514-4887-9acf-a313f7055e30"}]': finished 2024-09-11T07:13:47.255 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:46 smithi071 ceph-mon[37727]: osdmap e15: 3 total, 2 up, 3 in 2024-09-11T07:13:47.255 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:46 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T07:13:47.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:46 smithi196 ceph-mon[30773]: pgmap v58: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:13:47.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:46 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/2808425287' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a6a12661-7514-4887-9acf-a313f7055e30"}]: dispatch 2024-09-11T07:13:47.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:46 smithi196 ceph-mon[30773]: from='client.34172 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a6a12661-7514-4887-9acf-a313f7055e30"}]: dispatch 2024-09-11T07:13:47.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:46 smithi196 ceph-mon[30773]: from='client.34172 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a6a12661-7514-4887-9acf-a313f7055e30"}]': finished 2024-09-11T07:13:47.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:46 smithi196 ceph-mon[30773]: osdmap e15: 3 total, 2 up, 3 in 2024-09-11T07:13:47.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:46 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T07:13:48.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:47 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/1272669369' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T07:13:48.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:47 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/1272669369' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T07:13:48.339 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:47 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/1272669369' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T07:13:49.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:48 smithi196 ceph-mon[30773]: pgmap v60: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:13:49.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:48 smithi071 ceph-mon[27013]: pgmap v60: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:13:49.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:48 smithi071 ceph-mon[37727]: pgmap v60: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:13:51.193 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:50 smithi071 ceph-mon[27013]: pgmap v61: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:13:51.193 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:50 smithi071 ceph-mon[37727]: pgmap v61: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:13:51.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:50 smithi196 ceph-mon[30773]: pgmap v61: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:13:53.270 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:52 smithi071 ceph-mon[27013]: pgmap v62: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:13:53.270 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:52 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-11T07:13:53.271 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:52 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:13:53.271 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:52 smithi071 ceph-mon[37727]: pgmap v62: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:13:53.271 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:52 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-11T07:13:53.271 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:52 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:13:53.284 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:52 smithi196 ceph-mon[30773]: pgmap v62: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:13:53.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:52 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-11T07:13:53.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:52 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:13:54.198 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:53 smithi071 ceph-mon[27013]: Deploying daemon osd.2 on smithi071 2024-09-11T07:13:54.199 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:53 smithi071 ceph-mon[37727]: Deploying daemon osd.2 on smithi071 2024-09-11T07:13:54.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:53 smithi196 ceph-mon[30773]: Deploying daemon osd.2 on smithi071 2024-09-11T07:13:55.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:54 smithi196 ceph-mon[30773]: pgmap v63: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:13:55.327 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:54 smithi071 ceph-mon[27013]: pgmap v63: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:13:55.328 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:54 smithi071 ceph-mon[37727]: pgmap v63: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:13:57.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:56 smithi196 ceph-mon[30773]: pgmap v64: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:13:57.293 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:56 smithi071 ceph-mon[27013]: pgmap v64: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:13:57.293 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:56 smithi071 ceph-mon[37727]: pgmap v64: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:13:58.260 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:57 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:13:58.260 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:57 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:13:58.260 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:57 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:13:58.260 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:57 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:13:58.260 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:57 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:13:58.260 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:57 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:13:58.284 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:57 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:13:58.284 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:57 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:13:58.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:57 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:13:59.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:13:58 smithi196 ceph-mon[30773]: pgmap v65: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:13:59.301 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:13:58 smithi071 ceph-mon[27013]: pgmap v65: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:13:59.301 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:13:58 smithi071 ceph-mon[37727]: pgmap v65: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:14:00.434 INFO:teuthology.orchestra.run.smithi071.stdout:Created osd(s) 2 on host 'smithi071' 2024-09-11T07:14:01.042 DEBUG:teuthology.orchestra.run.smithi071:osd.2> sudo journalctl -f -n 0 -u ceph-a81cffe6-700c-11ef-bcea-c7b262605968@osd.2.service 2024-09-11T07:14:01.045 INFO:tasks.cephadm:Deploying osd.3 on smithi071 with /dev/vg_nvme/lv_1... 2024-09-11T07:14:01.045 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a81cffe6-700c-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-09-11T07:14:01.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:00 smithi196 ceph-mon[30773]: pgmap v66: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:14:01.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:00 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:14:01.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:00 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:14:01.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:00 smithi196 ceph-mon[30773]: from='osd.2 v1:172.21.15.71:6809/3105396883' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-11T07:14:01.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:00 smithi196 ceph-mon[30773]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-11T07:14:01.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:00 smithi071 ceph-mon[27013]: pgmap v66: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:14:01.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:00 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:14:01.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:00 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:14:01.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:00 smithi071 ceph-mon[27013]: from='osd.2 v1:172.21.15.71:6809/3105396883' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-11T07:14:01.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:00 smithi071 ceph-mon[27013]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-11T07:14:01.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:00 smithi071 ceph-mon[37727]: pgmap v66: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:14:01.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:00 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:14:01.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:00 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:14:01.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:00 smithi071 ceph-mon[37727]: from='osd.2 v1:172.21.15.71:6809/3105396883' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-11T07:14:01.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:00 smithi071 ceph-mon[37727]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-11T07:14:01.341 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:14:02.447 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:02 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:14:02.447 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:02 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:14:02.448 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:02 smithi071 ceph-mon[37727]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-11T07:14:02.448 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:02 smithi071 ceph-mon[37727]: osdmap e16: 3 total, 2 up, 3 in 2024-09-11T07:14:02.448 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:02 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T07:14:02.448 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:02 smithi071 ceph-mon[37727]: from='osd.2 v1:172.21.15.71:6809/3105396883' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-09-11T07:14:02.448 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:02 smithi071 ceph-mon[37727]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-09-11T07:14:02.449 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:02 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:14:02.449 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:02 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:14:02.449 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:02 smithi071 ceph-mon[27013]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-11T07:14:02.449 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:02 smithi071 ceph-mon[27013]: osdmap e16: 3 total, 2 up, 3 in 2024-09-11T07:14:02.450 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:02 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T07:14:02.450 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:02 smithi071 ceph-mon[27013]: from='osd.2 v1:172.21.15.71:6809/3105396883' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-09-11T07:14:02.450 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:02 smithi071 ceph-mon[27013]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-09-11T07:14:02.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:02 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:14:02.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:02 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:14:02.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:02 smithi196 ceph-mon[30773]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-11T07:14:02.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:02 smithi196 ceph-mon[30773]: osdmap e16: 3 total, 2 up, 3 in 2024-09-11T07:14:02.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:02 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T07:14:02.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:02 smithi196 ceph-mon[30773]: from='osd.2 v1:172.21.15.71:6809/3105396883' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-09-11T07:14:02.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:02 smithi196 ceph-mon[30773]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-09-11T07:14:02.744 INFO:journalctl@ceph.osd.2.smithi071.stdout:Sep 11 07:14:02 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-2[53217]: 2024-09-11T07:14:02.443+0000 7fdcb492a640 -1 osd.2 0 waiting for initial osdmap 2024-09-11T07:14:02.744 INFO:journalctl@ceph.osd.2.smithi071.stdout:Sep 11 07:14:02 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-2[53217]: 2024-09-11T07:14:02.447+0000 7fdcb1765640 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T07:14:03.434 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:03 smithi071 ceph-mon[27013]: pgmap v68: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:14:03.434 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:03 smithi071 ceph-mon[27013]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-09-11T07:14:03.434 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:03 smithi071 ceph-mon[27013]: osdmap e17: 3 total, 2 up, 3 in 2024-09-11T07:14:03.435 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:03 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T07:14:03.435 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:03 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T07:14:03.435 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:03 smithi071 ceph-mon[37727]: pgmap v68: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:14:03.435 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:03 smithi071 ceph-mon[37727]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-09-11T07:14:03.435 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:03 smithi071 ceph-mon[37727]: osdmap e17: 3 total, 2 up, 3 in 2024-09-11T07:14:03.435 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:03 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T07:14:03.435 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:03 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T07:14:03.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:03 smithi196 ceph-mon[30773]: pgmap v68: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:14:03.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:03 smithi196 ceph-mon[30773]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-09-11T07:14:03.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:03 smithi196 ceph-mon[30773]: osdmap e17: 3 total, 2 up, 3 in 2024-09-11T07:14:03.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:03 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T07:14:03.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:03 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T07:14:04.126 INFO:teuthology.orchestra.run.smithi071.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-09-11T07:14:04.221 INFO:teuthology.orchestra.run.smithi071.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-09-11T07:14:04.241 INFO:teuthology.orchestra.run.smithi071.stderr: stderr: 10+0 records in 2024-09-11T07:14:04.241 INFO:teuthology.orchestra.run.smithi071.stderr:10+0 records out 2024-09-11T07:14:04.241 INFO:teuthology.orchestra.run.smithi071.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0173571 s, 604 MB/s 2024-09-11T07:14:04.241 INFO:teuthology.orchestra.run.smithi071.stderr:--> Zapping successful for: 2024-09-11T07:14:04.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:04 smithi196 ceph-mon[30773]: purged_snaps scrub starts 2024-09-11T07:14:04.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:04 smithi196 ceph-mon[30773]: purged_snaps scrub ok 2024-09-11T07:14:04.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:04 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T07:14:04.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:04 smithi196 ceph-mon[30773]: osd.2 v1:172.21.15.71:6809/3105396883 boot 2024-09-11T07:14:04.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:04 smithi196 ceph-mon[30773]: osdmap e18: 3 total, 3 up, 3 in 2024-09-11T07:14:04.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:04 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T07:14:04.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:04 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-09-11T07:14:04.548 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:04 smithi071 ceph-mon[27013]: purged_snaps scrub starts 2024-09-11T07:14:04.548 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:04 smithi071 ceph-mon[27013]: purged_snaps scrub ok 2024-09-11T07:14:04.548 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:04 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T07:14:04.548 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:04 smithi071 ceph-mon[27013]: osd.2 v1:172.21.15.71:6809/3105396883 boot 2024-09-11T07:14:04.548 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:04 smithi071 ceph-mon[27013]: osdmap e18: 3 total, 3 up, 3 in 2024-09-11T07:14:04.548 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:04 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T07:14:04.548 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:04 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-09-11T07:14:04.549 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:04 smithi071 ceph-mon[37727]: purged_snaps scrub starts 2024-09-11T07:14:04.549 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:04 smithi071 ceph-mon[37727]: purged_snaps scrub ok 2024-09-11T07:14:04.549 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:04 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T07:14:04.549 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:04 smithi071 ceph-mon[37727]: osd.2 v1:172.21.15.71:6809/3105396883 boot 2024-09-11T07:14:04.549 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:04 smithi071 ceph-mon[37727]: osdmap e18: 3 total, 3 up, 3 in 2024-09-11T07:14:04.549 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:04 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T07:14:04.549 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:04 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-09-11T07:14:04.850 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a81cffe6-700c-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi071:vg_nvme/lv_1 2024-09-11T07:14:05.102 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:14:05.512 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:05 smithi071 ceph-mon[27013]: pgmap v71: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:14:05.512 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:05 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-09-11T07:14:05.512 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:05 smithi071 ceph-mon[27013]: osdmap e19: 3 total, 3 up, 3 in 2024-09-11T07:14:05.512 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:05 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T07:14:05.512 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:05 smithi071 ceph-mon[37727]: pgmap v71: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:14:05.512 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:05 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-09-11T07:14:05.512 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:05 smithi071 ceph-mon[37727]: osdmap e19: 3 total, 3 up, 3 in 2024-09-11T07:14:05.513 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:05 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T07:14:05.534 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:05 smithi196 ceph-mon[30773]: pgmap v71: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-11T07:14:05.534 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:05 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-09-11T07:14:05.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:05 smithi196 ceph-mon[30773]: osdmap e19: 3 total, 3 up, 3 in 2024-09-11T07:14:05.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:05 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T07:14:05.839 INFO:journalctl@ceph.osd.0.smithi071.stdout:Sep 11 07:14:05 smithi071 sudo[57297]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-11T07:14:05.839 INFO:journalctl@ceph.osd.0.smithi071.stdout:Sep 11 07:14:05 smithi071 sudo[57297]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T07:14:05.839 INFO:journalctl@ceph.osd.0.smithi071.stdout:Sep 11 07:14:05 smithi071 sudo[57297]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T07:14:05.839 INFO:journalctl@ceph.osd.0.smithi071.stdout:Sep 11 07:14:05 smithi071 sudo[57297]: pam_unix(sudo:session): session closed for user root 2024-09-11T07:14:06.265 INFO:journalctl@ceph.osd.0.smithi071.stdout:Sep 11 07:14:06 smithi071 sudo[57302]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-11T07:14:06.265 INFO:journalctl@ceph.osd.0.smithi071.stdout:Sep 11 07:14:06 smithi071 sudo[57302]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T07:14:06.265 INFO:journalctl@ceph.osd.0.smithi071.stdout:Sep 11 07:14:06 smithi071 sudo[57302]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T07:14:06.265 INFO:journalctl@ceph.osd.0.smithi071.stdout:Sep 11 07:14:06 smithi071 sudo[57302]: pam_unix(sudo:session): session closed for user root 2024-09-11T07:14:06.533 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:06 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-11T07:14:06.534 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:06 smithi071 ceph-mon[27013]: osdmap e20: 3 total, 3 up, 3 in 2024-09-11T07:14:06.534 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:06 smithi071 ceph-mon[27013]: pgmap v74: 1 pgs: 1 unknown; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-09-11T07:14:06.534 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:06 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:14:06.534 INFO:journalctl@ceph.osd.1.smithi071.stdout:Sep 11 07:14:06 smithi071 sudo[57309]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-11T07:14:06.534 INFO:journalctl@ceph.osd.1.smithi071.stdout:Sep 11 07:14:06 smithi071 sudo[57309]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T07:14:06.534 INFO:journalctl@ceph.osd.1.smithi071.stdout:Sep 11 07:14:06 smithi071 sudo[57309]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T07:14:06.534 INFO:journalctl@ceph.osd.1.smithi071.stdout:Sep 11 07:14:06 smithi071 sudo[57309]: pam_unix(sudo:session): session closed for user root 2024-09-11T07:14:06.534 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:06 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-11T07:14:06.535 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:06 smithi071 ceph-mon[37727]: osdmap e20: 3 total, 3 up, 3 in 2024-09-11T07:14:06.535 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:06 smithi071 ceph-mon[37727]: pgmap v74: 1 pgs: 1 unknown; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-09-11T07:14:06.535 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:06 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:14:06.783 INFO:journalctl@ceph.osd.1.smithi071.stdout:Sep 11 07:14:06 smithi071 sudo[57332]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-11T07:14:06.783 INFO:journalctl@ceph.osd.1.smithi071.stdout:Sep 11 07:14:06 smithi071 sudo[57332]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T07:14:06.784 INFO:journalctl@ceph.osd.1.smithi071.stdout:Sep 11 07:14:06 smithi071 sudo[57332]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T07:14:06.784 INFO:journalctl@ceph.osd.1.smithi071.stdout:Sep 11 07:14:06 smithi071 sudo[57332]: pam_unix(sudo:session): session closed for user root 2024-09-11T07:14:06.784 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:06 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-11T07:14:06.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:06 smithi196 ceph-mon[30773]: osdmap e20: 3 total, 3 up, 3 in 2024-09-11T07:14:06.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:06 smithi196 ceph-mon[30773]: pgmap v74: 1 pgs: 1 unknown; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-09-11T07:14:06.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:06 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:14:07.053 INFO:journalctl@ceph.osd.2.smithi071.stdout:Sep 11 07:14:06 smithi071 sudo[57337]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-11T07:14:07.053 INFO:journalctl@ceph.osd.2.smithi071.stdout:Sep 11 07:14:06 smithi071 sudo[57337]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T07:14:07.053 INFO:journalctl@ceph.osd.2.smithi071.stdout:Sep 11 07:14:06 smithi071 sudo[57337]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T07:14:07.053 INFO:journalctl@ceph.osd.2.smithi071.stdout:Sep 11 07:14:06 smithi071 sudo[57337]: pam_unix(sudo:session): session closed for user root 2024-09-11T07:14:07.309 INFO:journalctl@ceph.osd.2.smithi071.stdout:Sep 11 07:14:07 smithi071 sudo[57405]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-11T07:14:07.309 INFO:journalctl@ceph.osd.2.smithi071.stdout:Sep 11 07:14:07 smithi071 sudo[57405]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T07:14:07.309 INFO:journalctl@ceph.osd.2.smithi071.stdout:Sep 11 07:14:07 smithi071 sudo[57405]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T07:14:07.309 INFO:journalctl@ceph.osd.2.smithi071.stdout:Sep 11 07:14:07 smithi071 sudo[57405]: pam_unix(sudo:session): session closed for user root 2024-09-11T07:14:07.573 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:07 smithi071 sudo[57415]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-11T07:14:07.574 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:07 smithi071 sudo[57415]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T07:14:07.574 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:07 smithi071 sudo[57415]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T07:14:07.574 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:07 smithi071 sudo[57415]: pam_unix(sudo:session): session closed for user root 2024-09-11T07:14:07.839 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:07 smithi071 ceph-mon[37727]: mgrmap e15: y(active, since 2m), standbys: x 2024-09-11T07:14:07.839 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:07 smithi071 ceph-mon[37727]: osdmap e21: 3 total, 3 up, 3 in 2024-09-11T07:14:07.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:07 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:14:07.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:07 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:14:07.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:07 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:14:07.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:07 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:14:07.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:07 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:14:07.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:07 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:14:07.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:07 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:14:07.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:07 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:14:07.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:07 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:14:07.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:07 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:14:07.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:07 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:14:07.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:07 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:14:07.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:07 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:14:07.841 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:07 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T07:14:07.841 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:07 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:14:07.841 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:07 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:14:07.841 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:07 smithi071 sudo[57438]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-11T07:14:07.841 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:07 smithi071 sudo[57438]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T07:14:07.841 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:07 smithi071 sudo[57438]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T07:14:07.841 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:07 smithi071 sudo[57438]: pam_unix(sudo:session): session closed for user root 2024-09-11T07:14:07.841 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:07 smithi071 ceph-mon[27013]: mgrmap e15: y(active, since 2m), standbys: x 2024-09-11T07:14:07.841 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:07 smithi071 ceph-mon[27013]: osdmap e21: 3 total, 3 up, 3 in 2024-09-11T07:14:07.841 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:07 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:14:07.842 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:07 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:14:07.842 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:07 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:14:07.842 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:07 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:14:07.842 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:07 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:14:07.842 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:07 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:14:07.842 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:07 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:14:07.842 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:07 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:14:07.842 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:07 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:14:07.842 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:07 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:14:07.842 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:07 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:14:07.842 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:07 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:14:07.842 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:07 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:14:07.842 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:07 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T07:14:07.842 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:07 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:14:07.843 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:07 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:14:07.843 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:07 smithi196 ceph-mon[30773]: mgrmap e15: y(active, since 2m), standbys: x 2024-09-11T07:14:07.843 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:07 smithi196 ceph-mon[30773]: osdmap e21: 3 total, 3 up, 3 in 2024-09-11T07:14:07.843 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:07 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:14:07.843 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:07 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:14:07.843 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:07 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:14:07.843 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:07 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:14:07.843 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:07 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:14:07.843 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:07 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:14:07.843 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:07 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:14:07.844 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:07 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:14:07.844 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:07 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:14:07.844 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:07 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:14:07.844 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:07 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:14:07.844 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:07 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:14:07.844 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:07 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:14:07.844 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:07 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T07:14:07.844 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:07 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:14:07.844 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:07 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:14:07.844 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:07 smithi196 sudo[33252]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-11T07:14:07.844 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:07 smithi196 sudo[33252]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T07:14:07.844 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:07 smithi196 sudo[33252]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T07:14:07.844 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:07 smithi196 sudo[33252]: pam_unix(sudo:session): session closed for user root 2024-09-11T07:14:08.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:08 smithi196 sudo[33255]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-11T07:14:08.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:08 smithi196 sudo[33255]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T07:14:08.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:08 smithi196 sudo[33255]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T07:14:08.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:08 smithi196 sudo[33255]: pam_unix(sudo:session): session closed for user root 2024-09-11T07:14:08.589 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:08 smithi071 sudo[57578]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-11T07:14:08.589 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:08 smithi071 sudo[57578]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T07:14:08.589 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:08 smithi071 sudo[57578]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T07:14:08.589 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:08 smithi071 sudo[57578]: pam_unix(sudo:session): session closed for user root 2024-09-11T07:14:08.895 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:08 smithi071 ceph-mon[27013]: Detected new or changed devices on smithi071 2024-09-11T07:14:08.896 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:08 smithi071 ceph-mon[27013]: Adjusting osd_memory_target on smithi071 to 5354M 2024-09-11T07:14:08.896 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:08 smithi071 ceph-mon[27013]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T07:14:08.896 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:08 smithi071 ceph-mon[27013]: from='client.34199 v1:172.21.15.71:0/1617822174' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:14:08.896 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:08 smithi071 ceph-mon[27013]: pgmap v76: 1 pgs: 1 active+clean; 320 KiB data, 872 MiB used, 267 GiB / 268 GiB avail 2024-09-11T07:14:08.896 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:08 smithi071 ceph-mon[27013]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T07:14:08.896 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:08 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T07:14:08.896 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:08 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T07:14:08.896 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:08 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T07:14:08.896 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:08 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:14:08.896 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:08 smithi071 ceph-mon[27013]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T07:14:08.897 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:08 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:14:08.897 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:08 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:14:08.897 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:08 smithi071 ceph-mon[27013]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T07:14:08.897 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:08 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:14:08.897 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:08 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T07:14:08.897 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:08 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:14:08.897 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:08 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:14:08.897 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:08 smithi071 sudo[57582]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-11T07:14:08.897 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:08 smithi071 sudo[57582]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-11T07:14:08.897 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:08 smithi071 sudo[57582]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-11T07:14:08.898 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:08 smithi071 sudo[57582]: pam_unix(sudo:session): session closed for user root 2024-09-11T07:14:08.898 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:08 smithi071 ceph-mon[37727]: Detected new or changed devices on smithi071 2024-09-11T07:14:08.898 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:08 smithi071 ceph-mon[37727]: Adjusting osd_memory_target on smithi071 to 5354M 2024-09-11T07:14:08.898 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:08 smithi071 ceph-mon[37727]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T07:14:08.898 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:08 smithi071 ceph-mon[37727]: from='client.34199 v1:172.21.15.71:0/1617822174' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:14:08.898 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:08 smithi071 ceph-mon[37727]: pgmap v76: 1 pgs: 1 active+clean; 320 KiB data, 872 MiB used, 267 GiB / 268 GiB avail 2024-09-11T07:14:08.898 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:08 smithi071 ceph-mon[37727]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T07:14:08.898 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:08 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T07:14:08.898 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:08 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T07:14:08.898 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:08 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T07:14:08.898 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:08 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:14:08.899 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:08 smithi071 ceph-mon[37727]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T07:14:08.899 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:08 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:14:08.899 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:08 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:14:08.899 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:08 smithi071 ceph-mon[37727]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T07:14:08.899 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:08 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:14:08.899 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:08 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T07:14:08.899 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:08 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:14:08.899 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:08 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:14:09.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:08 smithi196 ceph-mon[30773]: Detected new or changed devices on smithi071 2024-09-11T07:14:09.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:08 smithi196 ceph-mon[30773]: Adjusting osd_memory_target on smithi071 to 5354M 2024-09-11T07:14:09.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:08 smithi196 ceph-mon[30773]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T07:14:09.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:08 smithi196 ceph-mon[30773]: from='client.34199 v1:172.21.15.71:0/1617822174' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:14:09.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:08 smithi196 ceph-mon[30773]: pgmap v76: 1 pgs: 1 active+clean; 320 KiB data, 872 MiB used, 267 GiB / 268 GiB avail 2024-09-11T07:14:09.036 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:08 smithi196 ceph-mon[30773]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T07:14:09.036 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:08 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T07:14:09.036 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:08 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T07:14:09.036 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:08 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T07:14:09.036 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:08 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:14:09.036 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:08 smithi196 ceph-mon[30773]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T07:14:09.037 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:08 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:14:09.037 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:08 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:14:09.037 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:08 smithi196 ceph-mon[30773]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T07:14:09.037 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:08 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:14:09.037 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:08 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T07:14:09.037 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:08 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:14:09.038 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:08 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:14:09.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:09 smithi071 ceph-mon[37727]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T07:14:09.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:09 smithi071 ceph-mon[37727]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T07:14:09.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:09 smithi071 ceph-mon[27013]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T07:14:09.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:09 smithi071 ceph-mon[27013]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T07:14:10.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:09 smithi196 ceph-mon[30773]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-11T07:14:10.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:09 smithi196 ceph-mon[30773]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-11T07:14:11.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:10 smithi196 ceph-mon[30773]: pgmap v77: 1 pgs: 1 active+clean; 320 KiB data, 873 MiB used, 267 GiB / 268 GiB avail 2024-09-11T07:14:11.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:10 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/436323993' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a66db725-6221-44e6-aa78-3bb92957d455"}]: dispatch 2024-09-11T07:14:11.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:10 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/436323993' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a66db725-6221-44e6-aa78-3bb92957d455"}]': finished 2024-09-11T07:14:11.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:10 smithi196 ceph-mon[30773]: osdmap e22: 4 total, 3 up, 4 in 2024-09-11T07:14:11.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:10 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T07:14:11.036 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:10 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/2544670854' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T07:14:11.089 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:10 smithi071 ceph-mon[37727]: pgmap v77: 1 pgs: 1 active+clean; 320 KiB data, 873 MiB used, 267 GiB / 268 GiB avail 2024-09-11T07:14:11.089 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:10 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/436323993' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a66db725-6221-44e6-aa78-3bb92957d455"}]: dispatch 2024-09-11T07:14:11.089 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:10 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/436323993' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a66db725-6221-44e6-aa78-3bb92957d455"}]': finished 2024-09-11T07:14:11.089 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:10 smithi071 ceph-mon[37727]: osdmap e22: 4 total, 3 up, 4 in 2024-09-11T07:14:11.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:10 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T07:14:11.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:10 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/2544670854' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T07:14:11.090 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:10 smithi071 ceph-mon[27013]: pgmap v77: 1 pgs: 1 active+clean; 320 KiB data, 873 MiB used, 267 GiB / 268 GiB avail 2024-09-11T07:14:11.090 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:10 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/436323993' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a66db725-6221-44e6-aa78-3bb92957d455"}]: dispatch 2024-09-11T07:14:11.090 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:10 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/436323993' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a66db725-6221-44e6-aa78-3bb92957d455"}]': finished 2024-09-11T07:14:11.090 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:10 smithi071 ceph-mon[27013]: osdmap e22: 4 total, 3 up, 4 in 2024-09-11T07:14:11.090 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:10 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T07:14:11.090 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:10 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/2544670854' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T07:14:12.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:12 smithi071 ceph-mon[27013]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 5.3 KiB/s rd, 256 KiB/s wr, 17 op/s 2024-09-11T07:14:12.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:12 smithi071 ceph-mon[37727]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 5.3 KiB/s rd, 256 KiB/s wr, 17 op/s 2024-09-11T07:14:13.034 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:12 smithi196 ceph-mon[30773]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 5.3 KiB/s rd, 256 KiB/s wr, 17 op/s 2024-09-11T07:14:14.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:14 smithi071 ceph-mon[27013]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 3.8 KiB/s rd, 184 KiB/s wr, 12 op/s 2024-09-11T07:14:14.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:14 smithi071 ceph-mon[37727]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 3.8 KiB/s rd, 184 KiB/s wr, 12 op/s 2024-09-11T07:14:15.034 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:14 smithi196 ceph-mon[30773]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 3.8 KiB/s rd, 184 KiB/s wr, 12 op/s 2024-09-11T07:14:16.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:16 smithi071 ceph-mon[27013]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 3.0 KiB/s rd, 144 KiB/s wr, 9 op/s 2024-09-11T07:14:16.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:16 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-11T07:14:16.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:16 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:14:16.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:16 smithi071 ceph-mon[27013]: Deploying daemon osd.3 on smithi071 2024-09-11T07:14:16.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:16 smithi071 ceph-mon[37727]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 3.0 KiB/s rd, 144 KiB/s wr, 9 op/s 2024-09-11T07:14:16.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:16 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-11T07:14:16.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:16 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:14:16.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:16 smithi071 ceph-mon[37727]: Deploying daemon osd.3 on smithi071 2024-09-11T07:14:17.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:16 smithi196 ceph-mon[30773]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 3.0 KiB/s rd, 144 KiB/s wr, 9 op/s 2024-09-11T07:14:17.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:16 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-11T07:14:17.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:16 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:14:17.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:16 smithi196 ceph-mon[30773]: Deploying daemon osd.3 on smithi071 2024-09-11T07:14:18.838 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:18 smithi071 ceph-mon[27013]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 2.7 KiB/s rd, 131 KiB/s wr, 8 op/s 2024-09-11T07:14:18.839 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:18 smithi071 ceph-mon[37727]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 2.7 KiB/s rd, 131 KiB/s wr, 8 op/s 2024-09-11T07:14:19.034 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:18 smithi196 ceph-mon[30773]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 2.7 KiB/s rd, 131 KiB/s wr, 8 op/s 2024-09-11T07:14:20.939 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:20 smithi071 ceph-mon[37727]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 2.7 KiB/s rd, 131 KiB/s wr, 8 op/s 2024-09-11T07:14:20.939 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:20 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:14:20.939 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:20 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:14:20.939 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:20 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:14:20.939 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:20 smithi071 ceph-mon[27013]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 2.7 KiB/s rd, 131 KiB/s wr, 8 op/s 2024-09-11T07:14:20.940 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:20 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:14:20.940 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:20 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:14:20.940 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:20 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:14:21.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:20 smithi196 ceph-mon[30773]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 2.7 KiB/s rd, 131 KiB/s wr, 8 op/s 2024-09-11T07:14:21.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:20 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:14:21.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:20 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:14:21.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:20 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:14:22.894 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:22 smithi071 ceph-mon[27013]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 2.3 KiB/s rd, 111 KiB/s wr, 7 op/s 2024-09-11T07:14:22.894 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:22 smithi071 ceph-mon[37727]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 2.3 KiB/s rd, 111 KiB/s wr, 7 op/s 2024-09-11T07:14:23.034 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:22 smithi196 ceph-mon[30773]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 2.3 KiB/s rd, 111 KiB/s wr, 7 op/s 2024-09-11T07:14:23.989 INFO:teuthology.orchestra.run.smithi071.stdout:Created osd(s) 3 on host 'smithi071' 2024-09-11T07:14:24.566 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:24 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:14:24.566 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:24 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:14:24.567 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:24 smithi071 ceph-mon[27013]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T07:14:24.567 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:24 smithi071 ceph-mon[27013]: from='osd.3 v1:172.21.15.71:6813/295183903' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-11T07:14:24.567 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:24 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:14:24.567 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:24 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:14:24.567 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:24 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:14:24.567 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:24 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:14:24.567 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:24 smithi071 ceph-mon[37727]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T07:14:24.567 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:24 smithi071 ceph-mon[37727]: from='osd.3 v1:172.21.15.71:6813/295183903' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-11T07:14:24.567 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:24 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:14:24.567 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:24 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:14:24.590 DEBUG:teuthology.orchestra.run.smithi071:osd.3> sudo journalctl -f -n 0 -u ceph-a81cffe6-700c-11ef-bcea-c7b262605968@osd.3.service 2024-09-11T07:14:24.593 INFO:tasks.cephadm:Deploying osd.4 on smithi196 with /dev/vg_nvme/lv_4... 2024-09-11T07:14:24.593 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a81cffe6-700c-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-09-11T07:14:24.621 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:24 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:14:24.621 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:24 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:14:24.622 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:24 smithi196 ceph-mon[30773]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T07:14:24.622 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:24 smithi196 ceph-mon[30773]: from='osd.3 v1:172.21.15.71:6813/295183903' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-11T07:14:24.622 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:24 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:14:24.622 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:24 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:14:24.838 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:14:25.397 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:25 smithi071 ceph-mon[37727]: from='osd.3 v1:172.21.15.71:6813/295183903' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-11T07:14:25.398 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:25 smithi071 ceph-mon[37727]: osdmap e23: 4 total, 3 up, 4 in 2024-09-11T07:14:25.398 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:25 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T07:14:25.398 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:25 smithi071 ceph-mon[37727]: from='osd.3 v1:172.21.15.71:6813/295183903' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-09-11T07:14:25.398 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:25 smithi071 ceph-mon[27013]: from='osd.3 v1:172.21.15.71:6813/295183903' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-11T07:14:25.398 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:25 smithi071 ceph-mon[27013]: osdmap e23: 4 total, 3 up, 4 in 2024-09-11T07:14:25.398 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:25 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T07:14:25.398 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:25 smithi071 ceph-mon[27013]: from='osd.3 v1:172.21.15.71:6813/295183903' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-09-11T07:14:25.398 INFO:journalctl@ceph.osd.3.smithi071.stdout:Sep 11 07:14:25 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-3[59319]: 2024-09-11T07:14:25.329+0000 7f978c919640 -1 osd.3 0 waiting for initial osdmap 2024-09-11T07:14:25.399 INFO:journalctl@ceph.osd.3.smithi071.stdout:Sep 11 07:14:25 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-3[59319]: 2024-09-11T07:14:25.338+0000 7f9788752640 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T07:14:25.534 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:25 smithi196 ceph-mon[30773]: from='osd.3 v1:172.21.15.71:6813/295183903' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-11T07:14:25.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:25 smithi196 ceph-mon[30773]: osdmap e23: 4 total, 3 up, 4 in 2024-09-11T07:14:25.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:25 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T07:14:25.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:25 smithi196 ceph-mon[30773]: from='osd.3 v1:172.21.15.71:6813/295183903' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-09-11T07:14:26.444 INFO:teuthology.orchestra.run.smithi196.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-09-11T07:14:26.526 INFO:teuthology.orchestra.run.smithi196.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-09-11T07:14:26.539 INFO:teuthology.orchestra.run.smithi196.stderr: stderr: 10+0 records in 2024-09-11T07:14:26.539 INFO:teuthology.orchestra.run.smithi196.stderr:10+0 records out 2024-09-11T07:14:26.539 INFO:teuthology.orchestra.run.smithi196.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114384 s, 917 MB/s 2024-09-11T07:14:26.539 INFO:teuthology.orchestra.run.smithi196.stderr:--> Zapping successful for: 2024-09-11T07:14:26.597 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:26 smithi071 ceph-mon[27013]: from='osd.3 v1:172.21.15.71:6813/295183903' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-09-11T07:14:26.597 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:26 smithi071 ceph-mon[27013]: osdmap e24: 4 total, 3 up, 4 in 2024-09-11T07:14:26.597 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:26 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T07:14:26.597 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:26 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T07:14:26.597 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:26 smithi071 ceph-mon[27013]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T07:14:26.597 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:26 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T07:14:26.597 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:26 smithi071 ceph-mon[37727]: from='osd.3 v1:172.21.15.71:6813/295183903' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-09-11T07:14:26.597 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:26 smithi071 ceph-mon[37727]: osdmap e24: 4 total, 3 up, 4 in 2024-09-11T07:14:26.598 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:26 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T07:14:26.598 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:26 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T07:14:26.598 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:26 smithi071 ceph-mon[37727]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T07:14:26.598 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:26 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T07:14:26.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:26 smithi196 ceph-mon[30773]: from='osd.3 v1:172.21.15.71:6813/295183903' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-09-11T07:14:26.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:26 smithi196 ceph-mon[30773]: osdmap e24: 4 total, 3 up, 4 in 2024-09-11T07:14:26.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:26 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T07:14:26.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:26 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T07:14:26.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:26 smithi196 ceph-mon[30773]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-11T07:14:26.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:26 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T07:14:27.093 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a81cffe6-700c-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi196:vg_nvme/lv_4 2024-09-11T07:14:27.395 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:14:27.589 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:27 smithi071 ceph-mon[37727]: purged_snaps scrub starts 2024-09-11T07:14:27.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:27 smithi071 ceph-mon[37727]: purged_snaps scrub ok 2024-09-11T07:14:27.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:27 smithi071 ceph-mon[37727]: osd.3 v1:172.21.15.71:6813/295183903 boot 2024-09-11T07:14:27.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:27 smithi071 ceph-mon[37727]: osdmap e25: 4 total, 4 up, 4 in 2024-09-11T07:14:27.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:27 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T07:14:27.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:27 smithi071 ceph-mon[37727]: Detected new or changed devices on smithi071 2024-09-11T07:14:27.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:27 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:14:27.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:27 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:14:27.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:27 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:14:27.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:27 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:14:27.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:27 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:14:27.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:27 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:14:27.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:27 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:14:27.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:27 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:14:27.591 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:27 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:14:27.591 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:27 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:14:27.591 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:27 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:14:27.591 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:27 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:14:27.591 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:27 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:14:27.591 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:27 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:14:27.591 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:27 smithi071 ceph-mon[27013]: purged_snaps scrub starts 2024-09-11T07:14:27.591 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:27 smithi071 ceph-mon[27013]: purged_snaps scrub ok 2024-09-11T07:14:27.591 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:27 smithi071 ceph-mon[27013]: osd.3 v1:172.21.15.71:6813/295183903 boot 2024-09-11T07:14:27.591 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:27 smithi071 ceph-mon[27013]: osdmap e25: 4 total, 4 up, 4 in 2024-09-11T07:14:27.592 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:27 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T07:14:27.592 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:27 smithi071 ceph-mon[27013]: Detected new or changed devices on smithi071 2024-09-11T07:14:27.592 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:27 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:14:27.592 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:27 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:14:27.592 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:27 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:14:27.592 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:27 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:14:27.592 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:27 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:14:27.592 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:27 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:14:27.592 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:27 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:14:27.592 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:27 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:14:27.592 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:27 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:14:27.592 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:27 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:14:27.592 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:27 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:14:27.592 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:27 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:14:27.593 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:27 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:14:27.593 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:27 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:14:27.601 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:27 smithi196 ceph-mon[30773]: purged_snaps scrub starts 2024-09-11T07:14:27.601 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:27 smithi196 ceph-mon[30773]: purged_snaps scrub ok 2024-09-11T07:14:27.601 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:27 smithi196 ceph-mon[30773]: osd.3 v1:172.21.15.71:6813/295183903 boot 2024-09-11T07:14:27.601 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:27 smithi196 ceph-mon[30773]: osdmap e25: 4 total, 4 up, 4 in 2024-09-11T07:14:27.601 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:27 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T07:14:27.601 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:27 smithi196 ceph-mon[30773]: Detected new or changed devices on smithi071 2024-09-11T07:14:27.601 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:27 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:14:27.602 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:27 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:14:27.602 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:27 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:14:27.602 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:27 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:14:27.602 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:27 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:14:27.602 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:27 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:14:27.602 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:27 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:14:27.602 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:27 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:14:27.602 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:27 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:14:27.602 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:27 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:14:27.602 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:27 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:14:27.602 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:27 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:14:27.602 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:27 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:14:27.602 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:27 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:14:28.633 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:28 smithi196 ceph-mon[30773]: Adjusting osd_memory_target on smithi071 to 4016M 2024-09-11T07:14:28.633 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:28 smithi196 ceph-mon[30773]: osdmap e26: 4 total, 4 up, 4 in 2024-09-11T07:14:28.634 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:28 smithi196 ceph-mon[30773]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:14:28.760 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:28 smithi071 ceph-mon[37727]: Adjusting osd_memory_target on smithi071 to 4016M 2024-09-11T07:14:28.760 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:28 smithi071 ceph-mon[37727]: osdmap e26: 4 total, 4 up, 4 in 2024-09-11T07:14:28.760 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:28 smithi071 ceph-mon[37727]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:14:28.760 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:28 smithi071 ceph-mon[27013]: Adjusting osd_memory_target on smithi071 to 4016M 2024-09-11T07:14:28.760 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:28 smithi071 ceph-mon[27013]: osdmap e26: 4 total, 4 up, 4 in 2024-09-11T07:14:28.761 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:28 smithi071 ceph-mon[27013]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:14:29.645 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:29 smithi196 ceph-mon[30773]: from='client.24232 v1:172.21.15.196:0/3029692111' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:14:29.645 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:29 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T07:14:29.645 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:29 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T07:14:29.645 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:29 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:14:29.730 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:29 smithi071 ceph-mon[37727]: from='client.24232 v1:172.21.15.196:0/3029692111' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:14:29.730 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:29 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T07:14:29.730 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:29 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T07:14:29.730 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:29 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:14:29.730 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:29 smithi071 ceph-mon[27013]: from='client.24232 v1:172.21.15.196:0/3029692111' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:14:29.730 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:29 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T07:14:29.730 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:29 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T07:14:29.730 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:29 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:14:30.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:30 smithi071 ceph-mon[27013]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:14:30.839 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:30 smithi071 ceph-mon[37727]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:14:30.890 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:30 smithi196 ceph-mon[30773]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:14:31.732 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:31 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.196:0/3321912916' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1ac47bc6-7006-4b9f-afe3-676e6f12cee8"}]: dispatch 2024-09-11T07:14:31.732 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:31 smithi196 ceph-mon[30773]: from='client.24238 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1ac47bc6-7006-4b9f-afe3-676e6f12cee8"}]: dispatch 2024-09-11T07:14:31.732 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:31 smithi196 ceph-mon[30773]: from='client.24238 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1ac47bc6-7006-4b9f-afe3-676e6f12cee8"}]': finished 2024-09-11T07:14:31.733 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:31 smithi196 ceph-mon[30773]: osdmap e27: 5 total, 4 up, 5 in 2024-09-11T07:14:31.733 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:31 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T07:14:31.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:31 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.196:0/3321912916' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1ac47bc6-7006-4b9f-afe3-676e6f12cee8"}]: dispatch 2024-09-11T07:14:31.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:31 smithi071 ceph-mon[27013]: from='client.24238 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1ac47bc6-7006-4b9f-afe3-676e6f12cee8"}]: dispatch 2024-09-11T07:14:31.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:31 smithi071 ceph-mon[27013]: from='client.24238 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1ac47bc6-7006-4b9f-afe3-676e6f12cee8"}]': finished 2024-09-11T07:14:31.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:31 smithi071 ceph-mon[27013]: osdmap e27: 5 total, 4 up, 5 in 2024-09-11T07:14:31.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:31 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T07:14:31.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:31 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.196:0/3321912916' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1ac47bc6-7006-4b9f-afe3-676e6f12cee8"}]: dispatch 2024-09-11T07:14:31.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:31 smithi071 ceph-mon[37727]: from='client.24238 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1ac47bc6-7006-4b9f-afe3-676e6f12cee8"}]: dispatch 2024-09-11T07:14:31.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:31 smithi071 ceph-mon[37727]: from='client.24238 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1ac47bc6-7006-4b9f-afe3-676e6f12cee8"}]': finished 2024-09-11T07:14:31.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:31 smithi071 ceph-mon[37727]: osdmap e27: 5 total, 4 up, 5 in 2024-09-11T07:14:31.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:31 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T07:14:32.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:32 smithi071 ceph-mon[27013]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:14:32.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:32 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.196:0/1233826355' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T07:14:32.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:32 smithi071 ceph-mon[37727]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:14:32.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:32 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.196:0/1233826355' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T07:14:33.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:32 smithi196 ceph-mon[30773]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:14:33.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:32 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.196:0/1233826355' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T07:14:34.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:34 smithi071 ceph-mon[27013]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:14:34.839 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:34 smithi071 ceph-mon[37727]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:14:34.976 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:34 smithi196 ceph-mon[30773]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:14:36.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:36 smithi196 ceph-mon[30773]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:14:36.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:36 smithi071 ceph-mon[27013]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:14:36.839 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:36 smithi071 ceph-mon[37727]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:14:37.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:37 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-11T07:14:37.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:37 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:14:37.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:37 smithi196 ceph-mon[30773]: Deploying daemon osd.4 on smithi196 2024-09-11T07:14:38.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:37 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-11T07:14:38.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:37 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:14:38.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:37 smithi071 ceph-mon[27013]: Deploying daemon osd.4 on smithi196 2024-09-11T07:14:38.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:37 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-11T07:14:38.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:37 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:14:38.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:37 smithi071 ceph-mon[37727]: Deploying daemon osd.4 on smithi196 2024-09-11T07:14:39.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:38 smithi196 ceph-mon[30773]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:14:39.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:38 smithi071 ceph-mon[27013]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:14:39.089 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:38 smithi071 ceph-mon[37727]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:14:40.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:40 smithi196 ceph-mon[30773]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:14:41.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:40 smithi071 ceph-mon[27013]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:14:41.089 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:40 smithi071 ceph-mon[37727]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:14:42.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:42 smithi196 ceph-mon[30773]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:14:42.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:42 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:14:42.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:42 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:14:42.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:42 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:14:43.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:42 smithi071 ceph-mon[27013]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:14:43.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:42 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:14:43.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:42 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:14:43.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:42 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:14:43.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:42 smithi071 ceph-mon[37727]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:14:43.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:42 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:14:43.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:42 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:14:43.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:42 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:14:44.715 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:44 smithi196 ceph-mon[30773]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:14:44.715 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:44 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:14:44.715 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:44 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:14:44.999 INFO:teuthology.orchestra.run.smithi196.stdout:Created osd(s) 4 on host 'smithi196' 2024-09-11T07:14:45.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:44 smithi071 ceph-mon[27013]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:14:45.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:44 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:14:45.090 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:44 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:14:45.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:44 smithi071 ceph-mon[37727]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:14:45.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:44 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:14:45.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:44 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:14:45.560 DEBUG:teuthology.orchestra.run.smithi196:osd.4> sudo journalctl -f -n 0 -u ceph-a81cffe6-700c-11ef-bcea-c7b262605968@osd.4.service 2024-09-11T07:14:45.562 INFO:tasks.cephadm:Deploying osd.5 on smithi196 with /dev/vg_nvme/lv_3... 2024-09-11T07:14:45.562 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a81cffe6-700c-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-09-11T07:14:45.846 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:14:45.967 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:45 smithi196 ceph-mon[30773]: from='osd.4 v1:172.21.15.196:6800/4211374096' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-11T07:14:45.967 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:45 smithi196 ceph-mon[30773]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-11T07:14:45.967 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:45 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:14:45.967 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:45 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:14:46.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:45 smithi071 ceph-mon[27013]: from='osd.4 v1:172.21.15.196:6800/4211374096' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-11T07:14:46.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:45 smithi071 ceph-mon[27013]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-11T07:14:46.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:45 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:14:46.090 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:45 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:14:46.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:45 smithi071 ceph-mon[37727]: from='osd.4 v1:172.21.15.196:6800/4211374096' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-11T07:14:46.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:45 smithi071 ceph-mon[37727]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-11T07:14:46.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:45 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:14:46.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:45 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:14:46.888 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:46 smithi196 ceph-mon[30773]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:14:46.888 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:46 smithi196 ceph-mon[30773]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-11T07:14:46.888 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:46 smithi196 ceph-mon[30773]: osdmap e28: 5 total, 4 up, 5 in 2024-09-11T07:14:46.888 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:46 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T07:14:46.888 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:46 smithi196 ceph-mon[30773]: from='osd.4 v1:172.21.15.196:6800/4211374096' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-11T07:14:46.888 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:46 smithi196 ceph-mon[30773]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-11T07:14:46.888 INFO:journalctl@ceph.osd.4.smithi196.stdout:Sep 11 07:14:46 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-4[35524]: 2024-09-11T07:14:46.613+0000 7f8cfe730640 -1 osd.4 0 waiting for initial osdmap 2024-09-11T07:14:46.888 INFO:journalctl@ceph.osd.4.smithi196.stdout:Sep 11 07:14:46 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-4[35524]: 2024-09-11T07:14:46.618+0000 7f8cfa569640 -1 osd.4 29 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T07:14:47.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:46 smithi071 ceph-mon[27013]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:14:47.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:46 smithi071 ceph-mon[27013]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-11T07:14:47.090 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:46 smithi071 ceph-mon[27013]: osdmap e28: 5 total, 4 up, 5 in 2024-09-11T07:14:47.090 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:46 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T07:14:47.090 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:46 smithi071 ceph-mon[27013]: from='osd.4 v1:172.21.15.196:6800/4211374096' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-11T07:14:47.090 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:46 smithi071 ceph-mon[27013]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-11T07:14:47.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:46 smithi071 ceph-mon[37727]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:14:47.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:46 smithi071 ceph-mon[37727]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-11T07:14:47.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:46 smithi071 ceph-mon[37727]: osdmap e28: 5 total, 4 up, 5 in 2024-09-11T07:14:47.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:46 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T07:14:47.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:46 smithi071 ceph-mon[37727]: from='osd.4 v1:172.21.15.196:6800/4211374096' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-11T07:14:47.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:46 smithi071 ceph-mon[37727]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-11T07:14:47.715 INFO:teuthology.orchestra.run.smithi196.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-09-11T07:14:47.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:47 smithi196 ceph-mon[30773]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-09-11T07:14:47.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:47 smithi196 ceph-mon[30773]: osdmap e29: 5 total, 4 up, 5 in 2024-09-11T07:14:47.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:47 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T07:14:47.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:47 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T07:14:47.804 INFO:teuthology.orchestra.run.smithi196.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-09-11T07:14:47.819 INFO:teuthology.orchestra.run.smithi196.stderr: stderr: 10+0 records in 2024-09-11T07:14:47.819 INFO:teuthology.orchestra.run.smithi196.stderr:10+0 records out 2024-09-11T07:14:47.819 INFO:teuthology.orchestra.run.smithi196.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0130047 s, 806 MB/s 2024-09-11T07:14:47.820 INFO:teuthology.orchestra.run.smithi196.stderr:--> Zapping successful for: 2024-09-11T07:14:48.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:47 smithi071 ceph-mon[27013]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-09-11T07:14:48.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:47 smithi071 ceph-mon[27013]: osdmap e29: 5 total, 4 up, 5 in 2024-09-11T07:14:48.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:47 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T07:14:48.090 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:47 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T07:14:48.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:47 smithi071 ceph-mon[37727]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-09-11T07:14:48.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:47 smithi071 ceph-mon[37727]: osdmap e29: 5 total, 4 up, 5 in 2024-09-11T07:14:48.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:47 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T07:14:48.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:47 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T07:14:48.610 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a81cffe6-700c-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi196:vg_nvme/lv_3 2024-09-11T07:14:49.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:48 smithi196 ceph-mon[30773]: purged_snaps scrub starts 2024-09-11T07:14:49.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:48 smithi196 ceph-mon[30773]: purged_snaps scrub ok 2024-09-11T07:14:49.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:48 smithi196 ceph-mon[30773]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:14:49.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:48 smithi196 ceph-mon[30773]: osd.4 v1:172.21.15.196:6800/4211374096 boot 2024-09-11T07:14:49.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:48 smithi196 ceph-mon[30773]: osdmap e30: 5 total, 5 up, 5 in 2024-09-11T07:14:49.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:48 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T07:14:49.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:48 smithi196 ceph-mon[30773]: osdmap e31: 5 total, 5 up, 5 in 2024-09-11T07:14:49.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:48 smithi071 ceph-mon[27013]: purged_snaps scrub starts 2024-09-11T07:14:49.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:48 smithi071 ceph-mon[27013]: purged_snaps scrub ok 2024-09-11T07:14:49.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:48 smithi071 ceph-mon[27013]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:14:49.090 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:48 smithi071 ceph-mon[27013]: osd.4 v1:172.21.15.196:6800/4211374096 boot 2024-09-11T07:14:49.090 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:48 smithi071 ceph-mon[27013]: osdmap e30: 5 total, 5 up, 5 in 2024-09-11T07:14:49.090 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:48 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T07:14:49.090 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:48 smithi071 ceph-mon[27013]: osdmap e31: 5 total, 5 up, 5 in 2024-09-11T07:14:49.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:48 smithi071 ceph-mon[37727]: purged_snaps scrub starts 2024-09-11T07:14:49.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:48 smithi071 ceph-mon[37727]: purged_snaps scrub ok 2024-09-11T07:14:49.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:48 smithi071 ceph-mon[37727]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-11T07:14:49.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:48 smithi071 ceph-mon[37727]: osd.4 v1:172.21.15.196:6800/4211374096 boot 2024-09-11T07:14:49.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:48 smithi071 ceph-mon[37727]: osdmap e30: 5 total, 5 up, 5 in 2024-09-11T07:14:49.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:48 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T07:14:49.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:48 smithi071 ceph-mon[37727]: osdmap e31: 5 total, 5 up, 5 in 2024-09-11T07:14:49.091 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:14:50.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:50 smithi196 ceph-mon[30773]: osdmap e32: 5 total, 5 up, 5 in 2024-09-11T07:14:50.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:50 smithi196 ceph-mon[30773]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T07:14:50.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:50 smithi071 ceph-mon[27013]: osdmap e32: 5 total, 5 up, 5 in 2024-09-11T07:14:50.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:50 smithi071 ceph-mon[27013]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T07:14:50.839 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:50 smithi071 ceph-mon[37727]: osdmap e32: 5 total, 5 up, 5 in 2024-09-11T07:14:50.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:50 smithi071 ceph-mon[37727]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T07:14:52.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:52 smithi196 ceph-mon[30773]: Detected new or changed devices on smithi196 2024-09-11T07:14:52.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:52 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:14:52.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:52 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:14:52.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:52 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:14:52.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:52 smithi196 ceph-mon[30773]: Adjusting osd_memory_target on smithi196 to 17088M 2024-09-11T07:14:52.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:52 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:14:52.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:52 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:14:52.786 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:52 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:14:52.786 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:52 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:14:52.786 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:52 smithi196 ceph-mon[30773]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 116 KiB/s, 0 objects/s recovering 2024-09-11T07:14:52.786 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:52 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:14:52.786 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:52 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:14:52.786 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:52 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:14:52.786 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:52 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:14:52.786 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:52 smithi196 ceph-mon[30773]: from='client.24259 v1:172.21.15.196:0/2356169723' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:14:52.786 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:52 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T07:14:52.786 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:52 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T07:14:52.786 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:52 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:14:52.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:52 smithi071 ceph-mon[27013]: Detected new or changed devices on smithi196 2024-09-11T07:14:52.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:52 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:14:52.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:52 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:14:52.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:52 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:14:52.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:52 smithi071 ceph-mon[27013]: Adjusting osd_memory_target on smithi196 to 17088M 2024-09-11T07:14:52.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:52 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:14:52.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:52 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:14:52.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:52 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:14:52.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:52 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:14:52.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:52 smithi071 ceph-mon[27013]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 116 KiB/s, 0 objects/s recovering 2024-09-11T07:14:52.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:52 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:14:52.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:52 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:14:52.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:52 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:14:52.841 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:52 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:14:52.841 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:52 smithi071 ceph-mon[27013]: from='client.24259 v1:172.21.15.196:0/2356169723' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:14:52.841 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:52 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T07:14:52.841 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:52 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T07:14:52.841 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:52 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:14:52.841 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:52 smithi071 ceph-mon[37727]: Detected new or changed devices on smithi196 2024-09-11T07:14:52.841 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:52 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:14:52.841 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:52 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:14:52.841 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:52 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:14:52.841 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:52 smithi071 ceph-mon[37727]: Adjusting osd_memory_target on smithi196 to 17088M 2024-09-11T07:14:52.842 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:52 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:14:52.842 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:52 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:14:52.842 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:52 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:14:52.842 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:52 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:14:52.842 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:52 smithi071 ceph-mon[37727]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 116 KiB/s, 0 objects/s recovering 2024-09-11T07:14:52.842 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:52 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:14:52.842 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:52 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:14:52.842 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:52 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:14:52.842 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:52 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:14:52.842 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:52 smithi071 ceph-mon[37727]: from='client.24259 v1:172.21.15.196:0/2356169723' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:14:52.843 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:52 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T07:14:52.843 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:52 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T07:14:52.843 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:52 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:14:55.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:54 smithi196 ceph-mon[30773]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-11T07:14:55.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:54 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.196:0/2297384039' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "683fad79-fdac-4c84-aec8-6d8d3d4ec586"}]: dispatch 2024-09-11T07:14:55.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:54 smithi196 ceph-mon[30773]: from='client.24265 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "683fad79-fdac-4c84-aec8-6d8d3d4ec586"}]: dispatch 2024-09-11T07:14:55.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:54 smithi196 ceph-mon[30773]: from='client.24265 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "683fad79-fdac-4c84-aec8-6d8d3d4ec586"}]': finished 2024-09-11T07:14:55.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:54 smithi196 ceph-mon[30773]: osdmap e33: 6 total, 5 up, 6 in 2024-09-11T07:14:55.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:54 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T07:14:55.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:54 smithi071 ceph-mon[27013]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-11T07:14:55.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:54 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.196:0/2297384039' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "683fad79-fdac-4c84-aec8-6d8d3d4ec586"}]: dispatch 2024-09-11T07:14:55.090 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:54 smithi071 ceph-mon[27013]: from='client.24265 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "683fad79-fdac-4c84-aec8-6d8d3d4ec586"}]: dispatch 2024-09-11T07:14:55.090 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:54 smithi071 ceph-mon[27013]: from='client.24265 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "683fad79-fdac-4c84-aec8-6d8d3d4ec586"}]': finished 2024-09-11T07:14:55.090 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:54 smithi071 ceph-mon[27013]: osdmap e33: 6 total, 5 up, 6 in 2024-09-11T07:14:55.090 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:54 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T07:14:55.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:54 smithi071 ceph-mon[37727]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-11T07:14:55.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:54 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.196:0/2297384039' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "683fad79-fdac-4c84-aec8-6d8d3d4ec586"}]: dispatch 2024-09-11T07:14:55.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:54 smithi071 ceph-mon[37727]: from='client.24265 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "683fad79-fdac-4c84-aec8-6d8d3d4ec586"}]: dispatch 2024-09-11T07:14:55.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:54 smithi071 ceph-mon[37727]: from='client.24265 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "683fad79-fdac-4c84-aec8-6d8d3d4ec586"}]': finished 2024-09-11T07:14:55.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:54 smithi071 ceph-mon[37727]: osdmap e33: 6 total, 5 up, 6 in 2024-09-11T07:14:55.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:54 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T07:14:56.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:55 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.196:0/43221951' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T07:14:56.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:55 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.196:0/43221951' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T07:14:56.089 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:55 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.196:0/43221951' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T07:14:57.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:56 smithi196 ceph-mon[30773]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-09-11T07:14:57.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:56 smithi071 ceph-mon[27013]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-09-11T07:14:57.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:56 smithi071 ceph-mon[37727]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-09-11T07:14:58.740 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:14:58 smithi196 ceph-mon[30773]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-09-11T07:14:58.892 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:14:58 smithi071 ceph-mon[27013]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-09-11T07:14:58.892 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:14:58 smithi071 ceph-mon[37727]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-09-11T07:15:01.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:00 smithi196 ceph-mon[30773]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-11T07:15:01.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:00 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-11T07:15:01.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:00 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:15:01.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:00 smithi196 ceph-mon[30773]: Deploying daemon osd.5 on smithi196 2024-09-11T07:15:01.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:00 smithi071 ceph-mon[27013]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-11T07:15:01.090 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:00 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-11T07:15:01.090 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:00 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:15:01.090 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:00 smithi071 ceph-mon[27013]: Deploying daemon osd.5 on smithi196 2024-09-11T07:15:01.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:00 smithi071 ceph-mon[37727]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-11T07:15:01.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:00 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-11T07:15:01.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:00 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:15:01.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:00 smithi071 ceph-mon[37727]: Deploying daemon osd.5 on smithi196 2024-09-11T07:15:02.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:02 smithi196 ceph-mon[30773]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T07:15:03.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:02 smithi071 ceph-mon[27013]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T07:15:03.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:02 smithi071 ceph-mon[37727]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T07:15:05.034 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:04 smithi196 ceph-mon[30773]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T07:15:05.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:04 smithi071 ceph-mon[27013]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T07:15:05.089 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:04 smithi071 ceph-mon[37727]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T07:15:05.975 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:05 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:15:05.975 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:05 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:15:05.975 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:05 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:15:06.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:05 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:15:06.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:05 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:15:06.090 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:05 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:15:06.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:05 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:15:06.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:05 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:15:06.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:05 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:15:07.110 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:06 smithi196 ceph-mon[30773]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T07:15:07.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:06 smithi071 ceph-mon[27013]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T07:15:07.339 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:06 smithi071 ceph-mon[37727]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T07:15:08.095 INFO:teuthology.orchestra.run.smithi196.stdout:Created osd(s) 5 on host 'smithi196' 2024-09-11T07:15:08.713 DEBUG:teuthology.orchestra.run.smithi196:osd.5> sudo journalctl -f -n 0 -u ceph-a81cffe6-700c-11ef-bcea-c7b262605968@osd.5.service 2024-09-11T07:15:08.716 INFO:tasks.cephadm:Deploying osd.6 on smithi196 with /dev/vg_nvme/lv_2... 2024-09-11T07:15:08.716 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a81cffe6-700c-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-09-11T07:15:09.003 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:15:09.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:08 smithi196 ceph-mon[30773]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T07:15:09.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:08 smithi196 ceph-mon[30773]: from='osd.5 v1:172.21.15.196:6804/3113061101' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-11T07:15:09.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:08 smithi196 ceph-mon[30773]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-11T07:15:09.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:08 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:15:09.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:08 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:15:09.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:08 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:15:09.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:08 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:15:09.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:08 smithi071 ceph-mon[27013]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T07:15:09.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:08 smithi071 ceph-mon[27013]: from='osd.5 v1:172.21.15.196:6804/3113061101' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-11T07:15:09.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:08 smithi071 ceph-mon[27013]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-11T07:15:09.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:08 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:15:09.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:08 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:15:09.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:08 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:15:09.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:08 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:15:09.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:08 smithi071 ceph-mon[37727]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T07:15:09.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:08 smithi071 ceph-mon[37727]: from='osd.5 v1:172.21.15.196:6804/3113061101' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-11T07:15:09.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:08 smithi071 ceph-mon[37727]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-11T07:15:09.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:08 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:15:09.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:08 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:15:09.341 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:08 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:15:09.341 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:08 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:15:10.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:09 smithi196 ceph-mon[30773]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-11T07:15:10.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:09 smithi196 ceph-mon[30773]: osdmap e34: 6 total, 5 up, 6 in 2024-09-11T07:15:10.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:09 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T07:15:10.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:09 smithi196 ceph-mon[30773]: from='osd.5 v1:172.21.15.196:6804/3113061101' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-11T07:15:10.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:09 smithi196 ceph-mon[30773]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-11T07:15:10.194 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 11 07:15:09 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-5[41502]: 2024-09-11T07:15:09.908+0000 7fc9073c1640 -1 osd.5 0 waiting for initial osdmap 2024-09-11T07:15:10.194 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 11 07:15:09 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-5[41502]: 2024-09-11T07:15:09.912+0000 7fc9021f8640 -1 osd.5 35 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T07:15:10.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:09 smithi071 ceph-mon[27013]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-11T07:15:10.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:09 smithi071 ceph-mon[27013]: osdmap e34: 6 total, 5 up, 6 in 2024-09-11T07:15:10.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:09 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T07:15:10.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:09 smithi071 ceph-mon[27013]: from='osd.5 v1:172.21.15.196:6804/3113061101' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-11T07:15:10.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:09 smithi071 ceph-mon[27013]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-11T07:15:10.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:09 smithi071 ceph-mon[37727]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-11T07:15:10.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:09 smithi071 ceph-mon[37727]: osdmap e34: 6 total, 5 up, 6 in 2024-09-11T07:15:10.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:09 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T07:15:10.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:09 smithi071 ceph-mon[37727]: from='osd.5 v1:172.21.15.196:6804/3113061101' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-11T07:15:10.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:09 smithi071 ceph-mon[37727]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-11T07:15:10.724 INFO:teuthology.orchestra.run.smithi196.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-09-11T07:15:10.805 INFO:teuthology.orchestra.run.smithi196.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-09-11T07:15:10.821 INFO:teuthology.orchestra.run.smithi196.stderr: stderr: 10+0 records in 2024-09-11T07:15:10.821 INFO:teuthology.orchestra.run.smithi196.stderr:10+0 records out 2024-09-11T07:15:10.821 INFO:teuthology.orchestra.run.smithi196.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0137708 s, 761 MB/s 2024-09-11T07:15:10.821 INFO:teuthology.orchestra.run.smithi196.stderr:--> Zapping successful for: 2024-09-11T07:15:11.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:10 smithi196 ceph-mon[30773]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T07:15:11.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:10 smithi196 ceph-mon[30773]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-09-11T07:15:11.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:10 smithi196 ceph-mon[30773]: osdmap e35: 6 total, 5 up, 6 in 2024-09-11T07:15:11.193 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:10 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T07:15:11.194 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:10 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T07:15:11.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:10 smithi071 ceph-mon[27013]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T07:15:11.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:10 smithi071 ceph-mon[27013]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-09-11T07:15:11.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:10 smithi071 ceph-mon[27013]: osdmap e35: 6 total, 5 up, 6 in 2024-09-11T07:15:11.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:10 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T07:15:11.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:10 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T07:15:11.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:10 smithi071 ceph-mon[37727]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-11T07:15:11.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:10 smithi071 ceph-mon[37727]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-09-11T07:15:11.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:10 smithi071 ceph-mon[37727]: osdmap e35: 6 total, 5 up, 6 in 2024-09-11T07:15:11.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:10 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T07:15:11.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:10 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T07:15:11.403 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a81cffe6-700c-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi196:vg_nvme/lv_2 2024-09-11T07:15:12.207 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:11 smithi196 ceph-mon[30773]: purged_snaps scrub starts 2024-09-11T07:15:12.207 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:11 smithi196 ceph-mon[30773]: purged_snaps scrub ok 2024-09-11T07:15:12.207 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:11 smithi196 ceph-mon[30773]: osd.5 v1:172.21.15.196:6804/3113061101 boot 2024-09-11T07:15:12.207 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:11 smithi196 ceph-mon[30773]: osdmap e36: 6 total, 6 up, 6 in 2024-09-11T07:15:12.208 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:11 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T07:15:12.243 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:15:12.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:11 smithi071 ceph-mon[27013]: purged_snaps scrub starts 2024-09-11T07:15:12.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:11 smithi071 ceph-mon[27013]: purged_snaps scrub ok 2024-09-11T07:15:12.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:11 smithi071 ceph-mon[27013]: osd.5 v1:172.21.15.196:6804/3113061101 boot 2024-09-11T07:15:12.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:11 smithi071 ceph-mon[27013]: osdmap e36: 6 total, 6 up, 6 in 2024-09-11T07:15:12.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:11 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T07:15:12.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:11 smithi071 ceph-mon[37727]: purged_snaps scrub starts 2024-09-11T07:15:12.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:11 smithi071 ceph-mon[37727]: purged_snaps scrub ok 2024-09-11T07:15:12.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:11 smithi071 ceph-mon[37727]: osd.5 v1:172.21.15.196:6804/3113061101 boot 2024-09-11T07:15:12.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:11 smithi071 ceph-mon[37727]: osdmap e36: 6 total, 6 up, 6 in 2024-09-11T07:15:12.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:11 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T07:15:13.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:12 smithi196 ceph-mon[30773]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:15:13.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:12 smithi196 ceph-mon[30773]: osdmap e37: 6 total, 6 up, 6 in 2024-09-11T07:15:13.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:12 smithi071 ceph-mon[27013]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:15:13.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:12 smithi071 ceph-mon[27013]: osdmap e37: 6 total, 6 up, 6 in 2024-09-11T07:15:13.339 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:12 smithi071 ceph-mon[37727]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:15:13.339 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:12 smithi071 ceph-mon[37727]: osdmap e37: 6 total, 6 up, 6 in 2024-09-11T07:15:14.011 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:13 smithi196 ceph-mon[30773]: osdmap e38: 6 total, 6 up, 6 in 2024-09-11T07:15:14.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:13 smithi071 ceph-mon[27013]: osdmap e38: 6 total, 6 up, 6 in 2024-09-11T07:15:14.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:13 smithi071 ceph-mon[37727]: osdmap e38: 6 total, 6 up, 6 in 2024-09-11T07:15:15.015 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:14 smithi196 ceph-mon[30773]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:15:15.016 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:14 smithi196 ceph-mon[30773]: from='client.24286 v1:172.21.15.196:0/1145308233' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:15:15.016 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:14 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T07:15:15.016 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:14 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T07:15:15.016 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:14 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:15:15.016 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:14 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:15:15.016 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:14 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:15:15.016 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:14 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:15:15.016 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:14 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:15:15.016 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:14 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:15:15.016 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:14 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:15:15.017 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:14 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:15:15.017 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:14 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:15:15.017 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:14 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:15:15.017 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:14 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:15:15.017 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:14 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:15:15.017 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:14 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:15:15.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:14 smithi071 ceph-mon[27013]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:15:15.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:14 smithi071 ceph-mon[27013]: from='client.24286 v1:172.21.15.196:0/1145308233' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:15:15.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:14 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T07:15:15.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:14 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T07:15:15.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:14 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:15:15.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:14 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:15:15.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:14 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:15:15.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:14 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:15:15.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:14 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:15:15.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:14 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:15:15.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:14 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:15:15.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:14 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:15:15.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:14 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:15:15.341 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:14 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:15:15.341 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:14 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:15:15.341 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:14 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:15:15.341 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:14 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:15:15.341 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:14 smithi071 ceph-mon[37727]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:15:15.341 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:14 smithi071 ceph-mon[37727]: from='client.24286 v1:172.21.15.196:0/1145308233' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:15:15.341 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:14 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T07:15:15.341 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:14 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T07:15:15.341 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:14 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:15:15.341 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:14 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:15:15.341 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:14 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:15:15.342 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:14 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:15:15.342 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:14 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:15:15.342 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:14 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:15:15.342 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:14 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:15:15.342 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:14 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:15:15.342 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:14 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:15:15.342 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:14 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:15:15.342 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:14 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:15:15.342 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:14 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:15:15.342 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:14 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:15:16.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:15 smithi196 ceph-mon[30773]: Detected new or changed devices on smithi196 2024-09-11T07:15:16.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:15 smithi196 ceph-mon[30773]: Adjusting osd_memory_target on smithi196 to 8544M 2024-09-11T07:15:16.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:15 smithi071 ceph-mon[27013]: Detected new or changed devices on smithi196 2024-09-11T07:15:16.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:15 smithi071 ceph-mon[27013]: Adjusting osd_memory_target on smithi196 to 8544M 2024-09-11T07:15:16.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:15 smithi071 ceph-mon[37727]: Detected new or changed devices on smithi196 2024-09-11T07:15:16.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:15 smithi071 ceph-mon[37727]: Adjusting osd_memory_target on smithi196 to 8544M 2024-09-11T07:15:17.214 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:16 smithi196 ceph-mon[30773]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:15:17.214 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:16 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.196:0/2174006134' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "10eab0a3-9915-4c02-a147-44302ecfb9ef"}]: dispatch 2024-09-11T07:15:17.214 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:16 smithi196 ceph-mon[30773]: from='client.24292 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "10eab0a3-9915-4c02-a147-44302ecfb9ef"}]: dispatch 2024-09-11T07:15:17.214 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:16 smithi196 ceph-mon[30773]: from='client.24292 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "10eab0a3-9915-4c02-a147-44302ecfb9ef"}]': finished 2024-09-11T07:15:17.214 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:16 smithi196 ceph-mon[30773]: osdmap e39: 7 total, 6 up, 7 in 2024-09-11T07:15:17.214 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:16 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T07:15:17.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:16 smithi071 ceph-mon[27013]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:15:17.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:16 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.196:0/2174006134' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "10eab0a3-9915-4c02-a147-44302ecfb9ef"}]: dispatch 2024-09-11T07:15:17.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:16 smithi071 ceph-mon[27013]: from='client.24292 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "10eab0a3-9915-4c02-a147-44302ecfb9ef"}]: dispatch 2024-09-11T07:15:17.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:16 smithi071 ceph-mon[27013]: from='client.24292 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "10eab0a3-9915-4c02-a147-44302ecfb9ef"}]': finished 2024-09-11T07:15:17.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:16 smithi071 ceph-mon[27013]: osdmap e39: 7 total, 6 up, 7 in 2024-09-11T07:15:17.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:16 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T07:15:17.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:16 smithi071 ceph-mon[37727]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:15:17.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:16 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.196:0/2174006134' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "10eab0a3-9915-4c02-a147-44302ecfb9ef"}]: dispatch 2024-09-11T07:15:17.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:16 smithi071 ceph-mon[37727]: from='client.24292 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "10eab0a3-9915-4c02-a147-44302ecfb9ef"}]: dispatch 2024-09-11T07:15:17.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:16 smithi071 ceph-mon[37727]: from='client.24292 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "10eab0a3-9915-4c02-a147-44302ecfb9ef"}]': finished 2024-09-11T07:15:17.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:16 smithi071 ceph-mon[37727]: osdmap e39: 7 total, 6 up, 7 in 2024-09-11T07:15:17.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:16 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T07:15:18.284 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:17 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.196:0/1942036691' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T07:15:18.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:17 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.196:0/1942036691' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T07:15:18.339 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:17 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.196:0/1942036691' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T07:15:19.284 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:18 smithi196 ceph-mon[30773]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:15:19.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:18 smithi071 ceph-mon[27013]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:15:19.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:18 smithi071 ceph-mon[37727]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:15:21.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:20 smithi196 ceph-mon[30773]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:15:21.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:20 smithi071 ceph-mon[27013]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:15:21.339 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:20 smithi071 ceph-mon[37727]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:15:23.279 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:22 smithi196 ceph-mon[30773]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:15:23.279 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:22 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-11T07:15:23.280 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:22 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:15:23.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:22 smithi071 ceph-mon[27013]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:15:23.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:22 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-11T07:15:23.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:22 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:15:23.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:22 smithi071 ceph-mon[37727]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:15:23.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:22 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-11T07:15:23.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:22 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:15:24.284 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:23 smithi196 ceph-mon[30773]: Deploying daemon osd.6 on smithi196 2024-09-11T07:15:24.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:23 smithi071 ceph-mon[27013]: Deploying daemon osd.6 on smithi196 2024-09-11T07:15:24.339 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:23 smithi071 ceph-mon[37727]: Deploying daemon osd.6 on smithi196 2024-09-11T07:15:25.223 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:24 smithi196 ceph-mon[30773]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:15:25.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:24 smithi071 ceph-mon[27013]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:15:25.339 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:24 smithi071 ceph-mon[37727]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:15:27.207 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:26 smithi196 ceph-mon[30773]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:15:27.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:26 smithi071 ceph-mon[27013]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:15:27.339 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:26 smithi071 ceph-mon[37727]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:15:28.240 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:27 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:15:28.241 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:27 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:15:28.241 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:27 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:15:28.241 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:27 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:15:28.241 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:27 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:15:28.241 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:27 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:15:28.284 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:27 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:15:28.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:27 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:15:28.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:27 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:15:29.125 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:28 smithi196 ceph-mon[30773]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:15:29.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:28 smithi071 ceph-mon[27013]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:15:29.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:28 smithi071 ceph-mon[37727]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:15:30.570 INFO:teuthology.orchestra.run.smithi196.stdout:Created osd(s) 6 on host 'smithi196' 2024-09-11T07:15:31.231 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:30 smithi196 ceph-mon[30773]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:15:31.231 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:30 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:15:31.231 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:30 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:15:31.231 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:30 smithi196 ceph-mon[30773]: from='osd.6 v1:172.21.15.196:6808/307456429' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-11T07:15:31.231 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:30 smithi196 ceph-mon[30773]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-11T07:15:31.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:30 smithi071 ceph-mon[27013]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:15:31.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:30 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:15:31.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:30 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:15:31.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:30 smithi071 ceph-mon[27013]: from='osd.6 v1:172.21.15.196:6808/307456429' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-11T07:15:31.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:30 smithi071 ceph-mon[27013]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-11T07:15:31.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:30 smithi071 ceph-mon[37727]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:15:31.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:30 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:15:31.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:30 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:15:31.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:30 smithi071 ceph-mon[37727]: from='osd.6 v1:172.21.15.196:6808/307456429' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-11T07:15:31.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:30 smithi071 ceph-mon[37727]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-11T07:15:31.521 DEBUG:teuthology.orchestra.run.smithi196:osd.6> sudo journalctl -f -n 0 -u ceph-a81cffe6-700c-11ef-bcea-c7b262605968@osd.6.service 2024-09-11T07:15:31.523 INFO:tasks.cephadm:Deploying osd.7 on smithi196 with /dev/vg_nvme/lv_1... 2024-09-11T07:15:31.524 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a81cffe6-700c-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-09-11T07:15:31.831 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:15:32.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:32 smithi071 ceph-mon[27013]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:15:32.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:32 smithi071 ceph-mon[27013]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-11T07:15:32.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:32 smithi071 ceph-mon[27013]: osdmap e40: 7 total, 6 up, 7 in 2024-09-11T07:15:32.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:32 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T07:15:32.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:32 smithi071 ceph-mon[27013]: from='osd.6 v1:172.21.15.196:6808/307456429' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-11T07:15:32.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:32 smithi071 ceph-mon[27013]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-11T07:15:32.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:32 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:15:32.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:32 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:15:32.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:32 smithi071 ceph-mon[37727]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:15:32.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:32 smithi071 ceph-mon[37727]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-11T07:15:32.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:32 smithi071 ceph-mon[37727]: osdmap e40: 7 total, 6 up, 7 in 2024-09-11T07:15:32.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:32 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T07:15:32.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:32 smithi071 ceph-mon[37727]: from='osd.6 v1:172.21.15.196:6808/307456429' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-11T07:15:32.841 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:32 smithi071 ceph-mon[37727]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-11T07:15:32.841 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:32 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:15:32.841 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:32 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:15:32.929 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:32 smithi196 ceph-mon[30773]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:15:32.929 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:32 smithi196 ceph-mon[30773]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-11T07:15:32.929 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:32 smithi196 ceph-mon[30773]: osdmap e40: 7 total, 6 up, 7 in 2024-09-11T07:15:32.929 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:32 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T07:15:32.929 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:32 smithi196 ceph-mon[30773]: from='osd.6 v1:172.21.15.196:6808/307456429' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-11T07:15:32.929 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:32 smithi196 ceph-mon[30773]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-11T07:15:32.929 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:32 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:15:32.929 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:32 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:15:32.930 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 07:15:32 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-6[47249]: 2024-09-11T07:15:32.576+0000 7f4953431640 -1 osd.6 0 waiting for initial osdmap 2024-09-11T07:15:32.930 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 07:15:32 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-6[47249]: 2024-09-11T07:15:32.582+0000 7f494da67640 -1 osd.6 41 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T07:15:33.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:33 smithi196 ceph-mon[30773]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-09-11T07:15:33.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:33 smithi196 ceph-mon[30773]: osdmap e41: 7 total, 6 up, 7 in 2024-09-11T07:15:33.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:33 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T07:15:33.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:33 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T07:15:33.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:33 smithi196 ceph-mon[30773]: osd.6 v1:172.21.15.196:6808/307456429 boot 2024-09-11T07:15:33.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:33 smithi196 ceph-mon[30773]: osdmap e42: 7 total, 7 up, 7 in 2024-09-11T07:15:33.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:33 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T07:15:33.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:33 smithi071 ceph-mon[27013]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-09-11T07:15:33.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:33 smithi071 ceph-mon[27013]: osdmap e41: 7 total, 6 up, 7 in 2024-09-11T07:15:33.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:33 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T07:15:33.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:33 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T07:15:33.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:33 smithi071 ceph-mon[27013]: osd.6 v1:172.21.15.196:6808/307456429 boot 2024-09-11T07:15:33.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:33 smithi071 ceph-mon[27013]: osdmap e42: 7 total, 7 up, 7 in 2024-09-11T07:15:33.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:33 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T07:15:33.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:33 smithi071 ceph-mon[37727]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-09-11T07:15:33.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:33 smithi071 ceph-mon[37727]: osdmap e41: 7 total, 6 up, 7 in 2024-09-11T07:15:33.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:33 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T07:15:33.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:33 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T07:15:33.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:33 smithi071 ceph-mon[37727]: osd.6 v1:172.21.15.196:6808/307456429 boot 2024-09-11T07:15:33.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:33 smithi071 ceph-mon[37727]: osdmap e42: 7 total, 7 up, 7 in 2024-09-11T07:15:33.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:33 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T07:15:34.116 INFO:teuthology.orchestra.run.smithi196.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-09-11T07:15:34.214 INFO:teuthology.orchestra.run.smithi196.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-09-11T07:15:34.230 INFO:teuthology.orchestra.run.smithi196.stderr: stderr: 10+0 records in 2024-09-11T07:15:34.230 INFO:teuthology.orchestra.run.smithi196.stderr:10+0 records out 2024-09-11T07:15:34.230 INFO:teuthology.orchestra.run.smithi196.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0135918 s, 771 MB/s 2024-09-11T07:15:34.230 INFO:teuthology.orchestra.run.smithi196.stderr:--> Zapping successful for: 2024-09-11T07:15:34.689 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:34 smithi196 ceph-mon[30773]: purged_snaps scrub starts 2024-09-11T07:15:34.689 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:34 smithi196 ceph-mon[30773]: purged_snaps scrub ok 2024-09-11T07:15:34.689 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:34 smithi196 ceph-mon[30773]: pgmap v140: 1 pgs: 1 peering; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:15:34.689 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:34 smithi196 ceph-mon[30773]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-11T07:15:34.689 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:34 smithi196 ceph-mon[30773]: osdmap e43: 7 total, 7 up, 7 in 2024-09-11T07:15:34.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:34 smithi071 ceph-mon[27013]: purged_snaps scrub starts 2024-09-11T07:15:34.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:34 smithi071 ceph-mon[27013]: purged_snaps scrub ok 2024-09-11T07:15:34.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:34 smithi071 ceph-mon[27013]: pgmap v140: 1 pgs: 1 peering; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:15:34.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:34 smithi071 ceph-mon[27013]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-11T07:15:34.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:34 smithi071 ceph-mon[27013]: osdmap e43: 7 total, 7 up, 7 in 2024-09-11T07:15:34.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:34 smithi071 ceph-mon[37727]: purged_snaps scrub starts 2024-09-11T07:15:34.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:34 smithi071 ceph-mon[37727]: purged_snaps scrub ok 2024-09-11T07:15:34.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:34 smithi071 ceph-mon[37727]: pgmap v140: 1 pgs: 1 peering; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-11T07:15:34.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:34 smithi071 ceph-mon[37727]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-11T07:15:34.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:34 smithi071 ceph-mon[37727]: osdmap e43: 7 total, 7 up, 7 in 2024-09-11T07:15:35.486 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a81cffe6-700c-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi196:vg_nvme/lv_1 2024-09-11T07:15:36.199 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:15:36.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:36 smithi196 ceph-mon[30773]: osdmap e44: 7 total, 7 up, 7 in 2024-09-11T07:15:36.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:36 smithi196 ceph-mon[30773]: pgmap v143: 1 pgs: 1 peering; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T07:15:36.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:36 smithi196 ceph-mon[30773]: Detected new or changed devices on smithi196 2024-09-11T07:15:36.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:36 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:15:36.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:36 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:15:36.786 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:36 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:15:36.786 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:36 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:15:36.786 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:36 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:15:36.786 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:36 smithi196 ceph-mon[30773]: Adjusting osd_memory_target on smithi196 to 5696M 2024-09-11T07:15:36.786 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:36 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:15:36.786 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:36 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:15:36.786 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:36 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:15:36.786 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:36 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:15:36.786 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:36 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:15:36.786 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:36 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:15:36.786 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:36 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:15:36.786 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:36 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:15:36.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:36 smithi071 ceph-mon[27013]: osdmap e44: 7 total, 7 up, 7 in 2024-09-11T07:15:36.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:36 smithi071 ceph-mon[27013]: pgmap v143: 1 pgs: 1 peering; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T07:15:36.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:36 smithi071 ceph-mon[27013]: Detected new or changed devices on smithi196 2024-09-11T07:15:36.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:36 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:15:36.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:36 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:15:36.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:36 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:15:36.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:36 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:15:36.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:36 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:15:36.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:36 smithi071 ceph-mon[27013]: Adjusting osd_memory_target on smithi196 to 5696M 2024-09-11T07:15:36.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:36 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:15:36.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:36 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:15:36.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:36 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:15:36.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:36 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:15:36.841 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:36 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:15:36.841 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:36 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:15:36.841 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:36 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:15:36.841 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:36 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:15:36.841 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:36 smithi071 ceph-mon[37727]: osdmap e44: 7 total, 7 up, 7 in 2024-09-11T07:15:36.841 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:36 smithi071 ceph-mon[37727]: pgmap v143: 1 pgs: 1 peering; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T07:15:36.841 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:36 smithi071 ceph-mon[37727]: Detected new or changed devices on smithi196 2024-09-11T07:15:36.841 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:36 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:15:36.841 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:36 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:15:36.841 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:36 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:15:36.841 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:36 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:15:36.842 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:36 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:15:36.842 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:36 smithi071 ceph-mon[37727]: Adjusting osd_memory_target on smithi196 to 5696M 2024-09-11T07:15:36.842 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:36 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:15:36.842 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:36 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:15:36.842 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:36 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:15:36.842 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:36 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:15:36.842 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:36 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:15:36.842 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:36 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:15:36.842 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:36 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:15:36.842 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:36 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:15:39.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:38 smithi196 ceph-mon[30773]: pgmap v144: 1 pgs: 1 peering; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T07:15:39.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:38 smithi196 ceph-mon[30773]: from='client.24313 v1:172.21.15.196:0/1230889315' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:15:39.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:38 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T07:15:39.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:38 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T07:15:39.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:38 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:15:39.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:38 smithi071 ceph-mon[27013]: pgmap v144: 1 pgs: 1 peering; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T07:15:39.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:38 smithi071 ceph-mon[27013]: from='client.24313 v1:172.21.15.196:0/1230889315' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:15:39.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:38 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T07:15:39.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:38 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T07:15:39.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:38 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:15:39.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:38 smithi071 ceph-mon[37727]: pgmap v144: 1 pgs: 1 peering; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T07:15:39.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:38 smithi071 ceph-mon[37727]: from='client.24313 v1:172.21.15.196:0/1230889315' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi196:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:15:39.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:38 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-11T07:15:39.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:38 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-11T07:15:39.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:38 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:15:39.784 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:39 smithi196 ceph-mon[30773]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-11T07:15:39.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:39 smithi196 ceph-mon[30773]: Cluster is now healthy 2024-09-11T07:15:40.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:39 smithi071 ceph-mon[27013]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-11T07:15:40.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:39 smithi071 ceph-mon[27013]: Cluster is now healthy 2024-09-11T07:15:40.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:39 smithi071 ceph-mon[37727]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-11T07:15:40.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:39 smithi071 ceph-mon[37727]: Cluster is now healthy 2024-09-11T07:15:41.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:40 smithi196 ceph-mon[30773]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-09-11T07:15:41.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:40 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.196:0/365653141' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fb019207-8474-43d5-b431-7995248abf6e"}]: dispatch 2024-09-11T07:15:41.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:40 smithi196 ceph-mon[30773]: from='client.24319 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fb019207-8474-43d5-b431-7995248abf6e"}]: dispatch 2024-09-11T07:15:41.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:40 smithi196 ceph-mon[30773]: from='client.24319 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fb019207-8474-43d5-b431-7995248abf6e"}]': finished 2024-09-11T07:15:41.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:40 smithi196 ceph-mon[30773]: osdmap e45: 8 total, 7 up, 8 in 2024-09-11T07:15:41.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:40 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T07:15:41.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:40 smithi071 ceph-mon[27013]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-09-11T07:15:41.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:40 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.196:0/365653141' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fb019207-8474-43d5-b431-7995248abf6e"}]: dispatch 2024-09-11T07:15:41.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:40 smithi071 ceph-mon[27013]: from='client.24319 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fb019207-8474-43d5-b431-7995248abf6e"}]: dispatch 2024-09-11T07:15:41.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:40 smithi071 ceph-mon[27013]: from='client.24319 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fb019207-8474-43d5-b431-7995248abf6e"}]': finished 2024-09-11T07:15:41.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:40 smithi071 ceph-mon[27013]: osdmap e45: 8 total, 7 up, 8 in 2024-09-11T07:15:41.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:40 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T07:15:41.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:40 smithi071 ceph-mon[37727]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-09-11T07:15:41.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:40 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.196:0/365653141' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fb019207-8474-43d5-b431-7995248abf6e"}]: dispatch 2024-09-11T07:15:41.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:40 smithi071 ceph-mon[37727]: from='client.24319 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fb019207-8474-43d5-b431-7995248abf6e"}]: dispatch 2024-09-11T07:15:41.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:40 smithi071 ceph-mon[37727]: from='client.24319 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fb019207-8474-43d5-b431-7995248abf6e"}]': finished 2024-09-11T07:15:41.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:40 smithi071 ceph-mon[37727]: osdmap e45: 8 total, 7 up, 8 in 2024-09-11T07:15:41.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:40 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T07:15:42.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:41 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.196:0/924855587' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T07:15:42.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:41 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.196:0/924855587' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T07:15:42.089 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:41 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.196:0/924855587' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-11T07:15:42.946 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:42 smithi196 ceph-mon[30773]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-09-11T07:15:43.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:42 smithi071 ceph-mon[27013]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-09-11T07:15:43.089 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:42 smithi071 ceph-mon[37727]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-09-11T07:15:44.771 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:44 smithi196 ceph-mon[30773]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-09-11T07:15:45.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:44 smithi071 ceph-mon[27013]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-09-11T07:15:45.089 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:44 smithi071 ceph-mon[37727]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-09-11T07:15:46.757 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:46 smithi196 ceph-mon[30773]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-11T07:15:46.757 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:46 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-11T07:15:46.757 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:46 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:15:46.757 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:46 smithi196 ceph-mon[30773]: Deploying daemon osd.7 on smithi196 2024-09-11T07:15:47.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:46 smithi071 ceph-mon[27013]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-11T07:15:47.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:46 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-11T07:15:47.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:46 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:15:47.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:46 smithi071 ceph-mon[27013]: Deploying daemon osd.7 on smithi196 2024-09-11T07:15:47.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:46 smithi071 ceph-mon[37727]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-11T07:15:47.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:46 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-11T07:15:47.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:46 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:15:47.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:46 smithi071 ceph-mon[37727]: Deploying daemon osd.7 on smithi196 2024-09-11T07:15:48.735 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:48 smithi196 ceph-mon[30773]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-11T07:15:49.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:48 smithi071 ceph-mon[27013]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-11T07:15:49.089 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:48 smithi071 ceph-mon[37727]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-11T07:15:51.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:50 smithi196 ceph-mon[30773]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T07:15:51.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:50 smithi071 ceph-mon[27013]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T07:15:51.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:50 smithi071 ceph-mon[37727]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T07:15:51.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:51 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:15:51.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:51 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:15:51.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:51 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:15:52.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:51 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:15:52.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:51 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:15:52.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:51 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:15:52.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:51 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:15:52.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:51 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:15:52.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:51 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:15:53.162 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:52 smithi196 ceph-mon[30773]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T07:15:53.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:52 smithi071 ceph-mon[27013]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T07:15:53.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:52 smithi071 ceph-mon[37727]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T07:15:54.326 INFO:teuthology.orchestra.run.smithi196.stdout:Created osd(s) 7 on host 'smithi196' 2024-09-11T07:15:54.938 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:54 smithi196 ceph-mon[30773]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T07:15:54.938 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:54 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:15:54.938 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:54 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:15:54.938 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:54 smithi196 ceph-mon[30773]: from='osd.7 v1:172.21.15.196:6812/4134490927' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-11T07:15:54.938 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:54 smithi196 ceph-mon[30773]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-11T07:15:54.939 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:54 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:15:54.939 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:54 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:15:54.963 DEBUG:teuthology.orchestra.run.smithi196:osd.7> sudo journalctl -f -n 0 -u ceph-a81cffe6-700c-11ef-bcea-c7b262605968@osd.7.service 2024-09-11T07:15:54.965 INFO:tasks.cephadm:Waiting for 8 OSDs to come up... 2024-09-11T07:15:54.965 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a81cffe6-700c-11ef-bcea-c7b262605968 -- ceph osd stat -f json 2024-09-11T07:15:54.993 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:54 smithi071 ceph-mon[27013]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T07:15:54.993 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:54 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:15:54.993 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:54 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:15:54.994 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:54 smithi071 ceph-mon[27013]: from='osd.7 v1:172.21.15.196:6812/4134490927' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-11T07:15:54.994 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:54 smithi071 ceph-mon[27013]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-11T07:15:54.994 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:54 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:15:54.994 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:54 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:15:54.994 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:54 smithi071 ceph-mon[37727]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T07:15:54.994 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:54 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:15:54.994 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:54 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:15:54.994 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:54 smithi071 ceph-mon[37727]: from='osd.7 v1:172.21.15.196:6812/4134490927' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-11T07:15:54.994 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:54 smithi071 ceph-mon[37727]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-11T07:15:54.994 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:54 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:15:54.994 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:54 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:15:55.223 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:15:56.089 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:55 smithi071 ceph-mon[37727]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-11T07:15:56.089 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:55 smithi071 ceph-mon[37727]: osdmap e46: 8 total, 7 up, 8 in 2024-09-11T07:15:56.089 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:55 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T07:15:56.089 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:55 smithi071 ceph-mon[37727]: from='osd.7 v1:172.21.15.196:6812/4134490927' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-11T07:15:56.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:55 smithi071 ceph-mon[37727]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-11T07:15:56.090 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:55 smithi071 ceph-mon[27013]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-11T07:15:56.090 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:55 smithi071 ceph-mon[27013]: osdmap e46: 8 total, 7 up, 8 in 2024-09-11T07:15:56.090 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:55 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T07:15:56.090 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:55 smithi071 ceph-mon[27013]: from='osd.7 v1:172.21.15.196:6812/4134490927' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-11T07:15:56.090 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:55 smithi071 ceph-mon[27013]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-11T07:15:56.169 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:55 smithi196 ceph-mon[30773]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-11T07:15:56.169 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:55 smithi196 ceph-mon[30773]: osdmap e46: 8 total, 7 up, 8 in 2024-09-11T07:15:56.169 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:55 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T07:15:56.169 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:55 smithi196 ceph-mon[30773]: from='osd.7 v1:172.21.15.196:6812/4134490927' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-11T07:15:56.169 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:55 smithi196 ceph-mon[30773]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]: dispatch 2024-09-11T07:15:56.169 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 07:15:55 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-7[53063]: 2024-09-11T07:15:55.874+0000 7f2929bcd640 -1 osd.7 0 waiting for initial osdmap 2024-09-11T07:15:56.169 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 07:15:55 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-7[53063]: 2024-09-11T07:15:55.881+0000 7f2924a04640 -1 osd.7 47 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T07:15:56.931 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T07:15:57.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:56 smithi196 ceph-mon[30773]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T07:15:57.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:56 smithi196 ceph-mon[30773]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-09-11T07:15:57.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:56 smithi196 ceph-mon[30773]: osdmap e47: 8 total, 7 up, 8 in 2024-09-11T07:15:57.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:56 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T07:15:57.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:56 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T07:15:57.339 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:56 smithi071 ceph-mon[37727]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T07:15:57.339 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:56 smithi071 ceph-mon[37727]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-09-11T07:15:57.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:56 smithi071 ceph-mon[37727]: osdmap e47: 8 total, 7 up, 8 in 2024-09-11T07:15:57.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:56 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T07:15:57.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:56 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T07:15:57.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:56 smithi071 ceph-mon[27013]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-11T07:15:57.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:56 smithi071 ceph-mon[27013]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi196", "root=default"]}]': finished 2024-09-11T07:15:57.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:56 smithi071 ceph-mon[27013]: osdmap e47: 8 total, 7 up, 8 in 2024-09-11T07:15:57.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:56 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T07:15:57.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:56 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T07:15:57.663 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":48,"num_osds":8,"num_up_osds":8,"osd_up_since":1726038956,"num_in_osds":8,"osd_in_since":1726038940,"num_remapped_pgs":1} 2024-09-11T07:15:57.664 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid a81cffe6-700c-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T07:15:57.932 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:57 smithi071 ceph-mon[27013]: purged_snaps scrub starts 2024-09-11T07:15:57.932 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:57 smithi071 ceph-mon[27013]: purged_snaps scrub ok 2024-09-11T07:15:57.932 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:57 smithi071 ceph-mon[27013]: osd.7 v1:172.21.15.196:6812/4134490927 boot 2024-09-11T07:15:57.932 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:57 smithi071 ceph-mon[27013]: osdmap e48: 8 total, 8 up, 8 in 2024-09-11T07:15:57.932 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:57 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T07:15:57.932 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:57 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/1460929517' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-11T07:15:57.932 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:57 smithi071 ceph-mon[37727]: purged_snaps scrub starts 2024-09-11T07:15:57.933 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:57 smithi071 ceph-mon[37727]: purged_snaps scrub ok 2024-09-11T07:15:57.933 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:57 smithi071 ceph-mon[37727]: osd.7 v1:172.21.15.196:6812/4134490927 boot 2024-09-11T07:15:57.933 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:57 smithi071 ceph-mon[37727]: osdmap e48: 8 total, 8 up, 8 in 2024-09-11T07:15:57.933 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:57 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T07:15:57.933 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:57 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/1460929517' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-11T07:15:57.954 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:15:58.010 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:57 smithi196 ceph-mon[30773]: purged_snaps scrub starts 2024-09-11T07:15:58.010 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:57 smithi196 ceph-mon[30773]: purged_snaps scrub ok 2024-09-11T07:15:58.010 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:57 smithi196 ceph-mon[30773]: osd.7 v1:172.21.15.196:6812/4134490927 boot 2024-09-11T07:15:58.010 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:57 smithi196 ceph-mon[30773]: osdmap e48: 8 total, 8 up, 8 in 2024-09-11T07:15:58.010 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:57 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T07:15:58.010 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:57 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/1460929517' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-11T07:15:59.036 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:58 smithi071 ceph-mon[27013]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:15:59.036 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:58 smithi071 ceph-mon[27013]: osdmap e49: 8 total, 8 up, 8 in 2024-09-11T07:15:59.036 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:58 smithi071 ceph-mon[27013]: Detected new or changed devices on smithi196 2024-09-11T07:15:59.036 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:58 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:15:59.036 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:58 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:15:59.036 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:58 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:15:59.036 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:58 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:15:59.036 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:58 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:15:59.036 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:58 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:15:59.036 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:58 smithi071 ceph-mon[27013]: Adjusting osd_memory_target on smithi196 to 4272M 2024-09-11T07:15:59.036 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:58 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:15:59.037 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:58 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:15:59.037 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:58 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:15:59.037 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:58 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:15:59.037 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:58 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:15:59.037 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:58 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:15:59.037 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:58 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:15:59.037 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:58 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:15:59.037 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:58 smithi071 ceph-mon[27013]: osdmap e50: 8 total, 8 up, 8 in 2024-09-11T07:15:59.037 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:58 smithi071 ceph-mon[37727]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:15:59.037 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:58 smithi071 ceph-mon[37727]: osdmap e49: 8 total, 8 up, 8 in 2024-09-11T07:15:59.037 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:58 smithi071 ceph-mon[37727]: Detected new or changed devices on smithi196 2024-09-11T07:15:59.038 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:58 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:15:59.038 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:58 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:15:59.038 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:58 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:15:59.038 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:58 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:15:59.038 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:58 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:15:59.038 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:58 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:15:59.038 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:58 smithi071 ceph-mon[37727]: Adjusting osd_memory_target on smithi196 to 4272M 2024-09-11T07:15:59.038 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:58 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:15:59.038 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:58 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:15:59.038 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:58 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:15:59.038 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:58 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:15:59.038 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:58 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:15:59.039 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:58 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:15:59.039 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:58 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:15:59.039 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:58 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:15:59.039 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:58 smithi071 ceph-mon[37727]: osdmap e50: 8 total, 8 up, 8 in 2024-09-11T07:15:59.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:58 smithi196 ceph-mon[30773]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:15:59.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:58 smithi196 ceph-mon[30773]: osdmap e49: 8 total, 8 up, 8 in 2024-09-11T07:15:59.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:58 smithi196 ceph-mon[30773]: Detected new or changed devices on smithi196 2024-09-11T07:15:59.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:58 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:15:59.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:58 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:15:59.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:58 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:15:59.286 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:58 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:15:59.286 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:58 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:15:59.286 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:58 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:15:59.286 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:58 smithi196 ceph-mon[30773]: Adjusting osd_memory_target on smithi196 to 4272M 2024-09-11T07:15:59.286 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:58 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:15:59.286 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:58 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:15:59.286 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:58 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:15:59.286 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:58 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:15:59.286 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:58 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:15:59.286 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:58 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:15:59.286 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:58 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:15:59.286 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:58 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:15:59.287 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:58 smithi196 ceph-mon[30773]: osdmap e50: 8 total, 8 up, 8 in 2024-09-11T07:15:59.643 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T07:15:59.643 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":50,"fsid":"a81cffe6-700c-11ef-bcea-c7b262605968","created":"2024-09-11T07:11:08.733125+0000","modified":"2024-09-11T07:15:58.872087+0000","last_up_change":"2024-09-11T07:15:56.869561+0000","last_in_change":"2024-09-11T07:15:40.145796+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T07:14:03.821210+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"10eebb2a-001d-4106-b37b-3af068980575","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.71:6801","nonce":88530034}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.71:6802","nonce":88530034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.71:6804","nonce":88530034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.71:6803","nonce":88530034}]},"public_addr":"172.21.15.71:6801/88530034","cluster_addr":"172.21.15.71:6802/88530034","heartbeat_back_addr":"172.21.15.71:6804/88530034","heartbeat_front_addr":"172.21.15.71:6803/88530034","state":["exists","up"]},{"osd":1,"uuid":"c46d1c53-e519-4ac0-b557-91ee98f659fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.71:6805","nonce":2000038894}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.71:6806","nonce":2000038894}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.71:6808","nonce":2000038894}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.71:6807","nonce":2000038894}]},"public_addr":"172.21.15.71:6805/2000038894","cluster_addr":"172.21.15.71:6806/2000038894","heartbeat_back_addr":"172.21.15.71:6808/2000038894","heartbeat_front_addr":"172.21.15.71:6807/2000038894","state":["exists","up"]},{"osd":2,"uuid":"a6a12661-7514-4887-9acf-a313f7055e30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.71:6809","nonce":3105396883}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.71:6810","nonce":3105396883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.71:6812","nonce":3105396883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.71:6811","nonce":3105396883}]},"public_addr":"172.21.15.71:6809/3105396883","cluster_addr":"172.21.15.71:6810/3105396883","heartbeat_back_addr":"172.21.15.71:6812/3105396883","heartbeat_front_addr":"172.21.15.71:6811/3105396883","state":["exists","up"]},{"osd":3,"uuid":"a66db725-6221-44e6-aa78-3bb92957d455","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.71:6813","nonce":295183903}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.71:6814","nonce":295183903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.71:6816","nonce":295183903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.71:6815","nonce":295183903}]},"public_addr":"172.21.15.71:6813/295183903","cluster_addr":"172.21.15.71:6814/295183903","heartbeat_back_addr":"172.21.15.71:6816/295183903","heartbeat_front_addr":"172.21.15.71:6815/295183903","state":["exists","up"]},{"osd":4,"uuid":"1ac47bc6-7006-4b9f-afe3-676e6f12cee8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.196:6800","nonce":4211374096}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.196:6801","nonce":4211374096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.196:6803","nonce":4211374096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.196:6802","nonce":4211374096}]},"public_addr":"172.21.15.196:6800/4211374096","cluster_addr":"172.21.15.196:6801/4211374096","heartbeat_back_addr":"172.21.15.196:6803/4211374096","heartbeat_front_addr":"172.21.15.196:6802/4211374096","state":["exists","up"]},{"osd":5,"uuid":"683fad79-fdac-4c84-aec8-6d8d3d4ec586","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.196:6804","nonce":3113061101}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.196:6805","nonce":3113061101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.196:6807","nonce":3113061101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.196:6806","nonce":3113061101}]},"public_addr":"172.21.15.196:6804/3113061101","cluster_addr":"172.21.15.196:6805/3113061101","heartbeat_back_addr":"172.21.15.196:6807/3113061101","heartbeat_front_addr":"172.21.15.196:6806/3113061101","state":["exists","up"]},{"osd":6,"uuid":"10eab0a3-9915-4c02-a147-44302ecfb9ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.196:6808","nonce":307456429}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.196:6809","nonce":307456429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.196:6811","nonce":307456429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.196:6810","nonce":307456429}]},"public_addr":"172.21.15.196:6808/307456429","cluster_addr":"172.21.15.196:6809/307456429","heartbeat_back_addr":"172.21.15.196:6811/307456429","heartbeat_front_addr":"172.21.15.196:6810/307456429","state":["exists","up"]},{"osd":7,"uuid":"fb019207-8474-43d5-b431-7995248abf6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.196:6812","nonce":4134490927}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.196:6813","nonce":4134490927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.196:6815","nonce":4134490927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.196:6814","nonce":4134490927}]},"public_addr":"172.21.15.196:6812/4134490927","cluster_addr":"172.21.15.196:6813/4134490927","heartbeat_back_addr":"172.21.15.196:6815/4134490927","heartbeat_front_addr":"172.21.15.196:6814/4134490927","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T07:13:15.011171+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T07:13:38.198270+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T07:14:01.640980+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T07:14:24.678399+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T07:14:45.748090+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T07:15:08.952057+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T07:15:31.816720+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T07:15:55.021672+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.71:0/3954363371":"2024-09-12T07:11:57.528996+0000","172.21.15.71:0/1739611680":"2024-09-12T07:11:57.528996+0000","172.21.15.71:6800/3365647004":"2024-09-12T07:11:29.071185+0000","172.21.15.71:0/384458095":"2024-09-12T07:11:29.071185+0000","172.21.15.71:0/3185720173":"2024-09-12T07:11:29.071185+0000","172.21.15.71:6800/137476262":"2024-09-12T07:11:57.528996+0000","172.21.15.71:0/1493047649":"2024-09-12T07:11:57.528996+0000","172.21.15.71:0/3624555087":"2024-09-12T07:11:29.071185+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T07:15:59.917 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:15:59 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/4109662215' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T07:15:59.917 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:15:59 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/4109662215' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T07:16:00.218 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-11T07:14:03.821210+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2024-09-11T07:16:00.218 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid a81cffe6-700c-11ef-bcea-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-09-11T07:16:00.284 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:15:59 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/4109662215' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T07:16:00.459 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:16:01.148 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:00 smithi071 ceph-mon[27013]: pgmap v161: 1 pgs: 1 remapped+peering; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:16:01.149 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:00 smithi071 ceph-mon[27013]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-11T07:16:01.149 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:00 smithi071 ceph-mon[37727]: pgmap v161: 1 pgs: 1 remapped+peering; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:16:01.149 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:00 smithi071 ceph-mon[37727]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-11T07:16:01.284 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:00 smithi196 ceph-mon[30773]: pgmap v161: 1 pgs: 1 remapped+peering; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:16:01.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:00 smithi196 ceph-mon[30773]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-11T07:16:02.168 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 1 2024-09-11T07:16:02.855 INFO:tasks.cephadm:Adding ceph.rgw.foo.a on smithi071 2024-09-11T07:16:02.855 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a81cffe6-700c-11ef-bcea-c7b262605968 -- ceph orch apply rgw foo.a --placement '1;smithi071=foo.a' 2024-09-11T07:16:03.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:02 smithi071 ceph-mon[27013]: pgmap v162: 1 pgs: 1 remapped+peering; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:16:03.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:02 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/3785321012' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-11T07:16:03.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:02 smithi071 ceph-mon[37727]: pgmap v162: 1 pgs: 1 remapped+peering; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:16:03.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:02 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/3785321012' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-11T07:16:03.112 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:16:03.227 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:02 smithi196 ceph-mon[30773]: pgmap v162: 1 pgs: 1 remapped+peering; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:16:03.227 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:02 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/3785321012' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-11T07:16:04.230 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:03 smithi196 ceph-mon[30773]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-11T07:16:04.230 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:03 smithi196 ceph-mon[30773]: Cluster is now healthy 2024-09-11T07:16:04.339 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:03 smithi071 ceph-mon[37727]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-11T07:16:04.339 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:03 smithi071 ceph-mon[37727]: Cluster is now healthy 2024-09-11T07:16:04.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:03 smithi071 ceph-mon[27013]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-11T07:16:04.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:03 smithi071 ceph-mon[27013]: Cluster is now healthy 2024-09-11T07:16:04.944 INFO:teuthology.orchestra.run.smithi196.stdout:Scheduled rgw.foo.a update... 2024-09-11T07:16:05.265 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:04 smithi196 ceph-mon[30773]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:16:05.265 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:04 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:16:05.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:04 smithi071 ceph-mon[27013]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:16:05.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:04 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:16:05.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:04 smithi071 ceph-mon[37727]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:16:05.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:04 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:16:05.563 DEBUG:teuthology.orchestra.run.smithi071:rgw.foo.a> sudo journalctl -f -n 0 -u ceph-a81cffe6-700c-11ef-bcea-c7b262605968@rgw.foo.a.service 2024-09-11T07:16:05.565 INFO:tasks.cephadm:Adding ceph.iscsi.iscsi.a on smithi196 2024-09-11T07:16:05.565 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a81cffe6-700c-11ef-bcea-c7b262605968 -- ceph osd pool create datapool 3 3 replicated 2024-09-11T07:16:05.812 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:16:06.143 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:05 smithi196 ceph-mon[30773]: from='client.24358 v1:172.21.15.196:0/3774048670' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi071=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:16:06.143 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:05 smithi196 ceph-mon[30773]: Saving service rgw.foo.a spec with placement smithi071=foo.a;count:1 2024-09-11T07:16:06.143 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:05 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:16:06.143 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:05 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:16:06.143 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:05 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:16:06.143 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:05 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:16:06.143 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:05 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-09-11T07:16:06.143 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:05 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-09-11T07:16:06.144 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:05 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:16:06.144 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:05 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:16:06.144 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:05 smithi196 ceph-mon[30773]: Deploying daemon rgw.foo.a on smithi071 2024-09-11T07:16:06.262 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:05 smithi071 ceph-mon[27013]: from='client.24358 v1:172.21.15.196:0/3774048670' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi071=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:16:06.262 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:05 smithi071 ceph-mon[27013]: Saving service rgw.foo.a spec with placement smithi071=foo.a;count:1 2024-09-11T07:16:06.262 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:05 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:16:06.262 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:05 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:16:06.262 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:05 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:16:06.262 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:05 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:16:06.264 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:05 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-09-11T07:16:06.264 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:05 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-09-11T07:16:06.264 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:05 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:16:06.264 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:05 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:16:06.264 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:05 smithi071 ceph-mon[27013]: Deploying daemon rgw.foo.a on smithi071 2024-09-11T07:16:06.264 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:05 smithi071 ceph-mon[37727]: from='client.24358 v1:172.21.15.196:0/3774048670' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi071=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:16:06.264 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:05 smithi071 ceph-mon[37727]: Saving service rgw.foo.a spec with placement smithi071=foo.a;count:1 2024-09-11T07:16:06.264 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:05 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:16:06.264 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:05 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:16:06.265 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:05 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:16:06.265 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:05 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:16:06.265 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:05 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-09-11T07:16:06.265 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:05 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-09-11T07:16:06.265 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:05 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:16:06.265 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:05 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:16:06.265 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:05 smithi071 ceph-mon[37727]: Deploying daemon rgw.foo.a on smithi071 2024-09-11T07:16:06.962 INFO:journalctl@ceph.rgw.foo.a.smithi071.stdout:Sep 11 07:16:06 smithi071 podman[63929]: 2024-09-11 07:16:06.886819747 +0000 UTC m=+0.132958311 container create 1b92fd942af38b2ed3c429aea193881422c090d4a0ce02cc1c1699759e10ea6d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-rgw-foo-a, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T07:16:07.034 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:06 smithi196 ceph-mon[30773]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:16:07.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:06 smithi071 ceph-mon[27013]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:16:07.339 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:06 smithi071 ceph-mon[37727]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:16:07.340 INFO:journalctl@ceph.rgw.foo.a.smithi071.stdout:Sep 11 07:16:07 smithi071 podman[63929]: 2024-09-11 07:16:07.122788778 +0000 UTC m=+0.368927307 container init 1b92fd942af38b2ed3c429aea193881422c090d4a0ce02cc1c1699759e10ea6d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-rgw-foo-a, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, io.buildah.version=1.37.2) 2024-09-11T07:16:07.340 INFO:journalctl@ceph.rgw.foo.a.smithi071.stdout:Sep 11 07:16:07 smithi071 podman[63929]: 2024-09-11 07:16:07.12556619 +0000 UTC m=+0.371704706 container start 1b92fd942af38b2ed3c429aea193881422c090d4a0ce02cc1c1699759e10ea6d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-rgw-foo-a, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T07:16:07.340 INFO:journalctl@ceph.rgw.foo.a.smithi071.stdout:Sep 11 07:16:07 smithi071 bash[63929]: 1b92fd942af38b2ed3c429aea193881422c090d4a0ce02cc1c1699759e10ea6d 2024-09-11T07:16:07.340 INFO:journalctl@ceph.rgw.foo.a.smithi071.stdout:Sep 11 07:16:07 smithi071 systemd[1]: Started Ceph rgw.foo.a for a81cffe6-700c-11ef-bcea-c7b262605968. 2024-09-11T07:16:07.969 INFO:teuthology.orchestra.run.smithi196.stderr:pool 'datapool' created 2024-09-11T07:16:08.579 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:08 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:16:08.579 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:08 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:16:08.579 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:08 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:16:08.579 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:08 smithi196 ceph-mon[30773]: Saving service rgw.foo.a spec with placement smithi071=foo.a;count:1 2024-09-11T07:16:08.579 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:08 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:16:08.579 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:08 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:16:08.579 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:08 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.196:0/469824447' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-11T07:16:08.579 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:08 smithi196 ceph-mon[30773]: from='client.24373 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-11T07:16:08.579 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:08 smithi196 ceph-mon[30773]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-11T07:16:08.579 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:08 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:16:08.580 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:08 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:16:08.580 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:08 smithi196 ceph-mon[30773]: from='client.24373 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-09-11T07:16:08.580 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:08 smithi196 ceph-mon[30773]: osdmap e51: 8 total, 8 up, 8 in 2024-09-11T07:16:08.580 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:08 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/1225071826' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-11T07:16:08.580 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:08 smithi196 ceph-mon[30773]: from='client.34349 ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-11T07:16:08.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:08 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:16:08.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:08 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:16:08.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:08 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:16:08.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:08 smithi071 ceph-mon[27013]: Saving service rgw.foo.a spec with placement smithi071=foo.a;count:1 2024-09-11T07:16:08.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:08 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:16:08.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:08 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:16:08.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:08 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.196:0/469824447' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-11T07:16:08.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:08 smithi071 ceph-mon[27013]: from='client.24373 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-11T07:16:08.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:08 smithi071 ceph-mon[27013]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-11T07:16:08.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:08 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:16:08.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:08 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:16:08.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:08 smithi071 ceph-mon[27013]: from='client.24373 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-09-11T07:16:08.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:08 smithi071 ceph-mon[27013]: osdmap e51: 8 total, 8 up, 8 in 2024-09-11T07:16:08.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:08 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/1225071826' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-11T07:16:08.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:08 smithi071 ceph-mon[27013]: from='client.34349 ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-11T07:16:08.591 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:08 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:16:08.591 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:08 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:16:08.591 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:08 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:16:08.591 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:08 smithi071 ceph-mon[37727]: Saving service rgw.foo.a spec with placement smithi071=foo.a;count:1 2024-09-11T07:16:08.591 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:08 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:16:08.591 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:08 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:16:08.591 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:08 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.196:0/469824447' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-11T07:16:08.591 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:08 smithi071 ceph-mon[37727]: from='client.24373 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-11T07:16:08.591 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:08 smithi071 ceph-mon[37727]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-11T07:16:08.591 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:08 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:16:08.591 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:08 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:16:08.592 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:08 smithi071 ceph-mon[37727]: from='client.24373 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-09-11T07:16:08.592 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:08 smithi071 ceph-mon[37727]: osdmap e51: 8 total, 8 up, 8 in 2024-09-11T07:16:08.592 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:08 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/1225071826' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-11T07:16:08.592 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:08 smithi071 ceph-mon[37727]: from='client.34349 ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-11T07:16:08.606 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a81cffe6-700c-11ef-bcea-c7b262605968 -- rbd pool init datapool 2024-09-11T07:16:08.852 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:16:10.235 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:09 smithi196 ceph-mon[30773]: from='client.34349 ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-09-11T07:16:10.236 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:09 smithi196 ceph-mon[30773]: osdmap e52: 8 total, 8 up, 8 in 2024-09-11T07:16:10.236 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:09 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:16:10.236 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:09 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:16:10.236 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:09 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:16:10.236 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:09 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:16:10.339 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:09 smithi071 ceph-mon[37727]: from='client.34349 ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-09-11T07:16:10.339 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:09 smithi071 ceph-mon[37727]: osdmap e52: 8 total, 8 up, 8 in 2024-09-11T07:16:10.339 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:09 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:16:10.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:09 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:16:10.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:09 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:16:10.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:09 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:16:10.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:09 smithi071 ceph-mon[27013]: from='client.34349 ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-09-11T07:16:10.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:09 smithi071 ceph-mon[27013]: osdmap e52: 8 total, 8 up, 8 in 2024-09-11T07:16:10.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:09 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:16:10.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:09 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:16:10.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:09 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:16:10.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:09 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:16:11.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:10 smithi196 ceph-mon[30773]: Checking dashboard <-> RGW credentials 2024-09-11T07:16:11.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:10 smithi196 ceph-mon[30773]: pgmap v168: 36 pgs: 5 creating+peering, 26 unknown, 5 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-11T07:16:11.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:10 smithi196 ceph-mon[30773]: osdmap e53: 8 total, 8 up, 8 in 2024-09-11T07:16:11.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:10 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/3048411829' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-11T07:16:11.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:10 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/1225071826' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-11T07:16:11.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:10 smithi196 ceph-mon[30773]: from='client.34349 ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-11T07:16:11.286 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:10 smithi196 ceph-mon[30773]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T07:16:11.286 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:10 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.196:0/2670713266' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-11T07:16:11.286 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:10 smithi196 ceph-mon[30773]: from='client.24400 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-11T07:16:11.339 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:10 smithi071 ceph-mon[37727]: Checking dashboard <-> RGW credentials 2024-09-11T07:16:11.339 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:10 smithi071 ceph-mon[37727]: pgmap v168: 36 pgs: 5 creating+peering, 26 unknown, 5 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-11T07:16:11.339 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:10 smithi071 ceph-mon[37727]: osdmap e53: 8 total, 8 up, 8 in 2024-09-11T07:16:11.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:10 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/3048411829' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-11T07:16:11.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:10 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/1225071826' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-11T07:16:11.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:10 smithi071 ceph-mon[37727]: from='client.34349 ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-11T07:16:11.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:10 smithi071 ceph-mon[37727]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T07:16:11.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:10 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.196:0/2670713266' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-11T07:16:11.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:10 smithi071 ceph-mon[37727]: from='client.24400 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-11T07:16:11.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:10 smithi071 ceph-mon[27013]: Checking dashboard <-> RGW credentials 2024-09-11T07:16:11.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:10 smithi071 ceph-mon[27013]: pgmap v168: 36 pgs: 5 creating+peering, 26 unknown, 5 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-11T07:16:11.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:10 smithi071 ceph-mon[27013]: osdmap e53: 8 total, 8 up, 8 in 2024-09-11T07:16:11.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:10 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/3048411829' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-11T07:16:11.341 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:10 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/1225071826' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-11T07:16:11.341 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:10 smithi071 ceph-mon[27013]: from='client.34349 ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-11T07:16:11.341 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:10 smithi071 ceph-mon[27013]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T07:16:11.341 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:10 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.196:0/2670713266' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-11T07:16:11.341 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:10 smithi071 ceph-mon[27013]: from='client.24400 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-11T07:16:12.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:11 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/3048411829' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-11T07:16:12.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:11 smithi196 ceph-mon[30773]: from='client.34349 ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-11T07:16:12.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:11 smithi196 ceph-mon[30773]: from='client.24400 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-09-11T07:16:12.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:11 smithi196 ceph-mon[30773]: osdmap e54: 8 total, 8 up, 8 in 2024-09-11T07:16:12.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:11 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/3048411829' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-11T07:16:12.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:11 smithi071 ceph-mon[27013]: from='client.34349 ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-11T07:16:12.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:11 smithi071 ceph-mon[27013]: from='client.24400 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-09-11T07:16:12.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:11 smithi071 ceph-mon[27013]: osdmap e54: 8 total, 8 up, 8 in 2024-09-11T07:16:12.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:11 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/3048411829' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-11T07:16:12.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:11 smithi071 ceph-mon[37727]: from='client.34349 ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-11T07:16:12.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:11 smithi071 ceph-mon[37727]: from='client.24400 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-09-11T07:16:12.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:11 smithi071 ceph-mon[37727]: osdmap e54: 8 total, 8 up, 8 in 2024-09-11T07:16:13.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:13 smithi196 ceph-mon[30773]: pgmap v171: 68 pgs: 12 creating+peering, 39 unknown, 17 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 255 B/s wr, 1 op/s 2024-09-11T07:16:13.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:13 smithi196 ceph-mon[30773]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-11T07:16:13.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:13 smithi196 ceph-mon[30773]: Cluster is now healthy 2024-09-11T07:16:13.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:13 smithi196 ceph-mon[30773]: osdmap e55: 8 total, 8 up, 8 in 2024-09-11T07:16:13.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:13 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/1225071826' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-11T07:16:13.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:13 smithi196 ceph-mon[30773]: from='client.34349 ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-11T07:16:13.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:13 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/3048411829' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-11T07:16:13.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:13 smithi071 ceph-mon[27013]: pgmap v171: 68 pgs: 12 creating+peering, 39 unknown, 17 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 255 B/s wr, 1 op/s 2024-09-11T07:16:13.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:13 smithi071 ceph-mon[27013]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-11T07:16:13.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:13 smithi071 ceph-mon[27013]: Cluster is now healthy 2024-09-11T07:16:13.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:13 smithi071 ceph-mon[27013]: osdmap e55: 8 total, 8 up, 8 in 2024-09-11T07:16:13.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:13 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/1225071826' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-11T07:16:13.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:13 smithi071 ceph-mon[27013]: from='client.34349 ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-11T07:16:13.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:13 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/3048411829' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-11T07:16:13.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:13 smithi071 ceph-mon[37727]: pgmap v171: 68 pgs: 12 creating+peering, 39 unknown, 17 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 255 B/s wr, 1 op/s 2024-09-11T07:16:13.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:13 smithi071 ceph-mon[37727]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-11T07:16:13.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:13 smithi071 ceph-mon[37727]: Cluster is now healthy 2024-09-11T07:16:13.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:13 smithi071 ceph-mon[37727]: osdmap e55: 8 total, 8 up, 8 in 2024-09-11T07:16:13.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:13 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/1225071826' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-11T07:16:13.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:13 smithi071 ceph-mon[37727]: from='client.34349 ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-11T07:16:13.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:13 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/3048411829' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-11T07:16:13.597 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a81cffe6-700c-11ef-bcea-c7b262605968 -- ceph orch apply iscsi datapool admin admin --trusted_ip_list 172.21.15.196 --placement '1;smithi196=iscsi.a' 2024-09-11T07:16:13.844 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:16:14.284 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:14 smithi196 ceph-mon[30773]: from='client.34349 ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-11T07:16:14.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:14 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/3048411829' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-11T07:16:14.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:14 smithi196 ceph-mon[30773]: osdmap e56: 8 total, 8 up, 8 in 2024-09-11T07:16:14.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:14 smithi071 ceph-mon[27013]: from='client.34349 ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-11T07:16:14.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:14 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/3048411829' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-11T07:16:14.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:14 smithi071 ceph-mon[27013]: osdmap e56: 8 total, 8 up, 8 in 2024-09-11T07:16:14.339 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:14 smithi071 ceph-mon[37727]: from='client.34349 ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-11T07:16:14.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:14 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/3048411829' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-11T07:16:14.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:14 smithi071 ceph-mon[37727]: osdmap e56: 8 total, 8 up, 8 in 2024-09-11T07:16:15.262 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:15 smithi196 ceph-mon[30773]: pgmap v174: 100 pgs: 23 creating+peering, 25 unknown, 52 active+clean; 578 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 5.0 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-09-11T07:16:15.262 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:15 smithi196 ceph-mon[30773]: osdmap e57: 8 total, 8 up, 8 in 2024-09-11T07:16:15.262 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:15 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/1997245885' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-11T07:16:15.262 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:15 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/3013933509' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-11T07:16:15.263 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:15 smithi196 ceph-mon[30773]: from='client.34394 ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-11T07:16:15.263 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:15 smithi196 ceph-mon[30773]: from='client.34388 ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-11T07:16:15.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:15 smithi071 ceph-mon[27013]: pgmap v174: 100 pgs: 23 creating+peering, 25 unknown, 52 active+clean; 578 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 5.0 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-09-11T07:16:15.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:15 smithi071 ceph-mon[27013]: osdmap e57: 8 total, 8 up, 8 in 2024-09-11T07:16:15.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:15 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/1997245885' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-11T07:16:15.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:15 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/3013933509' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-11T07:16:15.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:15 smithi071 ceph-mon[27013]: from='client.34394 ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-11T07:16:15.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:15 smithi071 ceph-mon[27013]: from='client.34388 ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-11T07:16:15.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:15 smithi071 ceph-mon[37727]: pgmap v174: 100 pgs: 23 creating+peering, 25 unknown, 52 active+clean; 578 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 5.0 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-09-11T07:16:15.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:15 smithi071 ceph-mon[37727]: osdmap e57: 8 total, 8 up, 8 in 2024-09-11T07:16:15.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:15 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/1997245885' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-11T07:16:15.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:15 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/3013933509' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-11T07:16:15.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:15 smithi071 ceph-mon[37727]: from='client.34394 ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-11T07:16:15.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:15 smithi071 ceph-mon[37727]: from='client.34388 ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-11T07:16:15.660 INFO:teuthology.orchestra.run.smithi196.stdout:Scheduled iscsi.datapool update... 2024-09-11T07:16:16.256 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:16 smithi196 ceph-mon[30773]: from='client.34394 ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-11T07:16:16.256 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:16 smithi196 ceph-mon[30773]: from='client.34388 ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-11T07:16:16.256 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:16 smithi196 ceph-mon[30773]: osdmap e58: 8 total, 8 up, 8 in 2024-09-11T07:16:16.256 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:16 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/1997245885' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-11T07:16:16.256 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:16 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/3013933509' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-11T07:16:16.256 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:16 smithi196 ceph-mon[30773]: from='client.34394 ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-11T07:16:16.256 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:16 smithi196 ceph-mon[30773]: from='client.34388 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-11T07:16:16.256 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:16 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:16:16.285 INFO:tasks.cephadm:Distributing iscsi-gateway.cfg... 2024-09-11T07:16:16.285 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-11T07:16:16.285 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/etc/ceph/iscsi-gateway.cfg 2024-09-11T07:16:16.311 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:16 smithi071 ceph-mon[27013]: from='client.34394 ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-11T07:16:16.311 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:16 smithi071 ceph-mon[27013]: from='client.34388 ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-11T07:16:16.311 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:16 smithi071 ceph-mon[27013]: osdmap e58: 8 total, 8 up, 8 in 2024-09-11T07:16:16.311 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:16 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/1997245885' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-11T07:16:16.311 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:16 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/3013933509' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-11T07:16:16.311 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:16 smithi071 ceph-mon[27013]: from='client.34394 ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-11T07:16:16.311 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:16 smithi071 ceph-mon[27013]: from='client.34388 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-11T07:16:16.312 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:16 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:16:16.312 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:16 smithi071 ceph-mon[37727]: from='client.34394 ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-11T07:16:16.312 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:16 smithi071 ceph-mon[37727]: from='client.34388 ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-11T07:16:16.312 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:16 smithi071 ceph-mon[37727]: osdmap e58: 8 total, 8 up, 8 in 2024-09-11T07:16:16.312 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:16 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/1997245885' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-11T07:16:16.312 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:16 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/3013933509' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-11T07:16:16.312 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:16 smithi071 ceph-mon[37727]: from='client.34394 ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-11T07:16:16.312 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:16 smithi071 ceph-mon[37727]: from='client.34388 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-11T07:16:16.312 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:16 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:16:16.316 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-11T07:16:16.316 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/etc/ceph/iscsi-gateway.cfg 2024-09-11T07:16:16.345 DEBUG:teuthology.orchestra.run.smithi196:iscsi.iscsi.a> sudo journalctl -f -n 0 -u ceph-a81cffe6-700c-11ef-bcea-c7b262605968@iscsi.iscsi.a.service 2024-09-11T07:16:16.388 INFO:tasks.cephadm:Adding prometheus.a on smithi196 2024-09-11T07:16:16.388 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a81cffe6-700c-11ef-bcea-c7b262605968 -- ceph orch apply prometheus '1;smithi196=a' 2024-09-11T07:16:17.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:17 smithi196 ceph-mon[30773]: pgmap v177: 132 pgs: 20 creating+peering, 43 unknown, 69 active+clean; 578 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-09-11T07:16:17.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:17 smithi196 ceph-mon[30773]: from='client.24418 v1:172.21.15.196:0/1013741166' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.196", "placement": "1;smithi196=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:16:17.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:17 smithi196 ceph-mon[30773]: Saving service iscsi.datapool spec with placement smithi196=iscsi.a;count:1 2024-09-11T07:16:17.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:17 smithi196 ceph-mon[30773]: from='client.34394 ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-11T07:16:17.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:17 smithi196 ceph-mon[30773]: from='client.34388 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-11T07:16:17.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:17 smithi196 ceph-mon[30773]: osdmap e59: 8 total, 8 up, 8 in 2024-09-11T07:16:17.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:17 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:16:17.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:17 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:16:17.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:17 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:16:17.286 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:17 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-09-11T07:16:17.286 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:17 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-09-11T07:16:17.286 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:17 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:16:17.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:17 smithi071 ceph-mon[27013]: pgmap v177: 132 pgs: 20 creating+peering, 43 unknown, 69 active+clean; 578 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-09-11T07:16:17.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:17 smithi071 ceph-mon[27013]: from='client.24418 v1:172.21.15.196:0/1013741166' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.196", "placement": "1;smithi196=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:16:17.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:17 smithi071 ceph-mon[27013]: Saving service iscsi.datapool spec with placement smithi196=iscsi.a;count:1 2024-09-11T07:16:17.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:17 smithi071 ceph-mon[27013]: from='client.34394 ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-11T07:16:17.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:17 smithi071 ceph-mon[27013]: from='client.34388 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-11T07:16:17.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:17 smithi071 ceph-mon[27013]: osdmap e59: 8 total, 8 up, 8 in 2024-09-11T07:16:17.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:17 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:16:17.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:17 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:16:17.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:17 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:16:17.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:17 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-09-11T07:16:17.341 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:17 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-09-11T07:16:17.341 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:17 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:16:17.341 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:17 smithi071 ceph-mon[37727]: pgmap v177: 132 pgs: 20 creating+peering, 43 unknown, 69 active+clean; 578 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-09-11T07:16:17.341 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:17 smithi071 ceph-mon[37727]: from='client.24418 v1:172.21.15.196:0/1013741166' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.196", "placement": "1;smithi196=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:16:17.341 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:17 smithi071 ceph-mon[37727]: Saving service iscsi.datapool spec with placement smithi196=iscsi.a;count:1 2024-09-11T07:16:17.341 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:17 smithi071 ceph-mon[37727]: from='client.34394 ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-11T07:16:17.341 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:17 smithi071 ceph-mon[37727]: from='client.34388 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-11T07:16:17.341 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:17 smithi071 ceph-mon[37727]: osdmap e59: 8 total, 8 up, 8 in 2024-09-11T07:16:17.341 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:17 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:16:17.342 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:17 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:16:17.342 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:17 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:16:17.342 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:17 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-09-11T07:16:17.342 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:17 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-09-11T07:16:17.342 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:17 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:16:17.723 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:16:18.190 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:18 smithi196 ceph-mon[30773]: Deploying daemon iscsi.iscsi.a on smithi196 2024-09-11T07:16:18.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:18 smithi071 ceph-mon[27013]: Deploying daemon iscsi.iscsi.a on smithi196 2024-09-11T07:16:18.339 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:18 smithi071 ceph-mon[37727]: Deploying daemon iscsi.iscsi.a on smithi196 2024-09-11T07:16:19.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:19 smithi196 ceph-mon[30773]: pgmap v179: 132 pgs: 9 creating+peering, 10 unknown, 113 active+clean; 579 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 21 KiB/s rd, 2.4 KiB/s wr, 51 op/s 2024-09-11T07:16:19.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:19 smithi071 ceph-mon[27013]: pgmap v179: 132 pgs: 9 creating+peering, 10 unknown, 113 active+clean; 579 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 21 KiB/s rd, 2.4 KiB/s wr, 51 op/s 2024-09-11T07:16:19.339 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:19 smithi071 ceph-mon[37727]: pgmap v179: 132 pgs: 9 creating+peering, 10 unknown, 113 active+clean; 579 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 21 KiB/s rd, 2.4 KiB/s wr, 51 op/s 2024-09-11T07:16:19.363 INFO:teuthology.orchestra.run.smithi196.stdout:Scheduled prometheus update... 2024-09-11T07:16:20.338 DEBUG:teuthology.orchestra.run.smithi196:prometheus.a> sudo journalctl -f -n 0 -u ceph-a81cffe6-700c-11ef-bcea-c7b262605968@prometheus.a.service 2024-09-11T07:16:20.341 INFO:tasks.cephadm:Adding node-exporter.a on smithi071 2024-09-11T07:16:20.341 INFO:tasks.cephadm:Adding node-exporter.b on smithi196 2024-09-11T07:16:20.341 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a81cffe6-700c-11ef-bcea-c7b262605968 -- ceph orch apply node-exporter '2;smithi071=a;smithi196=b' 2024-09-11T07:16:20.650 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:20 smithi196 ceph-mon[30773]: from='client.24439 v1:172.21.15.196:0/3006154774' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi196=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:16:20.651 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:20 smithi196 ceph-mon[30773]: Saving service prometheus spec with placement smithi196=a;count:1 2024-09-11T07:16:20.651 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:20 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:16:20.651 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:20 smithi196 ceph-mon[30773]: pgmap v180: 132 pgs: 132 active+clean; 581 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 58 KiB/s rd, 4.5 KiB/s wr, 141 op/s 2024-09-11T07:16:20.839 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:20 smithi071 ceph-mon[37727]: from='client.24439 v1:172.21.15.196:0/3006154774' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi196=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:16:20.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:20 smithi071 ceph-mon[37727]: Saving service prometheus spec with placement smithi196=a;count:1 2024-09-11T07:16:20.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:20 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:16:20.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:20 smithi071 ceph-mon[37727]: pgmap v180: 132 pgs: 132 active+clean; 581 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 58 KiB/s rd, 4.5 KiB/s wr, 141 op/s 2024-09-11T07:16:20.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:20 smithi071 ceph-mon[27013]: from='client.24439 v1:172.21.15.196:0/3006154774' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi196=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:16:20.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:20 smithi071 ceph-mon[27013]: Saving service prometheus spec with placement smithi196=a;count:1 2024-09-11T07:16:20.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:20 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:16:20.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:20 smithi071 ceph-mon[27013]: pgmap v180: 132 pgs: 132 active+clean; 581 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 58 KiB/s rd, 4.5 KiB/s wr, 141 op/s 2024-09-11T07:16:20.863 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:16:22.159 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:21 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:16:22.159 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:21 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:16:22.159 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:21 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:16:22.159 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:21 smithi196 ceph-mon[30773]: Checking pool "datapool" exists for service iscsi.datapool 2024-09-11T07:16:22.159 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:21 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:16:22.159 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:21 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:16:22.160 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:21 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.196:0/3236712401' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-11T07:16:22.339 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:21 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:16:22.339 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:21 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:16:22.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:21 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:16:22.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:21 smithi071 ceph-mon[37727]: Checking pool "datapool" exists for service iscsi.datapool 2024-09-11T07:16:22.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:21 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:16:22.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:21 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:16:22.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:21 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.196:0/3236712401' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-11T07:16:22.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:21 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:16:22.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:21 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:16:22.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:21 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:16:22.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:21 smithi071 ceph-mon[27013]: Checking pool "datapool" exists for service iscsi.datapool 2024-09-11T07:16:22.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:21 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:16:22.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:21 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:16:22.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:21 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.196:0/3236712401' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-11T07:16:22.673 INFO:teuthology.orchestra.run.smithi196.stdout:Scheduled node-exporter update... 2024-09-11T07:16:23.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:22 smithi196 ceph-mon[30773]: pgmap v181: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 61 KiB/s rd, 4.8 KiB/s wr, 148 op/s 2024-09-11T07:16:23.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:22 smithi196 ceph-mon[30773]: mgrmap e16: y(active, since 4m), standbys: x 2024-09-11T07:16:23.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:22 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:16:23.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:22 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:16:23.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:22 smithi071 ceph-mon[27013]: pgmap v181: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 61 KiB/s rd, 4.8 KiB/s wr, 148 op/s 2024-09-11T07:16:23.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:22 smithi071 ceph-mon[27013]: mgrmap e16: y(active, since 4m), standbys: x 2024-09-11T07:16:23.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:22 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:16:23.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:22 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:16:23.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:22 smithi071 ceph-mon[37727]: pgmap v181: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 61 KiB/s rd, 4.8 KiB/s wr, 148 op/s 2024-09-11T07:16:23.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:22 smithi071 ceph-mon[37727]: mgrmap e16: y(active, since 4m), standbys: x 2024-09-11T07:16:23.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:22 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:16:23.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:22 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:16:23.416 DEBUG:teuthology.orchestra.run.smithi071:node-exporter.a> sudo journalctl -f -n 0 -u ceph-a81cffe6-700c-11ef-bcea-c7b262605968@node-exporter.a.service 2024-09-11T07:16:23.418 DEBUG:teuthology.orchestra.run.smithi196:node-exporter.b> sudo journalctl -f -n 0 -u ceph-a81cffe6-700c-11ef-bcea-c7b262605968@node-exporter.b.service 2024-09-11T07:16:23.421 INFO:tasks.cephadm:Adding alertmanager.a on smithi071 2024-09-11T07:16:23.421 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a81cffe6-700c-11ef-bcea-c7b262605968 -- ceph orch apply alertmanager '1;smithi071=a' 2024-09-11T07:16:23.748 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:16:24.146 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:23 smithi196 ceph-mon[30773]: from='client.24475 v1:172.21.15.196:0/3009021280' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi071=a;smithi196=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:16:24.146 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:23 smithi196 ceph-mon[30773]: Saving service node-exporter spec with placement smithi071=a;smithi196=b;count:2 2024-09-11T07:16:24.146 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:23 smithi196 ceph-mon[30773]: osdmap e60: 8 total, 8 up, 8 in 2024-09-11T07:16:24.339 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:23 smithi071 ceph-mon[37727]: from='client.24475 v1:172.21.15.196:0/3009021280' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi071=a;smithi196=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:16:24.339 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:23 smithi071 ceph-mon[37727]: Saving service node-exporter spec with placement smithi071=a;smithi196=b;count:2 2024-09-11T07:16:24.339 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:23 smithi071 ceph-mon[37727]: osdmap e60: 8 total, 8 up, 8 in 2024-09-11T07:16:24.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:23 smithi071 ceph-mon[27013]: from='client.24475 v1:172.21.15.196:0/3009021280' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi071=a;smithi196=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:16:24.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:23 smithi071 ceph-mon[27013]: Saving service node-exporter spec with placement smithi071=a;smithi196=b;count:2 2024-09-11T07:16:24.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:23 smithi071 ceph-mon[27013]: osdmap e60: 8 total, 8 up, 8 in 2024-09-11T07:16:25.093 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:24 smithi196 ceph-mon[30773]: pgmap v183: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 58 KiB/s rd, 4.5 KiB/s wr, 139 op/s 2024-09-11T07:16:25.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:24 smithi071 ceph-mon[27013]: pgmap v183: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 58 KiB/s rd, 4.5 KiB/s wr, 139 op/s 2024-09-11T07:16:25.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:24 smithi071 ceph-mon[37727]: pgmap v183: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 58 KiB/s rd, 4.5 KiB/s wr, 139 op/s 2024-09-11T07:16:25.515 INFO:teuthology.orchestra.run.smithi196.stdout:Scheduled alertmanager update... 2024-09-11T07:16:26.600 DEBUG:teuthology.orchestra.run.smithi071:alertmanager.a> sudo journalctl -f -n 0 -u ceph-a81cffe6-700c-11ef-bcea-c7b262605968@alertmanager.a.service 2024-09-11T07:16:26.603 INFO:tasks.cephadm:Adding grafana.a on smithi196 2024-09-11T07:16:26.604 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a81cffe6-700c-11ef-bcea-c7b262605968 -- ceph orch apply grafana '1;smithi196=a' 2024-09-11T07:16:26.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:26 smithi196 ceph-mon[30773]: from='client.24481 v1:172.21.15.196:0/1327922230' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi071=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:16:26.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:26 smithi196 ceph-mon[30773]: Saving service alertmanager spec with placement smithi071=a;count:1 2024-09-11T07:16:26.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:26 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:16:26.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:26 smithi196 ceph-mon[30773]: pgmap v184: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 48 KiB/s rd, 3.7 KiB/s wr, 116 op/s 2024-09-11T07:16:26.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:26 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:16:26.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:26 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:16:26.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:26 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:16:26.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:26 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:16:26.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:26 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:16:26.839 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:26 smithi071 ceph-mon[37727]: from='client.24481 v1:172.21.15.196:0/1327922230' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi071=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:16:26.839 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:26 smithi071 ceph-mon[37727]: Saving service alertmanager spec with placement smithi071=a;count:1 2024-09-11T07:16:26.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:26 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:16:26.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:26 smithi071 ceph-mon[37727]: pgmap v184: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 48 KiB/s rd, 3.7 KiB/s wr, 116 op/s 2024-09-11T07:16:26.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:26 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:16:26.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:26 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:16:26.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:26 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:16:26.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:26 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:16:26.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:26 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:16:26.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:26 smithi071 ceph-mon[27013]: from='client.24481 v1:172.21.15.196:0/1327922230' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi071=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:16:26.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:26 smithi071 ceph-mon[27013]: Saving service alertmanager spec with placement smithi071=a;count:1 2024-09-11T07:16:26.841 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:26 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:16:26.841 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:26 smithi071 ceph-mon[27013]: pgmap v184: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 48 KiB/s rd, 3.7 KiB/s wr, 116 op/s 2024-09-11T07:16:26.841 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:26 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:16:26.841 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:26 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:16:26.841 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:26 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:16:26.841 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:26 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:16:26.841 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:26 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:16:26.854 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:16:27.761 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:27 smithi196 ceph-mon[30773]: Deploying daemon prometheus.a on smithi196 2024-09-11T07:16:27.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:27 smithi071 ceph-mon[27013]: Deploying daemon prometheus.a on smithi196 2024-09-11T07:16:27.839 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:27 smithi071 ceph-mon[37727]: Deploying daemon prometheus.a on smithi196 2024-09-11T07:16:28.919 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:28 smithi196 ceph-mon[30773]: pgmap v185: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 37 KiB/s rd, 2.7 KiB/s wr, 90 op/s 2024-09-11T07:16:28.920 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:28 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:16:29.003 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:28 smithi071 ceph-mon[27013]: pgmap v185: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 37 KiB/s rd, 2.7 KiB/s wr, 90 op/s 2024-09-11T07:16:29.003 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:28 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:16:29.003 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:28 smithi071 ceph-mon[37727]: pgmap v185: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 37 KiB/s rd, 2.7 KiB/s wr, 90 op/s 2024-09-11T07:16:29.003 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:28 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:16:29.352 INFO:teuthology.orchestra.run.smithi196.stdout:Scheduled grafana update... 2024-09-11T07:16:30.313 DEBUG:teuthology.orchestra.run.smithi196:grafana.a> sudo journalctl -f -n 0 -u ceph-a81cffe6-700c-11ef-bcea-c7b262605968@grafana.a.service 2024-09-11T07:16:30.315 INFO:tasks.cephadm:Setting up client nodes... 2024-09-11T07:16:30.316 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a81cffe6-700c-11ef-bcea-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-11T07:16:30.558 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:16:30.678 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:30 smithi071 ceph-mon[27013]: from='client.24487 v1:172.21.15.196:0/4193230475' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi196=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:16:30.678 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:30 smithi071 ceph-mon[27013]: Saving service grafana spec with placement smithi196=a;count:1 2024-09-11T07:16:30.678 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:30 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:16:30.678 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:30 smithi071 ceph-mon[27013]: pgmap v186: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 12 KiB/s rd, 1.1 KiB/s wr, 29 op/s 2024-09-11T07:16:30.678 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:30 smithi071 ceph-mon[37727]: from='client.24487 v1:172.21.15.196:0/4193230475' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi196=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:16:30.679 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:30 smithi071 ceph-mon[37727]: Saving service grafana spec with placement smithi196=a;count:1 2024-09-11T07:16:30.679 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:30 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:16:30.679 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:30 smithi071 ceph-mon[37727]: pgmap v186: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 12 KiB/s rd, 1.1 KiB/s wr, 29 op/s 2024-09-11T07:16:30.784 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:30 smithi196 ceph-mon[30773]: from='client.24487 v1:172.21.15.196:0/4193230475' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi196=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-11T07:16:30.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:30 smithi196 ceph-mon[30773]: Saving service grafana spec with placement smithi196=a;count:1 2024-09-11T07:16:30.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:30 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:16:30.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:30 smithi196 ceph-mon[30773]: pgmap v186: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 12 KiB/s rd, 1.1 KiB/s wr, 29 op/s 2024-09-11T07:16:31.355 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 11 07:16:31 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-iscsi-iscsi-a[58197]: debug there is no tcmu-runner data available 2024-09-11T07:16:31.719 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:31 smithi071 ceph-mon[27013]: from='client.34442 v1:172.21.15.196:0/1952921461' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:16:31.719 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:31 smithi071 ceph-mon[37727]: from='client.34442 v1:172.21.15.196:0/1952921461' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:16:31.784 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:31 smithi196 ceph-mon[30773]: from='client.34442 v1:172.21.15.196:0/1952921461' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:16:32.515 INFO:teuthology.orchestra.run.smithi071.stdout:[client.0] 2024-09-11T07:16:32.515 INFO:teuthology.orchestra.run.smithi071.stdout: key = AQDQQ+FmvZdUHhAAfjtsHBTJY6L87jHQOvG/bg== 2024-09-11T07:16:32.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:32 smithi196 ceph-mon[30773]: pgmap v187: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 102 B/s wr, 1 op/s 2024-09-11T07:16:32.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:32 smithi071 ceph-mon[27013]: pgmap v187: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 102 B/s wr, 1 op/s 2024-09-11T07:16:32.839 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:32 smithi071 ceph-mon[37727]: pgmap v187: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 102 B/s wr, 1 op/s 2024-09-11T07:16:33.202 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-11T07:16:33.202 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-11T07:16:33.202 DEBUG:teuthology.orchestra.run.smithi071:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-11T07:16:33.239 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a81cffe6-700c-11ef-bcea-c7b262605968 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-11T07:16:33.534 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:33 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/3676031484' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-11T07:16:33.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:33 smithi196 ceph-mon[30773]: from='client.34475 ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-11T07:16:33.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:33 smithi196 ceph-mon[30773]: from='client.34475 ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-11T07:16:33.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:33 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/3676031484' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-11T07:16:33.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:33 smithi071 ceph-mon[27013]: from='client.34475 ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-11T07:16:33.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:33 smithi071 ceph-mon[27013]: from='client.34475 ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-11T07:16:33.589 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:33 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/3676031484' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-11T07:16:33.589 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:33 smithi071 ceph-mon[37727]: from='client.34475 ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-11T07:16:33.589 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:33 smithi071 ceph-mon[37727]: from='client.34475 ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-11T07:16:34.676 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:34 smithi196 ceph-mon[30773]: pgmap v188: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1020 B/s rd, 102 B/s wr, 1 op/s 2024-09-11T07:16:34.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:34 smithi071 ceph-mon[27013]: pgmap v188: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1020 B/s rd, 102 B/s wr, 1 op/s 2024-09-11T07:16:34.839 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:34 smithi071 ceph-mon[37727]: pgmap v188: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1020 B/s rd, 102 B/s wr, 1 op/s 2024-09-11T07:16:35.946 INFO:teuthology.orchestra.run.smithi196.stderr:Inferring config /var/lib/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/mon.b/config 2024-09-11T07:16:37.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:36 smithi196 ceph-mon[30773]: pgmap v189: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 85 B/s wr, 1 op/s 2024-09-11T07:16:37.036 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 07:16:36 smithi196 systemd[1]: Starting Ceph prometheus.a for a81cffe6-700c-11ef-bcea-c7b262605968... 2024-09-11T07:16:37.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:36 smithi071 ceph-mon[27013]: pgmap v189: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 85 B/s wr, 1 op/s 2024-09-11T07:16:37.089 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:36 smithi071 ceph-mon[37727]: pgmap v189: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 85 B/s wr, 1 op/s 2024-09-11T07:16:37.840 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 07:16:37 smithi196 podman[59698]: 2024-09-11 07:16:37.476774314 +0000 UTC m=+0.017970581 image pull a07b618ecd1dce142bce4c52f0e80982eaf1f14265a2415c2d35978ccaa0a464 quay.io/prometheus/prometheus:v2.43.0 2024-09-11T07:16:37.840 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 07:16:37 smithi196 podman[59698]: 2024-09-11 07:16:37.585442702 +0000 UTC m=+0.126638967 container create effc2eee58c985d81e3191836b25652719637c072b8a662273c7cc5a3f66627e (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-11T07:16:37.840 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 07:16:37 smithi196 podman[59698]: 2024-09-11 07:16:37.838296462 +0000 UTC m=+0.379492746 container init effc2eee58c985d81e3191836b25652719637c072b8a662273c7cc5a3f66627e (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-11T07:16:38.157 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 07:16:37 smithi196 podman[59698]: 2024-09-11 07:16:37.841828781 +0000 UTC m=+0.383025036 container start effc2eee58c985d81e3191836b25652719637c072b8a662273c7cc5a3f66627e (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-11T07:16:38.158 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 07:16:37 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-prometheus-a[59726]: ts=2024-09-11T07:16:37.876Z caller=main.go:564 level=info msg="Starting Prometheus Server" mode=server version="(version=2.43.0, branch=HEAD, revision=edfc3bcd025dd6fe296c167a14a216cab1e552ee)" 2024-09-11T07:16:38.158 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 07:16:37 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-prometheus-a[59726]: ts=2024-09-11T07:16:37.876Z caller=main.go:569 level=info build_context="(go=go1.19.7, platform=linux/amd64, user=root@8a0ee342e522, date=20230321-12:56:07, tags=netgo,builtinassets)" 2024-09-11T07:16:38.158 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 07:16:37 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-prometheus-a[59726]: ts=2024-09-11T07:16:37.876Z caller=main.go:570 level=info host_details="(Linux 5.14.0-505.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Thu Sep 5 07:54:07 UTC 2024 x86_64 smithi196 (none))" 2024-09-11T07:16:38.158 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 07:16:37 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-prometheus-a[59726]: ts=2024-09-11T07:16:37.876Z caller=main.go:571 level=info fd_limits="(soft=1048576, hard=1048576)" 2024-09-11T07:16:38.158 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 07:16:37 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-prometheus-a[59726]: ts=2024-09-11T07:16:37.876Z caller=main.go:572 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2024-09-11T07:16:38.158 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 07:16:37 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-prometheus-a[59726]: ts=2024-09-11T07:16:37.878Z caller=web.go:561 level=info component=web msg="Start listening for connections" address=:9095 2024-09-11T07:16:38.158 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 07:16:37 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-prometheus-a[59726]: ts=2024-09-11T07:16:37.878Z caller=main.go:1005 level=info msg="Starting TSDB ..." 2024-09-11T07:16:38.158 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 07:16:37 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-prometheus-a[59726]: ts=2024-09-11T07:16:37.880Z caller=tls_config.go:232 level=info component=web msg="Listening on" address=[::]:9095 2024-09-11T07:16:38.158 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 07:16:37 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-prometheus-a[59726]: ts=2024-09-11T07:16:37.880Z caller=tls_config.go:235 level=info component=web msg="TLS is disabled." http2=false address=[::]:9095 2024-09-11T07:16:38.158 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 07:16:37 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-prometheus-a[59726]: ts=2024-09-11T07:16:37.883Z caller=head.go:587 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2024-09-11T07:16:38.158 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 07:16:37 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-prometheus-a[59726]: ts=2024-09-11T07:16:37.883Z caller=head.go:658 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=2.577µs 2024-09-11T07:16:38.158 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 07:16:37 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-prometheus-a[59726]: ts=2024-09-11T07:16:37.883Z caller=head.go:664 level=info component=tsdb msg="Replaying WAL, this may take a while" 2024-09-11T07:16:38.159 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 07:16:37 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-prometheus-a[59726]: ts=2024-09-11T07:16:37.883Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=0 2024-09-11T07:16:38.159 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 07:16:37 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-prometheus-a[59726]: ts=2024-09-11T07:16:37.883Z caller=head.go:772 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=30.842µs wal_replay_duration=255.817µs wbl_replay_duration=135ns total_replay_duration=318.25µs 2024-09-11T07:16:38.159 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 07:16:37 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-prometheus-a[59726]: ts=2024-09-11T07:16:37.884Z caller=main.go:1026 level=info fs_type=XFS_SUPER_MAGIC 2024-09-11T07:16:38.159 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 07:16:37 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-prometheus-a[59726]: ts=2024-09-11T07:16:37.884Z caller=main.go:1029 level=info msg="TSDB started" 2024-09-11T07:16:38.159 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 07:16:37 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-prometheus-a[59726]: ts=2024-09-11T07:16:37.884Z caller=main.go:1209 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2024-09-11T07:16:38.159 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 07:16:37 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-prometheus-a[59726]: ts=2024-09-11T07:16:37.904Z caller=main.go:1246 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=19.903882ms db_storage=826ns remote_storage=1.171µs web_handler=581ns query_engine=510ns scrape=832.727µs scrape_sd=24.499µs notify=1.034µs notify_sd=2.476µs rules=18.774181ms tracing=7.554µs 2024-09-11T07:16:38.159 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 07:16:37 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-prometheus-a[59726]: ts=2024-09-11T07:16:37.904Z caller=main.go:990 level=info msg="Server is ready to receive web requests." 2024-09-11T07:16:38.159 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 07:16:37 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-prometheus-a[59726]: ts=2024-09-11T07:16:37.904Z caller=manager.go:974 level=info component="rule manager" msg="Starting rule manager..." 2024-09-11T07:16:38.159 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 07:16:37 smithi196 bash[59698]: effc2eee58c985d81e3191836b25652719637c072b8a662273c7cc5a3f66627e 2024-09-11T07:16:38.159 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 07:16:37 smithi196 systemd[1]: Started Ceph prometheus.a for a81cffe6-700c-11ef-bcea-c7b262605968. 2024-09-11T07:16:38.758 INFO:teuthology.orchestra.run.smithi196.stdout:[client.1] 2024-09-11T07:16:38.758 INFO:teuthology.orchestra.run.smithi196.stdout: key = AQDWQ+Fm/RbNLBAAsG0aBD/de7DQBrgrmQfJ2A== 2024-09-11T07:16:39.285 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 07:16:39 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-x[32609]: ignoring --setuser ceph since I am not root 2024-09-11T07:16:39.285 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 07:16:39 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-x[32609]: ignoring --setgroup ceph since I am not root 2024-09-11T07:16:39.285 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 07:16:39 smithi196 ceph-mgr[32613]: -- 172.21.15.196:0/1054837597 <== mon.1 v1:172.21.15.196:6789/0 6 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (unknown 2833414162 0 0) 0x557ce6182820 con 0x557ce618f000 2024-09-11T07:16:39.285 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 07:16:39 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-x[32609]: 2024-09-11T07:16:39.164+0000 7f8623063ec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-11T07:16:39.285 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 07:16:39 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-x[32609]: 2024-09-11T07:16:39.219+0000 7f8623063ec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-11T07:16:39.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:39 smithi196 ceph-mon[30773]: pgmap v190: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 85 B/s wr, 1 op/s 2024-09-11T07:16:39.286 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:39 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:16:39.286 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:39 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:16:39.286 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:39 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:16:39.286 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:39 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-09-11T07:16:39.286 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:39 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.196:0/2061694623' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-11T07:16:39.286 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:39 smithi196 ceph-mon[30773]: from='client.24499 ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-11T07:16:39.286 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:39 smithi196 ceph-mon[30773]: from='client.24499 ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-11T07:16:39.287 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:39 smithi071 ceph-mon[27013]: pgmap v190: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 85 B/s wr, 1 op/s 2024-09-11T07:16:39.287 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:39 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:16:39.287 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:39 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:16:39.287 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:39 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:16:39.288 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:39 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-09-11T07:16:39.288 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:39 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.196:0/2061694623' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-11T07:16:39.288 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:39 smithi071 ceph-mon[27013]: from='client.24499 ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-11T07:16:39.288 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:39 smithi071 ceph-mon[27013]: from='client.24499 ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-11T07:16:39.288 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:39 smithi071 ceph-mon[37727]: pgmap v190: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 85 B/s wr, 1 op/s 2024-09-11T07:16:39.288 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:39 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:16:39.288 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:39 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:16:39.288 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:39 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' 2024-09-11T07:16:39.288 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:39 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-09-11T07:16:39.288 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:39 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.196:0/2061694623' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-11T07:16:39.289 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:39 smithi071 ceph-mon[37727]: from='client.24499 ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-11T07:16:39.289 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:39 smithi071 ceph-mon[37727]: from='client.24499 ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-11T07:16:39.289 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:16:39 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: ignoring --setuser ceph since I am not root 2024-09-11T07:16:39.289 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:16:39 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: ignoring --setgroup ceph since I am not root 2024-09-11T07:16:39.289 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:16:39 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: 2024-09-11T07:16:39.168+0000 7efd89423ec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-11T07:16:39.289 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:16:39 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: 2024-09-11T07:16:39.222+0000 7efd89423ec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-11T07:16:39.289 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:16:39 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: 2024-09-11T07:16:39.285+0000 7efd89423ec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-11T07:16:39.523 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-09-11T07:16:39.523 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2024-09-11T07:16:39.523 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-11T07:16:39.541 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:16:39 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: 2024-09-11T07:16:39.406+0000 7efd89423ec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-11T07:16:39.546 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 07:16:39 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-x[32609]: 2024-09-11T07:16:39.283+0000 7f8623063ec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-11T07:16:39.547 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 07:16:39 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-x[32609]: 2024-09-11T07:16:39.407+0000 7f8623063ec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-11T07:16:39.562 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-11T07:16:39.562 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-11T07:16:39.562 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid a81cffe6-700c-11ef-bcea-c7b262605968 -- ceph mgr dump --format=json 2024-09-11T07:16:39.805 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 07:16:39 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-x[32609]: 2024-09-11T07:16:39.547+0000 7f8623063ec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-11T07:16:39.805 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 07:16:39 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-x[32609]: 2024-09-11T07:16:39.802+0000 7f8623063ec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-11T07:16:39.816 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:16:39.818 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:16:39 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: 2024-09-11T07:16:39.539+0000 7efd89423ec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-11T07:16:39.818 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:16:39 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: 2024-09-11T07:16:39.816+0000 7efd89423ec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-11T07:16:40.086 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:40 smithi071 ceph-mon[27013]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-09-11T07:16:40.086 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:40 smithi071 ceph-mon[27013]: mgrmap e17: y(active, since 4m), standbys: x 2024-09-11T07:16:40.086 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:16:39 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: 2024-09-11T07:16:39.880+0000 7efd89423ec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-11T07:16:40.086 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:40 smithi071 ceph-mon[37727]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-09-11T07:16:40.086 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:40 smithi071 ceph-mon[37727]: mgrmap e17: y(active, since 4m), standbys: x 2024-09-11T07:16:40.284 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:40 smithi196 ceph-mon[30773]: from='mgr.14150 v1:172.21.15.71:0/2770671414' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-09-11T07:16:40.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:40 smithi196 ceph-mon[30773]: mgrmap e17: y(active, since 4m), standbys: x 2024-09-11T07:16:40.285 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 07:16:39 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-x[32609]: 2024-09-11T07:16:39.861+0000 7f8623063ec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-11T07:16:40.285 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 07:16:40 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-x[32609]: 2024-09-11T07:16:40.021+0000 7f8623063ec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-11T07:16:40.339 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:16:40 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: 2024-09-11T07:16:40.048+0000 7efd89423ec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-11T07:16:41.195 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:16:40 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: 2024-09-11T07:16:40.894+0000 7efd89423ec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-11T07:16:41.195 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:16:40 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: 2024-09-11T07:16:40.959+0000 7efd89423ec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-11T07:16:41.284 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 07:16:40 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-x[32609]: 2024-09-11T07:16:40.860+0000 7f8623063ec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-11T07:16:41.285 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 07:16:40 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-x[32609]: 2024-09-11T07:16:40.923+0000 7f8623063ec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-11T07:16:41.812 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:16:41 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: 2024-09-11T07:16:41.696+0000 7efd89423ec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-11T07:16:41.812 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:16:41 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: 2024-09-11T07:16:41.809+0000 7efd89423ec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-11T07:16:41.891 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T07:16:41.913 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 07:16:41 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-x[32609]: 2024-09-11T07:16:41.624+0000 7f8623063ec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-11T07:16:41.913 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 07:16:41 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-x[32609]: 2024-09-11T07:16:41.735+0000 7f8623063ec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-11T07:16:41.913 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 07:16:41 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-x[32609]: 2024-09-11T07:16:41.853+0000 7f8623063ec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-11T07:16:42.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:41 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/299163523' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-11T07:16:42.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:41 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/299163523' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-11T07:16:42.090 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:16:41 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: 2024-09-11T07:16:41.935+0000 7efd89423ec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-11T07:16:42.090 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:16:41 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: 2024-09-11T07:16:41.996+0000 7efd89423ec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-11T07:16:42.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:41 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/299163523' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-11T07:16:42.285 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 07:16:41 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-x[32609]: 2024-09-11T07:16:41.910+0000 7f8623063ec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-11T07:16:42.285 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 07:16:42 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-x[32609]: 2024-09-11T07:16:42.036+0000 7f8623063ec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-11T07:16:42.285 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 07:16:42 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-x[32609]: 2024-09-11T07:16:42.102+0000 7f8623063ec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-11T07:16:42.285 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 07:16:42 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-x[32609]: 2024-09-11T07:16:42.159+0000 7f8623063ec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-11T07:16:42.382 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:16:42 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: 2024-09-11T07:16:42.123+0000 7efd89423ec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-11T07:16:42.382 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:16:42 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: 2024-09-11T07:16:42.190+0000 7efd89423ec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-11T07:16:42.382 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:16:42 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: 2024-09-11T07:16:42.249+0000 7efd89423ec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-11T07:16:42.556 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 07:16:42 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-x[32609]: 2024-09-11T07:16:42.292+0000 7f8623063ec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-11T07:16:42.556 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 07:16:42 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-x[32609]: 2024-09-11T07:16:42.355+0000 7f8623063ec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-11T07:16:42.556 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 07:16:42 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-x[32609]: 2024-09-11T07:16:42.418+0000 7f8623063ec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-11T07:16:42.556 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 07:16:42 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-x[32609]: 2024-09-11T07:16:42.498+0000 7f8623063ec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-11T07:16:42.556 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 07:16:42 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-x[32609]: 2024-09-11T07:16:42.553+0000 7f8623063ec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-11T07:16:42.633 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:16:42 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: 2024-09-11T07:16:42.386+0000 7efd89423ec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-11T07:16:42.634 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:16:42 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: 2024-09-11T07:16:42.454+0000 7efd89423ec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-11T07:16:42.634 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:16:42 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: 2024-09-11T07:16:42.519+0000 7efd89423ec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-11T07:16:42.634 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:16:42 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: 2024-09-11T07:16:42.599+0000 7efd89423ec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-11T07:16:42.635 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":17,"flags":0,"active_gid":14150,"active_name":"y","active_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.71:6800","nonce":1383878156}]},"active_addr":"172.21.15.71:6800/1383878156","active_change":"2024-09-11T07:11:57.529100+0000","active_mgr_features":4540138320759226367,"available":true,"standbys":[{"gid":24112,"name":"x","mgr_features":4540138320759226367,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.71:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"cephadm","addrvec":[{"type":"v2","addr":"172.21.15.71:0","nonce":3588838621}]},{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.71:0","nonce":211690729}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.71:0","nonce":1568930701}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.71:0","nonce":101339495}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.71:0","nonce":853553681}]}]} 2024-09-11T07:16:42.639 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-11T07:16:42.639 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-11T07:16:42.639 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid a81cffe6-700c-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T07:16:42.885 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:16:42.994 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:16:42 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: 2024-09-11T07:16:42.661+0000 7efd89423ec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-11T07:16:42.994 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:16:42 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: 2024-09-11T07:16:42.720+0000 7efd89423ec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-11T07:16:43.034 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 07:16:42 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-x[32609]: 2024-09-11T07:16:42.609+0000 7f8623063ec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-11T07:16:43.591 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:43 smithi071 ceph-mon[27013]: Standby manager daemon x restarted 2024-09-11T07:16:43.591 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:43 smithi071 ceph-mon[27013]: Standby manager daemon x started 2024-09-11T07:16:43.591 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:43 smithi071 ceph-mon[27013]: from='mgr.? v1:172.21.15.196:0/1645284085' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-11T07:16:43.591 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:43 smithi071 ceph-mon[27013]: from='mgr.? v1:172.21.15.196:0/1645284085' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T07:16:43.591 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:43 smithi071 ceph-mon[27013]: from='mgr.? v1:172.21.15.196:0/1645284085' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-11T07:16:43.591 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:43 smithi071 ceph-mon[27013]: from='mgr.? v1:172.21.15.196:0/1645284085' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T07:16:43.591 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:16:43 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: 2024-09-11T07:16:43.503+0000 7efd89423ec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-11T07:16:43.592 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:16:43 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: [11/Sep/2024:07:16:43] ENGINE Bus STARTING 2024-09-11T07:16:43.592 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:16:43 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: CherryPy Checker: 2024-09-11T07:16:43.592 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:16:43 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: The Application mounted at '' has an empty config. 2024-09-11T07:16:43.592 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:16:43 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: 2024-09-11T07:16:43.592 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:43 smithi071 ceph-mon[37727]: Standby manager daemon x restarted 2024-09-11T07:16:43.592 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:43 smithi071 ceph-mon[37727]: Standby manager daemon x started 2024-09-11T07:16:43.592 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:43 smithi071 ceph-mon[37727]: from='mgr.? v1:172.21.15.196:0/1645284085' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-11T07:16:43.592 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:43 smithi071 ceph-mon[37727]: from='mgr.? v1:172.21.15.196:0/1645284085' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T07:16:43.592 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:43 smithi071 ceph-mon[37727]: from='mgr.? v1:172.21.15.196:0/1645284085' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-11T07:16:43.592 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:43 smithi071 ceph-mon[37727]: from='mgr.? v1:172.21.15.196:0/1645284085' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T07:16:43.687 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:43 smithi196 ceph-mon[30773]: Standby manager daemon x restarted 2024-09-11T07:16:43.687 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:43 smithi196 ceph-mon[30773]: Standby manager daemon x started 2024-09-11T07:16:43.687 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:43 smithi196 ceph-mon[30773]: from='mgr.? v1:172.21.15.196:0/1645284085' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-11T07:16:43.688 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:43 smithi196 ceph-mon[30773]: from='mgr.? v1:172.21.15.196:0/1645284085' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-11T07:16:43.688 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:43 smithi196 ceph-mon[30773]: from='mgr.? v1:172.21.15.196:0/1645284085' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-11T07:16:43.688 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:43 smithi196 ceph-mon[30773]: from='mgr.? v1:172.21.15.196:0/1645284085' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-11T07:16:43.688 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 07:16:43 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-x[32609]: 2024-09-11T07:16:43.343+0000 7f8623063ec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-11T07:16:43.688 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 07:16:43 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-x[32609]: [11/Sep/2024:07:16:43] ENGINE Bus STARTING 2024-09-11T07:16:43.688 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 07:16:43 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-x[32609]: CherryPy Checker: 2024-09-11T07:16:43.688 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 07:16:43 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-x[32609]: The Application mounted at '' has an empty config. 2024-09-11T07:16:43.688 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 07:16:43 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-x[32609]: 2024-09-11T07:16:43.688 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 07:16:43 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-x[32609]: [11/Sep/2024:07:16:43] ENGINE Serving on http://:::9283 2024-09-11T07:16:43.688 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 07:16:43 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-x[32609]: [11/Sep/2024:07:16:43] ENGINE Bus STARTED 2024-09-11T07:16:43.976 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:16:43 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: [11/Sep/2024:07:16:43] ENGINE Serving on http://:::9283 2024-09-11T07:16:43.976 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:16:43 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: [11/Sep/2024:07:16:43] ENGINE Bus STARTED 2024-09-11T07:16:44.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:44 smithi071 ceph-mon[37727]: mgrmap e18: y(active, since 4m), standbys: x 2024-09-11T07:16:44.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:44 smithi071 ceph-mon[37727]: Active manager daemon y restarted 2024-09-11T07:16:44.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:44 smithi071 ceph-mon[37727]: Activating manager daemon y 2024-09-11T07:16:44.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:44 smithi071 ceph-mon[37727]: osdmap e61: 8 total, 8 up, 8 in 2024-09-11T07:16:44.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:44 smithi071 ceph-mon[37727]: mgrmap e19: y(active, starting, since 0.00967015s), standbys: x 2024-09-11T07:16:44.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:44 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T07:16:44.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:44 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:16:44.591 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:44 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:16:44.591 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:44 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-11T07:16:44.591 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:44 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-11T07:16:44.591 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:44 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T07:16:44.591 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:44 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T07:16:44.591 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:44 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T07:16:44.591 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:44 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T07:16:44.591 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:44 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T07:16:44.591 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:44 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T07:16:44.591 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:44 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T07:16:44.591 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:44 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T07:16:44.591 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:44 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-11T07:16:44.591 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:44 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-11T07:16:44.591 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:44 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-11T07:16:44.591 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:44 smithi071 ceph-mon[37727]: Manager daemon y is now available 2024-09-11T07:16:44.592 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:44 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:16:44.592 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:44 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:16:44.592 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:44 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:16:44.592 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:44 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-11T07:16:44.592 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:44 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-11T07:16:44.592 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:44 smithi071 ceph-mon[27013]: mgrmap e18: y(active, since 4m), standbys: x 2024-09-11T07:16:44.592 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:44 smithi071 ceph-mon[27013]: Active manager daemon y restarted 2024-09-11T07:16:44.593 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:44 smithi071 ceph-mon[27013]: Activating manager daemon y 2024-09-11T07:16:44.593 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:44 smithi071 ceph-mon[27013]: osdmap e61: 8 total, 8 up, 8 in 2024-09-11T07:16:44.593 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:44 smithi071 ceph-mon[27013]: mgrmap e19: y(active, starting, since 0.00967015s), standbys: x 2024-09-11T07:16:44.593 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:44 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T07:16:44.593 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:44 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:16:44.594 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:44 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:16:44.594 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:44 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-11T07:16:44.594 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:44 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-11T07:16:44.594 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:44 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T07:16:44.594 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:44 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T07:16:44.594 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:44 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T07:16:44.594 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:44 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T07:16:44.594 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:44 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T07:16:44.594 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:44 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T07:16:44.594 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:44 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T07:16:44.594 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:44 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T07:16:44.594 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:44 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-11T07:16:44.595 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:44 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-11T07:16:44.595 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:44 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-11T07:16:44.595 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:44 smithi071 ceph-mon[27013]: Manager daemon y is now available 2024-09-11T07:16:44.595 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:44 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:16:44.595 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:44 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:16:44.595 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:44 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:16:44.595 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:44 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-11T07:16:44.595 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:44 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-11T07:16:44.639 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T07:16:44.640 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":61,"fsid":"a81cffe6-700c-11ef-bcea-c7b262605968","created":"2024-09-11T07:11:08.733125+0000","modified":"2024-09-11T07:16:43.506229+0000","last_up_change":"2024-09-11T07:15:56.869561+0000","last_in_change":"2024-09-11T07:15:40.145796+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T07:14:03.821210+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":".rgw.root","create_time":"2024-09-11T07:16:07.399467+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":3,"pool_name":"datapool","create_time":"2024-09-11T07:16:07.711690+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":56,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":4,"pool_name":"default.rgw.log","create_time":"2024-09-11T07:16:09.251445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":5,"pool_name":"default.rgw.control","create_time":"2024-09-11T07:16:11.232118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-09-11T07:16:13.308414+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}}}],"osds":[{"osd":0,"uuid":"10eebb2a-001d-4106-b37b-3af068980575","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.71:6801","nonce":88530034}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.71:6802","nonce":88530034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.71:6804","nonce":88530034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.71:6803","nonce":88530034}]},"public_addr":"172.21.15.71:6801/88530034","cluster_addr":"172.21.15.71:6802/88530034","heartbeat_back_addr":"172.21.15.71:6804/88530034","heartbeat_front_addr":"172.21.15.71:6803/88530034","state":["exists","up"]},{"osd":1,"uuid":"c46d1c53-e519-4ac0-b557-91ee98f659fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.71:6805","nonce":2000038894}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.71:6806","nonce":2000038894}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.71:6808","nonce":2000038894}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.71:6807","nonce":2000038894}]},"public_addr":"172.21.15.71:6805/2000038894","cluster_addr":"172.21.15.71:6806/2000038894","heartbeat_back_addr":"172.21.15.71:6808/2000038894","heartbeat_front_addr":"172.21.15.71:6807/2000038894","state":["exists","up"]},{"osd":2,"uuid":"a6a12661-7514-4887-9acf-a313f7055e30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.71:6809","nonce":3105396883}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.71:6810","nonce":3105396883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.71:6812","nonce":3105396883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.71:6811","nonce":3105396883}]},"public_addr":"172.21.15.71:6809/3105396883","cluster_addr":"172.21.15.71:6810/3105396883","heartbeat_back_addr":"172.21.15.71:6812/3105396883","heartbeat_front_addr":"172.21.15.71:6811/3105396883","state":["exists","up"]},{"osd":3,"uuid":"a66db725-6221-44e6-aa78-3bb92957d455","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.71:6813","nonce":295183903}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.71:6814","nonce":295183903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.71:6816","nonce":295183903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.71:6815","nonce":295183903}]},"public_addr":"172.21.15.71:6813/295183903","cluster_addr":"172.21.15.71:6814/295183903","heartbeat_back_addr":"172.21.15.71:6816/295183903","heartbeat_front_addr":"172.21.15.71:6815/295183903","state":["exists","up"]},{"osd":4,"uuid":"1ac47bc6-7006-4b9f-afe3-676e6f12cee8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.196:6800","nonce":4211374096}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.196:6801","nonce":4211374096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.196:6803","nonce":4211374096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.196:6802","nonce":4211374096}]},"public_addr":"172.21.15.196:6800/4211374096","cluster_addr":"172.21.15.196:6801/4211374096","heartbeat_back_addr":"172.21.15.196:6803/4211374096","heartbeat_front_addr":"172.21.15.196:6802/4211374096","state":["exists","up"]},{"osd":5,"uuid":"683fad79-fdac-4c84-aec8-6d8d3d4ec586","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.196:6804","nonce":3113061101}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.196:6805","nonce":3113061101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.196:6807","nonce":3113061101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.196:6806","nonce":3113061101}]},"public_addr":"172.21.15.196:6804/3113061101","cluster_addr":"172.21.15.196:6805/3113061101","heartbeat_back_addr":"172.21.15.196:6807/3113061101","heartbeat_front_addr":"172.21.15.196:6806/3113061101","state":["exists","up"]},{"osd":6,"uuid":"10eab0a3-9915-4c02-a147-44302ecfb9ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.196:6808","nonce":307456429}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.196:6809","nonce":307456429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.196:6811","nonce":307456429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.196:6810","nonce":307456429}]},"public_addr":"172.21.15.196:6808/307456429","cluster_addr":"172.21.15.196:6809/307456429","heartbeat_back_addr":"172.21.15.196:6811/307456429","heartbeat_front_addr":"172.21.15.196:6810/307456429","state":["exists","up"]},{"osd":7,"uuid":"fb019207-8474-43d5-b431-7995248abf6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.196:6812","nonce":4134490927}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.196:6813","nonce":4134490927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.196:6815","nonce":4134490927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.196:6814","nonce":4134490927}]},"public_addr":"172.21.15.196:6812/4134490927","cluster_addr":"172.21.15.196:6813/4134490927","heartbeat_back_addr":"172.21.15.196:6815/4134490927","heartbeat_front_addr":"172.21.15.196:6814/4134490927","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T07:13:15.011171+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T07:13:38.198270+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T07:14:01.640980+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T07:14:24.678399+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T07:14:45.748090+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T07:15:08.952057+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T07:15:31.816720+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T07:15:55.021672+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.71:0/211690729":"2024-09-12T07:16:43.506200+0000","172.21.15.71:0/3624555087":"2024-09-12T07:11:29.071185+0000","172.21.15.71:0/1493047649":"2024-09-12T07:11:57.528996+0000","172.21.15.71:6800/137476262":"2024-09-12T07:11:57.528996+0000","172.21.15.71:0/384458095":"2024-09-12T07:11:29.071185+0000","172.21.15.71:0/3588838621":"2024-09-12T07:16:43.506200+0000","172.21.15.71:6800/3365647004":"2024-09-12T07:11:29.071185+0000","172.21.15.71:0/1739611680":"2024-09-12T07:11:57.528996+0000","172.21.15.71:6800/1383878156":"2024-09-12T07:16:43.506200+0000","172.21.15.71:0/3185720173":"2024-09-12T07:11:29.071185+0000","172.21.15.71:0/853553681":"2024-09-12T07:16:43.506200+0000","172.21.15.71:0/3954363371":"2024-09-12T07:11:57.528996+0000","172.21.15.71:0/101339495":"2024-09-12T07:16:43.506200+0000","172.21.15.71:0/1568930701":"2024-09-12T07:16:43.506200+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T07:16:44.718 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 11 07:16:44 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-iscsi-iscsi-a[58197]: debug there is no tcmu-runner data available 2024-09-11T07:16:44.718 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:44 smithi196 ceph-mon[30773]: mgrmap e18: y(active, since 4m), standbys: x 2024-09-11T07:16:44.718 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:44 smithi196 ceph-mon[30773]: Active manager daemon y restarted 2024-09-11T07:16:44.718 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:44 smithi196 ceph-mon[30773]: Activating manager daemon y 2024-09-11T07:16:44.718 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:44 smithi196 ceph-mon[30773]: osdmap e61: 8 total, 8 up, 8 in 2024-09-11T07:16:44.718 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:44 smithi196 ceph-mon[30773]: mgrmap e19: y(active, starting, since 0.00967015s), standbys: x 2024-09-11T07:16:44.718 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:44 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-11T07:16:44.718 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:44 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-11T07:16:44.718 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:44 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-11T07:16:44.718 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:44 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-11T07:16:44.718 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:44 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-11T07:16:44.719 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:44 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-11T07:16:44.719 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:44 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T07:16:44.719 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:44 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T07:16:44.719 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:44 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-11T07:16:44.719 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:44 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T07:16:44.719 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:44 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T07:16:44.719 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:44 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T07:16:44.719 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:44 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-11T07:16:44.719 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:44 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-11T07:16:44.719 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:44 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-11T07:16:44.719 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:44 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-11T07:16:44.719 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:44 smithi196 ceph-mon[30773]: Manager daemon y is now available 2024-09-11T07:16:44.719 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:44 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:16:44.719 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:44 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:16:44.719 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:44 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:16:44.720 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:44 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-11T07:16:44.720 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:44 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-11T07:16:45.520 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:16:45 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: 2024-09-11T07:16:45.400+0000 7efd2c1a5640 -1 log_channel(cephadm) log [ERR] : [11/Sep/2024:07:16:45] ENGINE Error in HTTPServer.serve 2024-09-11T07:16:45.521 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:16:45 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: Traceback (most recent call last): 2024-09-11T07:16:45.521 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:16:45 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T07:16:45.521 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:16:45 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: self._connections.run(self.expiration_interval) 2024-09-11T07:16:45.521 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:16:45 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T07:16:45.521 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:16:45 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: self._run(expiration_interval) 2024-09-11T07:16:45.521 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:16:45 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T07:16:45.521 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:16:45 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T07:16:45.521 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:16:45 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T07:16:45.521 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:16:45 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T07:16:45.521 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:16:45 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T07:16:45.521 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:16:45 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: s = self.context.wrap_socket( 2024-09-11T07:16:45.521 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:16:45 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T07:16:45.522 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:16:45 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: return self.sslsocket_class._create( 2024-09-11T07:16:45.522 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:16:45 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T07:16:45.522 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:16:45 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: self.do_handshake() 2024-09-11T07:16:45.522 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:16:45 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T07:16:45.522 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:16:45 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: self._sslobj.do_handshake() 2024-09-11T07:16:45.522 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:16:45 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T07:16:45.522 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:16:45 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: 2024-09-11T07:16:45.544 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-11T07:16:45.545 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid a81cffe6-700c-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-11T07:16:45.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:45 smithi196 ceph-mon[30773]: mgrmap e20: y(active, since 1.0152s), standbys: x 2024-09-11T07:16:45.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:45 smithi196 ceph-mon[30773]: from='client.34442 v1:172.21.15.196:0/1952921461' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:16:45.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:45 smithi196 ceph-mon[30773]: pgmap v3: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:16:45.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:45 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/2301615453' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T07:16:45.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:45 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:16:45.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:45 smithi071 ceph-mon[27013]: mgrmap e20: y(active, since 1.0152s), standbys: x 2024-09-11T07:16:45.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:45 smithi071 ceph-mon[27013]: from='client.34442 v1:172.21.15.196:0/1952921461' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:16:45.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:45 smithi071 ceph-mon[27013]: pgmap v3: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:16:45.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:45 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/2301615453' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T07:16:45.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:45 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:16:45.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:45 smithi071 ceph-mon[37727]: mgrmap e20: y(active, since 1.0152s), standbys: x 2024-09-11T07:16:45.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:45 smithi071 ceph-mon[37727]: from='client.34442 v1:172.21.15.196:0/1952921461' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:16:45.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:45 smithi071 ceph-mon[37727]: pgmap v3: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:16:45.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:45 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/2301615453' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T07:16:45.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:45 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:16:45.875 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:16:46.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:46 smithi071 ceph-mon[27013]: [11/Sep/2024:07:16:45] ENGINE Bus STARTING 2024-09-11T07:16:46.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:46 smithi071 ceph-mon[27013]: [11/Sep/2024:07:16:45] ENGINE Serving on https://172.21.15.71:7150 2024-09-11T07:16:46.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:46 smithi071 ceph-mon[27013]: [11/Sep/2024:07:16:45] ENGINE Bus STARTED 2024-09-11T07:16:46.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:46 smithi071 ceph-mon[27013]: [11/Sep/2024:07:16:45] ENGINE Error in HTTPServer.serve 2024-09-11T07:16:46.590 INFO:journalctl@ceph.mon.a.smithi071.stdout: Traceback (most recent call last): 2024-09-11T07:16:46.590 INFO:journalctl@ceph.mon.a.smithi071.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T07:16:46.590 INFO:journalctl@ceph.mon.a.smithi071.stdout: self._connections.run(self.expiration_interval) 2024-09-11T07:16:46.590 INFO:journalctl@ceph.mon.a.smithi071.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T07:16:46.590 INFO:journalctl@ceph.mon.a.smithi071.stdout: self._run(expiration_interval) 2024-09-11T07:16:46.590 INFO:journalctl@ceph.mon.a.smithi071.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T07:16:46.590 INFO:journalctl@ceph.mon.a.smithi071.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T07:16:46.591 INFO:journalctl@ceph.mon.a.smithi071.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T07:16:46.591 INFO:journalctl@ceph.mon.a.smithi071.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T07:16:46.591 INFO:journalctl@ceph.mon.a.smithi071.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T07:16:46.591 INFO:journalctl@ceph.mon.a.smithi071.stdout: s = self.context.wrap_socket( 2024-09-11T07:16:46.591 INFO:journalctl@ceph.mon.a.smithi071.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T07:16:46.591 INFO:journalctl@ceph.mon.a.smithi071.stdout: return self.sslsocket_class._create( 2024-09-11T07:16:46.591 INFO:journalctl@ceph.mon.a.smithi071.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T07:16:46.591 INFO:journalctl@ceph.mon.a.smithi071.stdout: self.do_handshake() 2024-09-11T07:16:46.591 INFO:journalctl@ceph.mon.a.smithi071.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T07:16:46.591 INFO:journalctl@ceph.mon.a.smithi071.stdout: self._sslobj.do_handshake() 2024-09-11T07:16:46.591 INFO:journalctl@ceph.mon.a.smithi071.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T07:16:46.591 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:46 smithi071 ceph-mon[27013]: pgmap v4: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:16:46.591 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:46 smithi071 ceph-mon[27013]: mgrmap e21: y(active, since 2s), standbys: x 2024-09-11T07:16:46.592 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:46 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:16:46.592 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:46 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:16:46.592 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:46 smithi071 ceph-mon[37727]: [11/Sep/2024:07:16:45] ENGINE Bus STARTING 2024-09-11T07:16:46.592 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:46 smithi071 ceph-mon[37727]: [11/Sep/2024:07:16:45] ENGINE Serving on https://172.21.15.71:7150 2024-09-11T07:16:46.592 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:46 smithi071 ceph-mon[37727]: [11/Sep/2024:07:16:45] ENGINE Bus STARTED 2024-09-11T07:16:46.592 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:46 smithi071 ceph-mon[37727]: [11/Sep/2024:07:16:45] ENGINE Error in HTTPServer.serve 2024-09-11T07:16:46.592 INFO:journalctl@ceph.mon.c.smithi071.stdout: Traceback (most recent call last): 2024-09-11T07:16:46.592 INFO:journalctl@ceph.mon.c.smithi071.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T07:16:46.592 INFO:journalctl@ceph.mon.c.smithi071.stdout: self._connections.run(self.expiration_interval) 2024-09-11T07:16:46.592 INFO:journalctl@ceph.mon.c.smithi071.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T07:16:46.592 INFO:journalctl@ceph.mon.c.smithi071.stdout: self._run(expiration_interval) 2024-09-11T07:16:46.592 INFO:journalctl@ceph.mon.c.smithi071.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T07:16:46.593 INFO:journalctl@ceph.mon.c.smithi071.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T07:16:46.593 INFO:journalctl@ceph.mon.c.smithi071.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T07:16:46.593 INFO:journalctl@ceph.mon.c.smithi071.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T07:16:46.593 INFO:journalctl@ceph.mon.c.smithi071.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T07:16:46.593 INFO:journalctl@ceph.mon.c.smithi071.stdout: s = self.context.wrap_socket( 2024-09-11T07:16:46.593 INFO:journalctl@ceph.mon.c.smithi071.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T07:16:46.593 INFO:journalctl@ceph.mon.c.smithi071.stdout: return self.sslsocket_class._create( 2024-09-11T07:16:46.593 INFO:journalctl@ceph.mon.c.smithi071.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T07:16:46.593 INFO:journalctl@ceph.mon.c.smithi071.stdout: self.do_handshake() 2024-09-11T07:16:46.593 INFO:journalctl@ceph.mon.c.smithi071.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T07:16:46.593 INFO:journalctl@ceph.mon.c.smithi071.stdout: self._sslobj.do_handshake() 2024-09-11T07:16:46.593 INFO:journalctl@ceph.mon.c.smithi071.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T07:16:46.594 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:46 smithi071 ceph-mon[37727]: pgmap v4: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:16:46.594 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:46 smithi071 ceph-mon[37727]: mgrmap e21: y(active, since 2s), standbys: x 2024-09-11T07:16:46.594 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:46 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:16:46.594 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:46 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:16:46.785 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 07:16:46 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-x[32609]: ::ffff:172.21.15.196 - - [11/Sep/2024:07:16:46] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:16:46.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:46 smithi196 ceph-mon[30773]: [11/Sep/2024:07:16:45] ENGINE Bus STARTING 2024-09-11T07:16:46.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:46 smithi196 ceph-mon[30773]: [11/Sep/2024:07:16:45] ENGINE Serving on https://172.21.15.71:7150 2024-09-11T07:16:46.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:46 smithi196 ceph-mon[30773]: [11/Sep/2024:07:16:45] ENGINE Bus STARTED 2024-09-11T07:16:46.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:46 smithi196 ceph-mon[30773]: [11/Sep/2024:07:16:45] ENGINE Error in HTTPServer.serve 2024-09-11T07:16:46.785 INFO:journalctl@ceph.mon.b.smithi196.stdout: Traceback (most recent call last): 2024-09-11T07:16:46.785 INFO:journalctl@ceph.mon.b.smithi196.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-11T07:16:46.786 INFO:journalctl@ceph.mon.b.smithi196.stdout: self._connections.run(self.expiration_interval) 2024-09-11T07:16:46.786 INFO:journalctl@ceph.mon.b.smithi196.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-11T07:16:46.786 INFO:journalctl@ceph.mon.b.smithi196.stdout: self._run(expiration_interval) 2024-09-11T07:16:46.786 INFO:journalctl@ceph.mon.b.smithi196.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-11T07:16:46.786 INFO:journalctl@ceph.mon.b.smithi196.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-11T07:16:46.786 INFO:journalctl@ceph.mon.b.smithi196.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-11T07:16:46.786 INFO:journalctl@ceph.mon.b.smithi196.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-11T07:16:46.786 INFO:journalctl@ceph.mon.b.smithi196.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-11T07:16:46.786 INFO:journalctl@ceph.mon.b.smithi196.stdout: s = self.context.wrap_socket( 2024-09-11T07:16:46.786 INFO:journalctl@ceph.mon.b.smithi196.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-11T07:16:46.787 INFO:journalctl@ceph.mon.b.smithi196.stdout: return self.sslsocket_class._create( 2024-09-11T07:16:46.787 INFO:journalctl@ceph.mon.b.smithi196.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-11T07:16:46.787 INFO:journalctl@ceph.mon.b.smithi196.stdout: self.do_handshake() 2024-09-11T07:16:46.787 INFO:journalctl@ceph.mon.b.smithi196.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-11T07:16:46.787 INFO:journalctl@ceph.mon.b.smithi196.stdout: self._sslobj.do_handshake() 2024-09-11T07:16:46.787 INFO:journalctl@ceph.mon.b.smithi196.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-11T07:16:46.787 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:46 smithi196 ceph-mon[30773]: pgmap v4: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:16:46.787 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:46 smithi196 ceph-mon[30773]: mgrmap e21: y(active, since 2s), standbys: x 2024-09-11T07:16:46.787 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:46 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:16:46.787 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:46 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:16:47.632 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T07:16:47.632 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":61,"fsid":"a81cffe6-700c-11ef-bcea-c7b262605968","created":"2024-09-11T07:11:08.733125+0000","modified":"2024-09-11T07:16:43.506229+0000","last_up_change":"2024-09-11T07:15:56.869561+0000","last_in_change":"2024-09-11T07:15:40.145796+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T07:14:03.821210+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":".rgw.root","create_time":"2024-09-11T07:16:07.399467+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":3,"pool_name":"datapool","create_time":"2024-09-11T07:16:07.711690+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":56,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":4,"pool_name":"default.rgw.log","create_time":"2024-09-11T07:16:09.251445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":5,"pool_name":"default.rgw.control","create_time":"2024-09-11T07:16:11.232118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-09-11T07:16:13.308414+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}}}],"osds":[{"osd":0,"uuid":"10eebb2a-001d-4106-b37b-3af068980575","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.71:6801","nonce":88530034}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.71:6802","nonce":88530034}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.71:6804","nonce":88530034}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.71:6803","nonce":88530034}]},"public_addr":"172.21.15.71:6801/88530034","cluster_addr":"172.21.15.71:6802/88530034","heartbeat_back_addr":"172.21.15.71:6804/88530034","heartbeat_front_addr":"172.21.15.71:6803/88530034","state":["exists","up"]},{"osd":1,"uuid":"c46d1c53-e519-4ac0-b557-91ee98f659fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.71:6805","nonce":2000038894}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.71:6806","nonce":2000038894}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.71:6808","nonce":2000038894}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.71:6807","nonce":2000038894}]},"public_addr":"172.21.15.71:6805/2000038894","cluster_addr":"172.21.15.71:6806/2000038894","heartbeat_back_addr":"172.21.15.71:6808/2000038894","heartbeat_front_addr":"172.21.15.71:6807/2000038894","state":["exists","up"]},{"osd":2,"uuid":"a6a12661-7514-4887-9acf-a313f7055e30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.71:6809","nonce":3105396883}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.71:6810","nonce":3105396883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.71:6812","nonce":3105396883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.71:6811","nonce":3105396883}]},"public_addr":"172.21.15.71:6809/3105396883","cluster_addr":"172.21.15.71:6810/3105396883","heartbeat_back_addr":"172.21.15.71:6812/3105396883","heartbeat_front_addr":"172.21.15.71:6811/3105396883","state":["exists","up"]},{"osd":3,"uuid":"a66db725-6221-44e6-aa78-3bb92957d455","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.71:6813","nonce":295183903}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.71:6814","nonce":295183903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.71:6816","nonce":295183903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.71:6815","nonce":295183903}]},"public_addr":"172.21.15.71:6813/295183903","cluster_addr":"172.21.15.71:6814/295183903","heartbeat_back_addr":"172.21.15.71:6816/295183903","heartbeat_front_addr":"172.21.15.71:6815/295183903","state":["exists","up"]},{"osd":4,"uuid":"1ac47bc6-7006-4b9f-afe3-676e6f12cee8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.196:6800","nonce":4211374096}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.196:6801","nonce":4211374096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.196:6803","nonce":4211374096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.196:6802","nonce":4211374096}]},"public_addr":"172.21.15.196:6800/4211374096","cluster_addr":"172.21.15.196:6801/4211374096","heartbeat_back_addr":"172.21.15.196:6803/4211374096","heartbeat_front_addr":"172.21.15.196:6802/4211374096","state":["exists","up"]},{"osd":5,"uuid":"683fad79-fdac-4c84-aec8-6d8d3d4ec586","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.196:6804","nonce":3113061101}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.196:6805","nonce":3113061101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.196:6807","nonce":3113061101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.196:6806","nonce":3113061101}]},"public_addr":"172.21.15.196:6804/3113061101","cluster_addr":"172.21.15.196:6805/3113061101","heartbeat_back_addr":"172.21.15.196:6807/3113061101","heartbeat_front_addr":"172.21.15.196:6806/3113061101","state":["exists","up"]},{"osd":6,"uuid":"10eab0a3-9915-4c02-a147-44302ecfb9ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.196:6808","nonce":307456429}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.196:6809","nonce":307456429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.196:6811","nonce":307456429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.196:6810","nonce":307456429}]},"public_addr":"172.21.15.196:6808/307456429","cluster_addr":"172.21.15.196:6809/307456429","heartbeat_back_addr":"172.21.15.196:6811/307456429","heartbeat_front_addr":"172.21.15.196:6810/307456429","state":["exists","up"]},{"osd":7,"uuid":"fb019207-8474-43d5-b431-7995248abf6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.196:6812","nonce":4134490927}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.196:6813","nonce":4134490927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.196:6815","nonce":4134490927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.196:6814","nonce":4134490927}]},"public_addr":"172.21.15.196:6812/4134490927","cluster_addr":"172.21.15.196:6813/4134490927","heartbeat_back_addr":"172.21.15.196:6815/4134490927","heartbeat_front_addr":"172.21.15.196:6814/4134490927","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T07:13:15.011171+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T07:13:38.198270+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T07:14:01.640980+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T07:14:24.678399+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T07:14:45.748090+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T07:15:08.952057+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T07:15:31.816720+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T07:15:55.021672+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.71:0/211690729":"2024-09-12T07:16:43.506200+0000","172.21.15.71:0/3624555087":"2024-09-12T07:11:29.071185+0000","172.21.15.71:0/1493047649":"2024-09-12T07:11:57.528996+0000","172.21.15.71:6800/137476262":"2024-09-12T07:11:57.528996+0000","172.21.15.71:0/384458095":"2024-09-12T07:11:29.071185+0000","172.21.15.71:0/3588838621":"2024-09-12T07:16:43.506200+0000","172.21.15.71:6800/3365647004":"2024-09-12T07:11:29.071185+0000","172.21.15.71:0/1739611680":"2024-09-12T07:11:57.528996+0000","172.21.15.71:6800/1383878156":"2024-09-12T07:16:43.506200+0000","172.21.15.71:0/3185720173":"2024-09-12T07:11:29.071185+0000","172.21.15.71:0/853553681":"2024-09-12T07:16:43.506200+0000","172.21.15.71:0/3954363371":"2024-09-12T07:11:57.528996+0000","172.21.15.71:0/101339495":"2024-09-12T07:16:43.506200+0000","172.21.15.71:0/1568930701":"2024-09-12T07:16:43.506200+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T07:16:48.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:48 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:16:48.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:48 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:16:48.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:48 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:16:48.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:48 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:16:48.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:48 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:16:48.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:48 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:16:48.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:48 smithi071 ceph-mon[27013]: Adjusting osd_memory_target on smithi196 to 3760M 2024-09-11T07:16:48.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:48 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:16:48.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:48 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/4216297307' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T07:16:48.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:48 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:16:48.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:48 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:16:48.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:48 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:16:48.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:48 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:16:48.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:48 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:16:48.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:48 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:16:48.341 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:48 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:16:48.341 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:48 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:16:48.341 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:48 smithi071 ceph-mon[37727]: Adjusting osd_memory_target on smithi196 to 3760M 2024-09-11T07:16:48.341 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:48 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:16:48.341 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:48 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/4216297307' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T07:16:48.341 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:48 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:16:48.341 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:48 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:16:48.400 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid a81cffe6-700c-11ef-bcea-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-09-11T07:16:48.401 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid a81cffe6-700c-11ef-bcea-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-09-11T07:16:48.401 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid a81cffe6-700c-11ef-bcea-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-09-11T07:16:48.401 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid a81cffe6-700c-11ef-bcea-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-09-11T07:16:48.402 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid a81cffe6-700c-11ef-bcea-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-09-11T07:16:48.402 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid a81cffe6-700c-11ef-bcea-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-09-11T07:16:48.403 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid a81cffe6-700c-11ef-bcea-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-09-11T07:16:48.403 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid a81cffe6-700c-11ef-bcea-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-09-11T07:16:48.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:48 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:16:48.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:48 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:16:48.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:48 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:16:48.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:48 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:16:48.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:48 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:16:48.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:48 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:16:48.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:48 smithi196 ceph-mon[30773]: Adjusting osd_memory_target on smithi196 to 3760M 2024-09-11T07:16:48.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:48 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:16:48.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:48 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/4216297307' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T07:16:48.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:48 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:16:48.536 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:48 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:16:48.953 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:16:48.955 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:16:48.969 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:16:49.018 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:16:49.284 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:49 smithi196 ceph-mon[30773]: pgmap v5: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:16:49.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:49 smithi196 ceph-mon[30773]: mgrmap e22: y(active, since 4s), standbys: x 2024-09-11T07:16:49.337 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:49 smithi071 ceph-mon[27013]: pgmap v5: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:16:49.337 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:49 smithi071 ceph-mon[27013]: mgrmap e22: y(active, since 4s), standbys: x 2024-09-11T07:16:49.338 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:49 smithi071 ceph-mon[37727]: pgmap v5: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:16:49.338 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:49 smithi071 ceph-mon[37727]: mgrmap e22: y(active, since 4s), standbys: x 2024-09-11T07:16:50.448 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:16:50.448 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:16:50.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:50 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:16:50.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:50 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:16:50.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:50 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:16:50.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:50 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:16:50.536 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:50 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:16:50.536 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:50 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:16:50.536 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:50 smithi196 ceph-mon[30773]: Adjusting osd_memory_target on smithi071 to 3760M 2024-09-11T07:16:50.536 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:50 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:16:50.536 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:50 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:16:50.536 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:50 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:16:50.536 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:50 smithi196 ceph-mon[30773]: Updating smithi071:/etc/ceph/ceph.conf 2024-09-11T07:16:50.536 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:50 smithi196 ceph-mon[30773]: Updating smithi196:/etc/ceph/ceph.conf 2024-09-11T07:16:50.536 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:50 smithi196 ceph-mon[30773]: Updating smithi071:/var/lib/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T07:16:50.537 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:50 smithi196 ceph-mon[30773]: Updating smithi196:/var/lib/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T07:16:50.537 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:50 smithi196 ceph-mon[30773]: pgmap v6: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:16:50.537 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:50 smithi196 ceph-mon[30773]: Updating smithi071:/etc/ceph/ceph.client.admin.keyring 2024-09-11T07:16:50.537 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:50 smithi196 ceph-mon[30773]: Updating smithi196:/etc/ceph/ceph.client.admin.keyring 2024-09-11T07:16:50.537 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:50 smithi196 ceph-mon[30773]: Updating smithi071:/var/lib/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T07:16:50.537 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:50 smithi196 ceph-mon[30773]: Updating smithi196:/var/lib/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T07:16:50.537 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:50 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:16:50.537 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:50 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:16:50.537 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:50 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:16:50.537 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:50 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:16:50.537 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:50 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:16:50.573 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:50 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:16:50.573 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:50 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:16:50.573 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:50 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:16:50.573 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:50 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:16:50.574 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:50 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:16:50.574 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:50 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:16:50.574 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:50 smithi071 ceph-mon[27013]: Adjusting osd_memory_target on smithi071 to 3760M 2024-09-11T07:16:50.574 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:50 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:16:50.574 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:50 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:16:50.574 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:50 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:16:50.574 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:50 smithi071 ceph-mon[27013]: Updating smithi071:/etc/ceph/ceph.conf 2024-09-11T07:16:50.574 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:50 smithi071 ceph-mon[27013]: Updating smithi196:/etc/ceph/ceph.conf 2024-09-11T07:16:50.574 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:50 smithi071 ceph-mon[27013]: Updating smithi071:/var/lib/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T07:16:50.574 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:50 smithi071 ceph-mon[27013]: Updating smithi196:/var/lib/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T07:16:50.574 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:50 smithi071 ceph-mon[27013]: pgmap v6: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:16:50.575 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:50 smithi071 ceph-mon[27013]: Updating smithi071:/etc/ceph/ceph.client.admin.keyring 2024-09-11T07:16:50.575 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:50 smithi071 ceph-mon[27013]: Updating smithi196:/etc/ceph/ceph.client.admin.keyring 2024-09-11T07:16:50.575 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:50 smithi071 ceph-mon[27013]: Updating smithi071:/var/lib/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T07:16:50.575 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:50 smithi071 ceph-mon[27013]: Updating smithi196:/var/lib/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T07:16:50.575 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:50 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:16:50.575 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:50 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:16:50.575 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:50 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:16:50.575 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:50 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:16:50.575 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:50 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:16:50.575 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:50 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:16:50.576 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:50 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:16:50.576 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:50 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:16:50.576 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:50 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:16:50.576 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:50 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:16:50.576 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:50 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-11T07:16:50.576 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:50 smithi071 ceph-mon[37727]: Adjusting osd_memory_target on smithi071 to 3760M 2024-09-11T07:16:50.576 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:50 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:16:50.576 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:50 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:16:50.576 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:50 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:16:50.576 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:50 smithi071 ceph-mon[37727]: Updating smithi071:/etc/ceph/ceph.conf 2024-09-11T07:16:50.576 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:50 smithi071 ceph-mon[37727]: Updating smithi196:/etc/ceph/ceph.conf 2024-09-11T07:16:50.577 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:50 smithi071 ceph-mon[37727]: Updating smithi071:/var/lib/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T07:16:50.577 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:50 smithi071 ceph-mon[37727]: Updating smithi196:/var/lib/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-11T07:16:50.577 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:50 smithi071 ceph-mon[37727]: pgmap v6: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:16:50.577 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:50 smithi071 ceph-mon[37727]: Updating smithi071:/etc/ceph/ceph.client.admin.keyring 2024-09-11T07:16:50.577 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:50 smithi071 ceph-mon[37727]: Updating smithi196:/etc/ceph/ceph.client.admin.keyring 2024-09-11T07:16:50.577 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:50 smithi071 ceph-mon[37727]: Updating smithi071:/var/lib/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T07:16:50.577 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:50 smithi071 ceph-mon[37727]: Updating smithi196:/var/lib/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-11T07:16:50.577 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:50 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:16:50.577 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:50 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:16:50.577 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:50 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:16:50.577 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:50 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:16:50.577 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:50 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:16:51.490 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:51 smithi071 ceph-mon[27013]: Deploying daemon alertmanager.a on smithi071 2024-09-11T07:16:51.490 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:51 smithi071 ceph-mon[37727]: Deploying daemon alertmanager.a on smithi071 2024-09-11T07:16:51.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:51 smithi196 ceph-mon[30773]: Deploying daemon alertmanager.a on smithi071 2024-09-11T07:16:52.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:52 smithi196 ceph-mon[30773]: pgmap v7: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 26 KiB/s rd, 0 B/s wr, 11 op/s 2024-09-11T07:16:52.553 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:52 smithi071 ceph-mon[27013]: pgmap v7: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 26 KiB/s rd, 0 B/s wr, 11 op/s 2024-09-11T07:16:52.553 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:52 smithi071 ceph-mon[37727]: pgmap v7: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 26 KiB/s rd, 0 B/s wr, 11 op/s 2024-09-11T07:16:53.153 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:16:53.154 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:16:54.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:54 smithi071 ceph-mon[27013]: pgmap v8: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-11T07:16:54.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:54 smithi071 ceph-mon[37727]: pgmap v8: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-11T07:16:55.034 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:54 smithi196 ceph-mon[30773]: pgmap v8: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-11T07:16:55.035 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 11 07:16:54 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-iscsi-iscsi-a[58197]: debug there is no tcmu-runner data available 2024-09-11T07:16:55.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:55 smithi071 ceph-mon[27013]: from='client.34442 v1:172.21.15.196:0/1952921461' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:16:55.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:55 smithi071 ceph-mon[37727]: from='client.34442 v1:172.21.15.196:0/1952921461' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:16:56.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:55 smithi196 ceph-mon[30773]: from='client.34442 v1:172.21.15.196:0/1952921461' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:16:56.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:56 smithi071 ceph-mon[27013]: pgmap v9: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 17 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-11T07:16:56.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:56 smithi071 ceph-mon[37727]: pgmap v9: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 17 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-11T07:16:57.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:56 smithi196 ceph-mon[30773]: pgmap v9: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 17 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-11T07:16:57.035 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 07:16:56 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-x[32609]: ::ffff:172.21.15.196 - - [11/Sep/2024:07:16:56] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:16:58.564 INFO:teuthology.orchestra.run.smithi071.stdout:55834574889 2024-09-11T07:16:58.564 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid a81cffe6-700c-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-09-11T07:16:58.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:58 smithi071 ceph-mon[27013]: pgmap v10: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 15 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-11T07:16:58.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:16:58 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:16:58.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:58 smithi071 ceph-mon[37727]: pgmap v10: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 15 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-11T07:16:58.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:16:58 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:16:59.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:58 smithi196 ceph-mon[30773]: pgmap v10: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 15 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-11T07:16:59.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:16:58 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:17:00.763 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:17:00 smithi071 ceph-mon[27013]: pgmap v11: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 15 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-11T07:17:00.763 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:17:00 smithi071 ceph-mon[37727]: pgmap v11: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 15 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-11T07:17:00.874 INFO:teuthology.orchestra.run.smithi071.stdout:34359738413 2024-09-11T07:17:00.875 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid a81cffe6-700c-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-09-11T07:17:01.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:17:00 smithi196 ceph-mon[30773]: pgmap v11: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 15 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-11T07:17:02.805 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:17:02 smithi071 ceph-mon[27013]: pgmap v12: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 16 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-11T07:17:02.806 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:17:02 smithi071 ceph-mon[37727]: pgmap v12: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 16 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-11T07:17:03.034 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:17:02 smithi196 ceph-mon[30773]: pgmap v12: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 16 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-11T07:17:03.906 INFO:teuthology.orchestra.run.smithi071.stdout:206158430223 2024-09-11T07:17:03.906 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid a81cffe6-700c-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-09-11T07:17:04.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:17:04 smithi071 ceph-mon[27013]: pgmap v13: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:17:04.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:17:04 smithi071 ceph-mon[37727]: pgmap v13: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:17:04.868 INFO:teuthology.orchestra.run.smithi071.stdout:77309411365 2024-09-11T07:17:04.869 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid a81cffe6-700c-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-09-11T07:17:05.034 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:17:04 smithi196 ceph-mon[30773]: pgmap v13: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:17:05.035 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 11 07:17:04 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-iscsi-iscsi-a[58197]: debug there is no tcmu-runner data available 2024-09-11T07:17:05.748 INFO:teuthology.orchestra.run.smithi071.stdout:180388626452 2024-09-11T07:17:05.748 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid a81cffe6-700c-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-09-11T07:17:06.006 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:17:05 smithi071 ceph-mon[27013]: from='client.34442 v1:172.21.15.196:0/1952921461' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:17:06.006 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:17:05 smithi071 ceph-mon[37727]: from='client.34442 v1:172.21.15.196:0/1952921461' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:17:06.034 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:17:05 smithi196 ceph-mon[30773]: from='client.34442 v1:172.21.15.196:0/1952921461' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:17:06.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:17:06 smithi071 ceph-mon[27013]: pgmap v14: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-11T07:17:06.839 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:17:06 smithi071 ceph-mon[37727]: pgmap v14: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-11T07:17:06.936 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:17:06 smithi196 ceph-mon[30773]: pgmap v14: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-11T07:17:06.936 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 07:17:06 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-x[32609]: ::ffff:172.21.15.196 - - [11/Sep/2024:07:17:06] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:17:08.825 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:17:08 smithi071 ceph-mon[27013]: pgmap v15: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:17:08.825 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:17:08 smithi071 ceph-mon[37727]: pgmap v15: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:17:09.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:17:08 smithi196 ceph-mon[30773]: pgmap v15: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:17:09.580 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:17:09.580 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:17:09.600 INFO:teuthology.orchestra.run.smithi071.stdout:154618822681 2024-09-11T07:17:09.600 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid a81cffe6-700c-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-09-11T07:17:10.464 INFO:teuthology.orchestra.run.smithi071.stdout:107374182435 2024-09-11T07:17:10.464 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid a81cffe6-700c-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-09-11T07:17:10.722 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:17:10 smithi071 ceph-mon[27013]: pgmap v16: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:17:10.722 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:17:10 smithi071 ceph-mon[37727]: pgmap v16: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:17:11.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:17:10 smithi196 ceph-mon[30773]: pgmap v16: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:17:11.111 INFO:teuthology.orchestra.run.smithi071.stdout:128849018910 2024-09-11T07:17:11.111 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid a81cffe6-700c-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-09-11T07:17:12.076 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:17:12.076 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:17:12.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:17:12 smithi071 ceph-mon[27013]: pgmap v17: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:17:12.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:17:12 smithi071 ceph-mon[37727]: pgmap v17: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:17:13.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:17:12 smithi196 ceph-mon[30773]: pgmap v17: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:17:13.922 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:17:13 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:17:13.922 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:17:13 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:17:14.034 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:17:13 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:17:14.486 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:17:14.919 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:17:14 smithi071 ceph-mon[27013]: pgmap v18: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:17:14.919 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:17:14 smithi071 ceph-mon[37727]: pgmap v18: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:17:15.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:17:14 smithi196 ceph-mon[30773]: pgmap v18: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:17:15.035 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 11 07:17:14 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-iscsi-iscsi-a[58197]: debug there is no tcmu-runner data available 2024-09-11T07:17:15.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:17:15 smithi071 ceph-mon[27013]: from='client.34442 v1:172.21.15.196:0/1952921461' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:17:15.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:17:15 smithi071 ceph-mon[37727]: from='client.34442 v1:172.21.15.196:0/1952921461' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:17:16.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:17:15 smithi196 ceph-mon[30773]: from='client.34442 v1:172.21.15.196:0/1952921461' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:17:16.950 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:17:16 smithi071 ceph-mon[27013]: pgmap v19: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:17:16.951 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:17:16 smithi071 ceph-mon[37727]: pgmap v19: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:17:17.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:17:16 smithi196 ceph-mon[30773]: pgmap v19: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:17:17.035 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 07:17:16 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-x[32609]: ::ffff:172.21.15.196 - - [11/Sep/2024:07:17:16] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:17:17.198 INFO:teuthology.orchestra.run.smithi071.stdout:34359738417 2024-09-11T07:17:17.391 INFO:teuthology.orchestra.run.smithi071.stdout:77309411368 2024-09-11T07:17:18.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:17:17 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/2991575775' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-11T07:17:18.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:17:17 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/1980497594' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-11T07:17:18.072 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:17:17 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/2991575775' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-11T07:17:18.072 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:17:17 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/1980497594' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-11T07:17:18.073 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:17:17 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/2991575775' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-11T07:17:18.073 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:17:17 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/1980497594' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-11T07:17:18.102 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411365 got 77309411368 for osd.2 2024-09-11T07:17:18.102 DEBUG:teuthology.parallel:result is None 2024-09-11T07:17:18.716 INFO:teuthology.orchestra.run.smithi071.stdout:55834574894 2024-09-11T07:17:18.895 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:17:18 smithi071 ceph-mon[27013]: pgmap v20: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:17:18.895 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:17:18 smithi071 ceph-mon[37727]: pgmap v20: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:17:18.902 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:17:18.903 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:17:18.904 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:17:19.034 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:17:18 smithi196 ceph-mon[30773]: pgmap v20: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:17:19.592 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574889 got 55834574894 for osd.1 2024-09-11T07:17:19.592 DEBUG:teuthology.parallel:result is None 2024-09-11T07:17:19.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:17:19 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/637443506' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-11T07:17:19.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:17:19 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/637443506' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-11T07:17:19.959 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738413 got 34359738417 for osd.0 2024-09-11T07:17:19.959 DEBUG:teuthology.parallel:result is None 2024-09-11T07:17:20.034 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:17:19 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/637443506' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-11T07:17:20.916 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:17:20 smithi071 ceph-mon[27013]: pgmap v21: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:17:20.917 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:17:20 smithi071 ceph-mon[37727]: pgmap v21: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:17:21.034 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:17:20 smithi196 ceph-mon[30773]: pgmap v21: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:17:22.468 INFO:journalctl@ceph.alertmanager.a.smithi071.stdout:Sep 11 07:17:22 smithi071 podman[69923]: 2024-09-11 07:17:22.466561026 +0000 UTC m=+1.860873503 container create 5581897770a7032d42952e97b7967338d64267239d5d2fb227485c3fba4e36f1 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-11T07:17:22.787 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:17:22 smithi071 ceph-mon[27013]: pgmap v22: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:17:22.788 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:17:22 smithi071 ceph-mon[37727]: pgmap v22: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:17:23.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:17:22 smithi196 ceph-mon[30773]: pgmap v22: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:17:23.047 INFO:journalctl@ceph.alertmanager.a.smithi071.stdout:Sep 11 07:17:22 smithi071 podman[69923]: 2024-09-11 07:17:22.870393313 +0000 UTC m=+2.264705758 container init 5581897770a7032d42952e97b7967338d64267239d5d2fb227485c3fba4e36f1 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-11T07:17:23.048 INFO:journalctl@ceph.alertmanager.a.smithi071.stdout:Sep 11 07:17:22 smithi071 podman[69923]: 2024-09-11 07:17:22.874320167 +0000 UTC m=+2.268632598 container start 5581897770a7032d42952e97b7967338d64267239d5d2fb227485c3fba4e36f1 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-11T07:17:23.048 INFO:journalctl@ceph.alertmanager.a.smithi071.stdout:Sep 11 07:17:22 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-alertmanager-a[70009]: ts=2024-09-11T07:17:22.886Z caller=main.go:240 level=info msg="Starting Alertmanager" version="(version=0.25.0, branch=HEAD, revision=258fab7cdd551f2cf251ed0348f0ad7289aee789)" 2024-09-11T07:17:23.048 INFO:journalctl@ceph.alertmanager.a.smithi071.stdout:Sep 11 07:17:22 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-alertmanager-a[70009]: ts=2024-09-11T07:17:22.886Z caller=main.go:241 level=info build_context="(go=go1.19.4, user=root@abe866dd5717, date=20221222-14:51:36)" 2024-09-11T07:17:23.048 INFO:journalctl@ceph.alertmanager.a.smithi071.stdout:Sep 11 07:17:22 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-alertmanager-a[70009]: ts=2024-09-11T07:17:22.887Z caller=cluster.go:185 level=info component=cluster msg="setting advertise address explicitly" addr=172.21.15.71 port=9094 2024-09-11T07:17:23.048 INFO:journalctl@ceph.alertmanager.a.smithi071.stdout:Sep 11 07:17:22 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-alertmanager-a[70009]: ts=2024-09-11T07:17:22.888Z caller=cluster.go:681 level=info component=cluster msg="Waiting for gossip to settle..." interval=2s 2024-09-11T07:17:23.048 INFO:journalctl@ceph.alertmanager.a.smithi071.stdout:Sep 11 07:17:22 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-alertmanager-a[70009]: ts=2024-09-11T07:17:22.911Z caller=coordinator.go:113 level=info component=configuration msg="Loading configuration file" file=/etc/alertmanager/alertmanager.yml 2024-09-11T07:17:23.048 INFO:journalctl@ceph.alertmanager.a.smithi071.stdout:Sep 11 07:17:22 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-alertmanager-a[70009]: ts=2024-09-11T07:17:22.911Z caller=coordinator.go:126 level=info component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/alertmanager.yml 2024-09-11T07:17:23.048 INFO:journalctl@ceph.alertmanager.a.smithi071.stdout:Sep 11 07:17:22 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-alertmanager-a[70009]: ts=2024-09-11T07:17:22.914Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9093 2024-09-11T07:17:23.048 INFO:journalctl@ceph.alertmanager.a.smithi071.stdout:Sep 11 07:17:22 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-alertmanager-a[70009]: ts=2024-09-11T07:17:22.914Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9093 2024-09-11T07:17:23.048 INFO:journalctl@ceph.alertmanager.a.smithi071.stdout:Sep 11 07:17:22 smithi071 bash[69923]: 5581897770a7032d42952e97b7967338d64267239d5d2fb227485c3fba4e36f1 2024-09-11T07:17:23.048 INFO:journalctl@ceph.alertmanager.a.smithi071.stdout:Sep 11 07:17:22 smithi071 systemd[1]: Started Ceph alertmanager.a for a81cffe6-700c-11ef-bcea-c7b262605968. 2024-09-11T07:17:23.339 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:17:23 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: [11/Sep/2024:07:17:23] ENGINE Bus STOPPING 2024-09-11T07:17:23.339 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:17:23 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: [11/Sep/2024:07:17:23] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-11T07:17:23.339 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:17:23 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: [11/Sep/2024:07:17:23] ENGINE Bus STOPPED 2024-09-11T07:17:23.339 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:17:23 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: [11/Sep/2024:07:17:23] ENGINE Bus STARTING 2024-09-11T07:17:23.608 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:17:23 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: [11/Sep/2024:07:17:23] ENGINE Serving on http://:::9283 2024-09-11T07:17:23.609 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:17:23 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: [11/Sep/2024:07:17:23] ENGINE Bus STARTED 2024-09-11T07:17:23.819 INFO:teuthology.orchestra.run.smithi071.stdout:206158430227 2024-09-11T07:17:24.161 INFO:teuthology.orchestra.run.smithi071.stdout:107374182437 2024-09-11T07:17:24.173 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:17:24 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:17:24.173 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:17:24 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:17:24.173 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:17:24 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/1946670557' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-11T07:17:24.173 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:17:24 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:17:24.173 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:17:24 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:17:24.173 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:17:24 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:17:24.173 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:17:24 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:17:24.173 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:17:24 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-11T07:17:24.173 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:17:24 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:17:24.173 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:17:24 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:17:24.174 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:17:24 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:17:24.174 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:17:24 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:17:24.174 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:17:24 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/1946670557' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-11T07:17:24.174 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:17:24 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:17:24.174 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:17:24 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:17:24.174 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:17:24 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:17:24.174 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:17:24 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:17:24.174 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:17:24 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-11T07:17:24.174 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:17:24 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:17:24.174 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:17:24 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:17:24.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:17:24 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:17:24.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:17:24 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:17:24.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:17:24 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/1946670557' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-11T07:17:24.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:17:24 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:17:24.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:17:24 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:17:24.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:17:24 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:17:24.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:17:24 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:17:24.536 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:17:24 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-11T07:17:24.536 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:17:24 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:17:24.536 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:17:24 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:17:25.002 INFO:journalctl@ceph.alertmanager.a.smithi071.stdout:Sep 11 07:17:24 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-alertmanager-a[70009]: ts=2024-09-11T07:17:24.889Z caller=cluster.go:706 level=info component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.000516057s 2024-09-11T07:17:25.034 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 11 07:17:24 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-iscsi-iscsi-a[58197]: debug there is no tcmu-runner data available 2024-09-11T07:17:25.291 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:17:25 smithi071 ceph-mon[27013]: from='mon.0 v1:172.21.15.71:0/2658792160' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-11T07:17:25.291 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:17:25 smithi071 ceph-mon[27013]: Deploying daemon grafana.a on smithi196 2024-09-11T07:17:25.292 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:17:25 smithi071 ceph-mon[27013]: pgmap v23: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:17:25.292 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:17:25 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/1183249587' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-11T07:17:25.292 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:17:25 smithi071 ceph-mon[37727]: from='mon.0 v1:172.21.15.71:0/2658792160' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-11T07:17:25.292 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:17:25 smithi071 ceph-mon[37727]: Deploying daemon grafana.a on smithi196 2024-09-11T07:17:25.292 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:17:25 smithi071 ceph-mon[37727]: pgmap v23: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:17:25.292 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:17:25 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/1183249587' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-11T07:17:25.534 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:17:25 smithi196 ceph-mon[30773]: from='mon.0 v1:172.21.15.71:0/2658792160' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-11T07:17:25.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:17:25 smithi196 ceph-mon[30773]: Deploying daemon grafana.a on smithi196 2024-09-11T07:17:25.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:17:25 smithi196 ceph-mon[30773]: pgmap v23: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:17:25.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:17:25 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/1183249587' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-11T07:17:25.559 INFO:tasks.cephadm.ceph_manager.ceph:need seq 206158430223 got 206158430227 for osd.7 2024-09-11T07:17:25.560 DEBUG:teuthology.parallel:result is None 2024-09-11T07:17:25.869 INFO:teuthology.orchestra.run.smithi071.stdout:180388626456 2024-09-11T07:17:26.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:17:26 smithi071 ceph-mon[27013]: from='client.34442 v1:172.21.15.196:0/1952921461' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:17:26.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:17:26 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/2469958568' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-11T07:17:26.089 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:17:26 smithi071 ceph-mon[37727]: from='client.34442 v1:172.21.15.196:0/1952921461' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:17:26.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:17:26 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/2469958568' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-11T07:17:26.160 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182435 got 107374182437 for osd.3 2024-09-11T07:17:26.160 DEBUG:teuthology.parallel:result is None 2024-09-11T07:17:26.534 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:17:26 smithi196 ceph-mon[30773]: from='client.34442 v1:172.21.15.196:0/1952921461' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:17:26.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:17:26 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/2469958568' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-11T07:17:26.935 INFO:teuthology.orchestra.run.smithi071.stdout:128849018914 2024-09-11T07:17:27.034 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 07:17:26 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-x[32609]: ::ffff:172.21.15.196 - - [11/Sep/2024:07:17:26] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:17:27.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:17:27 smithi071 ceph-mon[27013]: pgmap v24: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:17:27.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:17:27 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/3689133235' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-11T07:17:27.089 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:17:27 smithi071 ceph-mon[37727]: pgmap v24: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:17:27.089 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:17:27 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/3689133235' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-11T07:17:27.145 INFO:teuthology.orchestra.run.smithi071.stdout:154618822685 2024-09-11T07:17:27.534 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:17:27 smithi196 ceph-mon[30773]: pgmap v24: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:17:27.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:17:27 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/3689133235' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-11T07:17:27.822 INFO:tasks.cephadm.ceph_manager.ceph:need seq 128849018910 got 128849018914 for osd.4 2024-09-11T07:17:27.822 DEBUG:teuthology.parallel:result is None 2024-09-11T07:17:28.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:17:28 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/1976027520' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-11T07:17:28.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:17:28 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/1976027520' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-11T07:17:28.427 INFO:tasks.cephadm.ceph_manager.ceph:need seq 154618822681 got 154618822685 for osd.5 2024-09-11T07:17:28.427 DEBUG:teuthology.parallel:result is None 2024-09-11T07:17:28.534 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:17:28 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/1976027520' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-11T07:17:28.829 INFO:tasks.cephadm.ceph_manager.ceph:need seq 180388626452 got 180388626456 for osd.6 2024-09-11T07:17:28.829 DEBUG:teuthology.parallel:result is None 2024-09-11T07:17:28.829 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-11T07:17:28.829 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid a81cffe6-700c-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T07:17:29.083 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:17:29.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:17:29 smithi071 ceph-mon[27013]: pgmap v25: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:17:29.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:17:29 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:17:29.339 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:17:29 smithi071 ceph-mon[37727]: pgmap v25: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:17:29.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:17:29 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:17:29.534 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:17:29 smithi196 ceph-mon[30773]: pgmap v25: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:17:29.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:17:29 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:17:31.062 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T07:17:31.064 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2024-09-11T07:17:31.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:17:31 smithi196 ceph-mon[30773]: pgmap v26: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:17:31.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:17:31 smithi071 ceph-mon[27013]: pgmap v26: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:17:31.339 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:17:31 smithi071 ceph-mon[37727]: pgmap v26: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:17:31.797 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2024-09-11T07:17:29.531072+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596188,"num_objects":196,"num_object_clones":0,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":196,"num_whiteouts":0,"num_read":922,"num_read_kb":779,"num_write":543,"num_write_kb":1671,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":548,"ondisk_log_size":548,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":2385388,"kb_used_data":6548,"kb_used_omap":13,"kb_used_meta":2378738,"kb_avail":747575828,"statfs":{"total":767960285184,"available":765517647872,"internally_reserved":0,"allocated":6705152,"data_stored":3631680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2435828416},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002871"},"pg_stats":[{"pgid":"6.1b","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.543739+0000","last_change":"2024-09-11T07:16:15.572595+0000","last_active":"2024-09-11T07:16:23.543739+0000","last_peered":"2024-09-11T07:16:23.543739+0000","last_clean":"2024-09-11T07:16:23.543739+0000","last_became_active":"2024-09-11T07:16:15.572183+0000","last_became_peered":"2024-09-11T07:16:15.572183+0000","last_unstale":"2024-09-11T07:16:23.543739+0000","last_undegraded":"2024-09-11T07:16:23.543739+0000","last_fullsized":"2024-09-11T07:16:23.543739+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:14.001856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:14.001856+0000","last_clean_scrub_stamp":"2024-09-11T07:16:14.001856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:02:23.907135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.696481+0000","last_change":"2024-09-11T07:16:08.985569+0000","last_active":"2024-09-11T07:16:23.696481+0000","last_peered":"2024-09-11T07:16:23.696481+0000","last_clean":"2024-09-11T07:16:23.696481+0000","last_became_active":"2024-09-11T07:16:08.984051+0000","last_became_peered":"2024-09-11T07:16:08.984051+0000","last_unstale":"2024-09-11T07:16:23.696481+0000","last_undegraded":"2024-09-11T07:16:23.696481+0000","last_fullsized":"2024-09-11T07:16:23.696481+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_clean_scrub_stamp":"2024-09-11T07:16:07.961130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:58:34.882566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"59'15","reported_seq":41,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.543865+0000","last_change":"2024-09-11T07:16:10.984677+0000","last_active":"2024-09-11T07:16:23.543865+0000","last_peered":"2024-09-11T07:16:23.543865+0000","last_clean":"2024-09-11T07:16:23.543865+0000","last_became_active":"2024-09-11T07:16:10.984359+0000","last_became_peered":"2024-09-11T07:16:10.984359+0000","last_unstale":"2024-09-11T07:16:23.543865+0000","last_undegraded":"2024-09-11T07:16:23.543865+0000","last_fullsized":"2024-09-11T07:16:23.543865+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:09.964392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:09.964392+0000","last_clean_scrub_stamp":"2024-09-11T07:16:09.964392+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:11:45.801540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.545701+0000","last_change":"2024-09-11T07:16:13.012630+0000","last_active":"2024-09-11T07:16:23.545701+0000","last_peered":"2024-09-11T07:16:23.545701+0000","last_clean":"2024-09-11T07:16:23.545701+0000","last_became_active":"2024-09-11T07:16:13.012330+0000","last_became_peered":"2024-09-11T07:16:13.012330+0000","last_unstale":"2024-09-11T07:16:23.545701+0000","last_undegraded":"2024-09-11T07:16:23.545701+0000","last_fullsized":"2024-09-11T07:16:23.545701+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:11.983377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:11.983377+0000","last_clean_scrub_stamp":"2024-09-11T07:16:11.983377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:42:50.740744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.544732+0000","last_change":"2024-09-11T07:16:08.979458+0000","last_active":"2024-09-11T07:16:23.544732+0000","last_peered":"2024-09-11T07:16:23.544732+0000","last_clean":"2024-09-11T07:16:23.544732+0000","last_became_active":"2024-09-11T07:16:08.979189+0000","last_became_peered":"2024-09-11T07:16:08.979189+0000","last_unstale":"2024-09-11T07:16:23.544732+0000","last_undegraded":"2024-09-11T07:16:23.544732+0000","last_fullsized":"2024-09-11T07:16:23.544732+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_clean_scrub_stamp":"2024-09-11T07:16:07.961130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:30:35.223490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"59'9","reported_seq":32,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.545708+0000","last_change":"2024-09-11T07:16:10.986505+0000","last_active":"2024-09-11T07:16:23.545708+0000","last_peered":"2024-09-11T07:16:23.545708+0000","last_clean":"2024-09-11T07:16:23.545708+0000","last_became_active":"2024-09-11T07:16:10.986133+0000","last_became_peered":"2024-09-11T07:16:10.986133+0000","last_unstale":"2024-09-11T07:16:23.545708+0000","last_undegraded":"2024-09-11T07:16:23.545708+0000","last_fullsized":"2024-09-11T07:16:23.545708+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:09.964392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:09.964392+0000","last_clean_scrub_stamp":"2024-09-11T07:16:09.964392+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:56:35.589886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.679635+0000","last_change":"2024-09-11T07:16:13.011740+0000","last_active":"2024-09-11T07:16:23.679635+0000","last_peered":"2024-09-11T07:16:23.679635+0000","last_clean":"2024-09-11T07:16:23.679635+0000","last_became_active":"2024-09-11T07:16:13.011464+0000","last_became_peered":"2024-09-11T07:16:13.011464+0000","last_unstale":"2024-09-11T07:16:23.679635+0000","last_undegraded":"2024-09-11T07:16:23.679635+0000","last_fullsized":"2024-09-11T07:16:23.679635+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:11.983377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:11.983377+0000","last_clean_scrub_stamp":"2024-09-11T07:16:11.983377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:20:12.606718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.546341+0000","last_change":"2024-09-11T07:16:15.024715+0000","last_active":"2024-09-11T07:16:23.546341+0000","last_peered":"2024-09-11T07:16:23.546341+0000","last_clean":"2024-09-11T07:16:23.546341+0000","last_became_active":"2024-09-11T07:16:15.024374+0000","last_became_peered":"2024-09-11T07:16:15.024374+0000","last_unstale":"2024-09-11T07:16:23.546341+0000","last_undegraded":"2024-09-11T07:16:23.546341+0000","last_fullsized":"2024-09-11T07:16:23.546341+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:14.001856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:14.001856+0000","last_clean_scrub_stamp":"2024-09-11T07:16:14.001856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:09:01.580201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.698547+0000","last_change":"2024-09-11T07:16:08.984792+0000","last_active":"2024-09-11T07:16:23.698547+0000","last_peered":"2024-09-11T07:16:23.698547+0000","last_clean":"2024-09-11T07:16:23.698547+0000","last_became_active":"2024-09-11T07:16:08.984218+0000","last_became_peered":"2024-09-11T07:16:08.984218+0000","last_unstale":"2024-09-11T07:16:23.698547+0000","last_undegraded":"2024-09-11T07:16:23.698547+0000","last_fullsized":"2024-09-11T07:16:23.698547+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_clean_scrub_stamp":"2024-09-11T07:16:07.961130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:58:17.686345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"59'5","reported_seq":26,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.545955+0000","last_change":"2024-09-11T07:16:10.990055+0000","last_active":"2024-09-11T07:16:23.545955+0000","last_peered":"2024-09-11T07:16:23.545955+0000","last_clean":"2024-09-11T07:16:23.545955+0000","last_became_active":"2024-09-11T07:16:10.989767+0000","last_became_peered":"2024-09-11T07:16:10.989767+0000","last_unstale":"2024-09-11T07:16:23.545955+0000","last_undegraded":"2024-09-11T07:16:23.545955+0000","last_fullsized":"2024-09-11T07:16:23.545955+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:09.964392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:09.964392+0000","last_clean_scrub_stamp":"2024-09-11T07:16:09.964392+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:49:30.478807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.697656+0000","last_change":"2024-09-11T07:16:13.008474+0000","last_active":"2024-09-11T07:16:23.697656+0000","last_peered":"2024-09-11T07:16:23.697656+0000","last_clean":"2024-09-11T07:16:23.697656+0000","last_became_active":"2024-09-11T07:16:13.008166+0000","last_became_peered":"2024-09-11T07:16:13.008166+0000","last_unstale":"2024-09-11T07:16:23.697656+0000","last_undegraded":"2024-09-11T07:16:23.697656+0000","last_fullsized":"2024-09-11T07:16:23.697656+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:11.983377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:11.983377+0000","last_clean_scrub_stamp":"2024-09-11T07:16:11.983377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:13:54.682507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.699431+0000","last_change":"2024-09-11T07:16:15.027351+0000","last_active":"2024-09-11T07:16:23.699431+0000","last_peered":"2024-09-11T07:16:23.699431+0000","last_clean":"2024-09-11T07:16:23.699431+0000","last_became_active":"2024-09-11T07:16:15.027004+0000","last_became_peered":"2024-09-11T07:16:15.027004+0000","last_unstale":"2024-09-11T07:16:23.699431+0000","last_undegraded":"2024-09-11T07:16:23.699431+0000","last_fullsized":"2024-09-11T07:16:23.699431+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:14.001856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:14.001856+0000","last_clean_scrub_stamp":"2024-09-11T07:16:14.001856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:22:45.696181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.698382+0000","last_change":"2024-09-11T07:16:08.981918+0000","last_active":"2024-09-11T07:16:23.698382+0000","last_peered":"2024-09-11T07:16:23.698382+0000","last_clean":"2024-09-11T07:16:23.698382+0000","last_became_active":"2024-09-11T07:16:08.981666+0000","last_became_peered":"2024-09-11T07:16:08.981666+0000","last_unstale":"2024-09-11T07:16:23.698382+0000","last_undegraded":"2024-09-11T07:16:23.698382+0000","last_fullsized":"2024-09-11T07:16:23.698382+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_clean_scrub_stamp":"2024-09-11T07:16:07.961130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:10:54.181537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"59'9","reported_seq":32,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.546141+0000","last_change":"2024-09-11T07:16:10.986357+0000","last_active":"2024-09-11T07:16:23.546141+0000","last_peered":"2024-09-11T07:16:23.546141+0000","last_clean":"2024-09-11T07:16:23.546141+0000","last_became_active":"2024-09-11T07:16:10.985996+0000","last_became_peered":"2024-09-11T07:16:10.985996+0000","last_unstale":"2024-09-11T07:16:23.546141+0000","last_undegraded":"2024-09-11T07:16:23.546141+0000","last_fullsized":"2024-09-11T07:16:23.546141+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:09.964392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:09.964392+0000","last_clean_scrub_stamp":"2024-09-11T07:16:09.964392+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:46:08.788947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.699640+0000","last_change":"2024-09-11T07:16:13.010847+0000","last_active":"2024-09-11T07:16:23.699640+0000","last_peered":"2024-09-11T07:16:23.699640+0000","last_clean":"2024-09-11T07:16:23.699640+0000","last_became_active":"2024-09-11T07:16:13.010563+0000","last_became_peered":"2024-09-11T07:16:13.010563+0000","last_unstale":"2024-09-11T07:16:23.699640+0000","last_undegraded":"2024-09-11T07:16:23.699640+0000","last_fullsized":"2024-09-11T07:16:23.699640+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:11.983377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:11.983377+0000","last_clean_scrub_stamp":"2024-09-11T07:16:11.983377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:02:34.695236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.695948+0000","last_change":"2024-09-11T07:16:15.029725+0000","last_active":"2024-09-11T07:16:23.695948+0000","last_peered":"2024-09-11T07:16:23.695948+0000","last_clean":"2024-09-11T07:16:23.695948+0000","last_became_active":"2024-09-11T07:16:15.029081+0000","last_became_peered":"2024-09-11T07:16:15.029081+0000","last_unstale":"2024-09-11T07:16:23.695948+0000","last_undegraded":"2024-09-11T07:16:23.695948+0000","last_fullsized":"2024-09-11T07:16:23.695948+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:14.001856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:14.001856+0000","last_clean_scrub_stamp":"2024-09-11T07:16:14.001856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:40:49.141340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.d","version":"59'17","reported_seq":44,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.546434+0000","last_change":"2024-09-11T07:16:10.990415+0000","last_active":"2024-09-11T07:16:23.546434+0000","last_peered":"2024-09-11T07:16:23.546434+0000","last_clean":"2024-09-11T07:16:23.546434+0000","last_became_active":"2024-09-11T07:16:10.989736+0000","last_became_peered":"2024-09-11T07:16:10.989736+0000","last_unstale":"2024-09-11T07:16:23.546434+0000","last_undegraded":"2024-09-11T07:16:23.546434+0000","last_fullsized":"2024-09-11T07:16:23.546434+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:09.964392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:09.964392+0000","last_clean_scrub_stamp":"2024-09-11T07:16:09.964392+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:01:09.015482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.698120+0000","last_change":"2024-09-11T07:16:08.981326+0000","last_active":"2024-09-11T07:16:23.698120+0000","last_peered":"2024-09-11T07:16:23.698120+0000","last_clean":"2024-09-11T07:16:23.698120+0000","last_became_active":"2024-09-11T07:16:08.981071+0000","last_became_peered":"2024-09-11T07:16:08.981071+0000","last_unstale":"2024-09-11T07:16:23.698120+0000","last_undegraded":"2024-09-11T07:16:23.698120+0000","last_fullsized":"2024-09-11T07:16:23.698120+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_clean_scrub_stamp":"2024-09-11T07:16:07.961130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:04:53.812989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.680070+0000","last_change":"2024-09-11T07:16:13.014168+0000","last_active":"2024-09-11T07:16:23.680070+0000","last_peered":"2024-09-11T07:16:23.680070+0000","last_clean":"2024-09-11T07:16:23.680070+0000","last_became_active":"2024-09-11T07:16:13.013895+0000","last_became_peered":"2024-09-11T07:16:13.013895+0000","last_unstale":"2024-09-11T07:16:23.680070+0000","last_undegraded":"2024-09-11T07:16:23.680070+0000","last_fullsized":"2024-09-11T07:16:23.680070+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:11.983377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:11.983377+0000","last_clean_scrub_stamp":"2024-09-11T07:16:11.983377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:28:12.498619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.546057+0000","last_change":"2024-09-11T07:16:15.020311+0000","last_active":"2024-09-11T07:16:23.546057+0000","last_peered":"2024-09-11T07:16:23.546057+0000","last_clean":"2024-09-11T07:16:23.546057+0000","last_became_active":"2024-09-11T07:16:15.020019+0000","last_became_peered":"2024-09-11T07:16:15.020019+0000","last_unstale":"2024-09-11T07:16:23.546057+0000","last_undegraded":"2024-09-11T07:16:23.546057+0000","last_fullsized":"2024-09-11T07:16:23.546057+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:14.001856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:14.001856+0000","last_clean_scrub_stamp":"2024-09-11T07:16:14.001856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:33:25.419358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"59'10","reported_seq":31,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.546265+0000","last_change":"2024-09-11T07:16:10.984319+0000","last_active":"2024-09-11T07:16:23.546265+0000","last_peered":"2024-09-11T07:16:23.546265+0000","last_clean":"2024-09-11T07:16:23.546265+0000","last_became_active":"2024-09-11T07:16:10.983958+0000","last_became_peered":"2024-09-11T07:16:10.983958+0000","last_unstale":"2024-09-11T07:16:23.546265+0000","last_undegraded":"2024-09-11T07:16:23.546265+0000","last_fullsized":"2024-09-11T07:16:23.546265+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:09.964392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:09.964392+0000","last_clean_scrub_stamp":"2024-09-11T07:16:09.964392+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:21:43.233451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"52'1","reported_seq":25,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.680530+0000","last_change":"2024-09-11T07:16:08.986406+0000","last_active":"2024-09-11T07:16:23.680530+0000","last_peered":"2024-09-11T07:16:23.680530+0000","last_clean":"2024-09-11T07:16:23.680530+0000","last_became_active":"2024-09-11T07:16:08.986138+0000","last_became_peered":"2024-09-11T07:16:08.986138+0000","last_unstale":"2024-09-11T07:16:23.680530+0000","last_undegraded":"2024-09-11T07:16:23.680530+0000","last_fullsized":"2024-09-11T07:16:23.680530+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_clean_scrub_stamp":"2024-09-11T07:16:07.961130+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:27:29.521041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"59'8","reported_seq":37,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:17:17.169448+0000","last_change":"2024-09-11T07:16:13.006522+0000","last_active":"2024-09-11T07:17:17.169448+0000","last_peered":"2024-09-11T07:17:17.169448+0000","last_clean":"2024-09-11T07:17:17.169448+0000","last_became_active":"2024-09-11T07:16:13.006248+0000","last_became_peered":"2024-09-11T07:16:13.006248+0000","last_unstale":"2024-09-11T07:17:17.169448+0000","last_undegraded":"2024-09-11T07:17:17.169448+0000","last_fullsized":"2024-09-11T07:17:17.169448+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:11.983377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:11.983377+0000","last_clean_scrub_stamp":"2024-09-11T07:16:11.983377+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:04:49.074899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.546013+0000","last_change":"2024-09-11T07:16:15.023222+0000","last_active":"2024-09-11T07:16:23.546013+0000","last_peered":"2024-09-11T07:16:23.546013+0000","last_clean":"2024-09-11T07:16:23.546013+0000","last_became_active":"2024-09-11T07:16:15.022930+0000","last_became_peered":"2024-09-11T07:16:15.022930+0000","last_unstale":"2024-09-11T07:16:23.546013+0000","last_undegraded":"2024-09-11T07:16:23.546013+0000","last_fullsized":"2024-09-11T07:16:23.546013+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:14.001856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:14.001856+0000","last_clean_scrub_stamp":"2024-09-11T07:16:14.001856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:10:45.341658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"59'15","reported_seq":41,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.680532+0000","last_change":"2024-09-11T07:16:10.995433+0000","last_active":"2024-09-11T07:16:23.680532+0000","last_peered":"2024-09-11T07:16:23.680532+0000","last_clean":"2024-09-11T07:16:23.680532+0000","last_became_active":"2024-09-11T07:16:10.994998+0000","last_became_peered":"2024-09-11T07:16:10.994998+0000","last_unstale":"2024-09-11T07:16:23.680532+0000","last_undegraded":"2024-09-11T07:16:23.680532+0000","last_fullsized":"2024-09-11T07:16:23.680532+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:09.964392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:09.964392+0000","last_clean_scrub_stamp":"2024-09-11T07:16:09.964392+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:21:17.126355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.679759+0000","last_change":"2024-09-11T07:16:08.985774+0000","last_active":"2024-09-11T07:16:23.679759+0000","last_peered":"2024-09-11T07:16:23.679759+0000","last_clean":"2024-09-11T07:16:23.679759+0000","last_became_active":"2024-09-11T07:16:08.985504+0000","last_became_peered":"2024-09-11T07:16:08.985504+0000","last_unstale":"2024-09-11T07:16:23.679759+0000","last_undegraded":"2024-09-11T07:16:23.679759+0000","last_fullsized":"2024-09-11T07:16:23.679759+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_clean_scrub_stamp":"2024-09-11T07:16:07.961130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:35:01.959999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"59'8","reported_seq":37,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:17:17.169620+0000","last_change":"2024-09-11T07:16:13.014053+0000","last_active":"2024-09-11T07:17:17.169620+0000","last_peered":"2024-09-11T07:17:17.169620+0000","last_clean":"2024-09-11T07:17:17.169620+0000","last_became_active":"2024-09-11T07:16:13.013714+0000","last_became_peered":"2024-09-11T07:16:13.013714+0000","last_unstale":"2024-09-11T07:17:17.169620+0000","last_undegraded":"2024-09-11T07:17:17.169620+0000","last_fullsized":"2024-09-11T07:17:17.169620+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:11.983377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:11.983377+0000","last_clean_scrub_stamp":"2024-09-11T07:16:11.983377+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:16:35.951931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.700500+0000","last_change":"2024-09-11T07:16:15.029085+0000","last_active":"2024-09-11T07:16:23.700500+0000","last_peered":"2024-09-11T07:16:23.700500+0000","last_clean":"2024-09-11T07:16:23.700500+0000","last_became_active":"2024-09-11T07:16:15.028772+0000","last_became_peered":"2024-09-11T07:16:15.028772+0000","last_unstale":"2024-09-11T07:16:23.700500+0000","last_undegraded":"2024-09-11T07:16:23.700500+0000","last_fullsized":"2024-09-11T07:16:23.700500+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:14.001856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:14.001856+0000","last_clean_scrub_stamp":"2024-09-11T07:16:14.001856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:05:48.972728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"59'11","reported_seq":35,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.546278+0000","last_change":"2024-09-11T07:16:10.990461+0000","last_active":"2024-09-11T07:16:23.546278+0000","last_peered":"2024-09-11T07:16:23.546278+0000","last_clean":"2024-09-11T07:16:23.546278+0000","last_became_active":"2024-09-11T07:16:10.989633+0000","last_became_peered":"2024-09-11T07:16:10.989633+0000","last_unstale":"2024-09-11T07:16:23.546278+0000","last_undegraded":"2024-09-11T07:16:23.546278+0000","last_fullsized":"2024-09-11T07:16:23.546278+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:09.964392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:09.964392+0000","last_clean_scrub_stamp":"2024-09-11T07:16:09.964392+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:17:17.718687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.696827+0000","last_change":"2024-09-11T07:16:08.983152+0000","last_active":"2024-09-11T07:16:23.696827+0000","last_peered":"2024-09-11T07:16:23.696827+0000","last_clean":"2024-09-11T07:16:23.696827+0000","last_became_active":"2024-09-11T07:16:08.982544+0000","last_became_peered":"2024-09-11T07:16:08.982544+0000","last_unstale":"2024-09-11T07:16:23.696827+0000","last_undegraded":"2024-09-11T07:16:23.696827+0000","last_fullsized":"2024-09-11T07:16:23.696827+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_clean_scrub_stamp":"2024-09-11T07:16:07.961130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:37:51.386475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.700209+0000","last_change":"2024-09-11T07:16:13.009722+0000","last_active":"2024-09-11T07:16:23.700209+0000","last_peered":"2024-09-11T07:16:23.700209+0000","last_clean":"2024-09-11T07:16:23.700209+0000","last_became_active":"2024-09-11T07:16:13.009403+0000","last_became_peered":"2024-09-11T07:16:13.009403+0000","last_unstale":"2024-09-11T07:16:23.700209+0000","last_undegraded":"2024-09-11T07:16:23.700209+0000","last_fullsized":"2024-09-11T07:16:23.700209+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:11.983377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:11.983377+0000","last_clean_scrub_stamp":"2024-09-11T07:16:11.983377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:53:22.629089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.544196+0000","last_change":"2024-09-11T07:16:15.569670+0000","last_active":"2024-09-11T07:16:23.544196+0000","last_peered":"2024-09-11T07:16:23.544196+0000","last_clean":"2024-09-11T07:16:23.544196+0000","last_became_active":"2024-09-11T07:16:15.569280+0000","last_became_peered":"2024-09-11T07:16:15.569280+0000","last_unstale":"2024-09-11T07:16:23.544196+0000","last_undegraded":"2024-09-11T07:16:23.544196+0000","last_fullsized":"2024-09-11T07:16:23.544196+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:14.001856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:14.001856+0000","last_clean_scrub_stamp":"2024-09-11T07:16:14.001856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:52:12.822802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"59'14","reported_seq":37,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.545851+0000","last_change":"2024-09-11T07:16:10.984372+0000","last_active":"2024-09-11T07:16:23.545851+0000","last_peered":"2024-09-11T07:16:23.545851+0000","last_clean":"2024-09-11T07:16:23.545851+0000","last_became_active":"2024-09-11T07:16:10.983949+0000","last_became_peered":"2024-09-11T07:16:10.983949+0000","last_unstale":"2024-09-11T07:16:23.545851+0000","last_undegraded":"2024-09-11T07:16:23.545851+0000","last_fullsized":"2024-09-11T07:16:23.545851+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:09.964392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:09.964392+0000","last_clean_scrub_stamp":"2024-09-11T07:16:09.964392+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:21:10.318092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.698279+0000","last_change":"2024-09-11T07:16:08.976869+0000","last_active":"2024-09-11T07:16:23.698279+0000","last_peered":"2024-09-11T07:16:23.698279+0000","last_clean":"2024-09-11T07:16:23.698279+0000","last_became_active":"2024-09-11T07:16:08.976593+0000","last_became_peered":"2024-09-11T07:16:08.976593+0000","last_unstale":"2024-09-11T07:16:23.698279+0000","last_undegraded":"2024-09-11T07:16:23.698279+0000","last_fullsized":"2024-09-11T07:16:23.698279+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_clean_scrub_stamp":"2024-09-11T07:16:07.961130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:23:50.495081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"59'8","reported_seq":41,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:17:22.169796+0000","last_change":"2024-09-11T07:16:13.011985+0000","last_active":"2024-09-11T07:17:22.169796+0000","last_peered":"2024-09-11T07:17:22.169796+0000","last_clean":"2024-09-11T07:17:22.169796+0000","last_became_active":"2024-09-11T07:16:13.011691+0000","last_became_peered":"2024-09-11T07:16:13.011691+0000","last_unstale":"2024-09-11T07:17:22.169796+0000","last_undegraded":"2024-09-11T07:17:22.169796+0000","last_fullsized":"2024-09-11T07:17:22.169796+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:11.983377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:11.983377+0000","last_clean_scrub_stamp":"2024-09-11T07:16:11.983377+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:28:31.156724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.696407+0000","last_change":"2024-09-11T07:16:15.571695+0000","last_active":"2024-09-11T07:16:23.696407+0000","last_peered":"2024-09-11T07:16:23.696407+0000","last_clean":"2024-09-11T07:16:23.696407+0000","last_became_active":"2024-09-11T07:16:15.571404+0000","last_became_peered":"2024-09-11T07:16:15.571404+0000","last_unstale":"2024-09-11T07:16:23.696407+0000","last_undegraded":"2024-09-11T07:16:23.696407+0000","last_fullsized":"2024-09-11T07:16:23.696407+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:14.001856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:14.001856+0000","last_clean_scrub_stamp":"2024-09-11T07:16:14.001856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:18:33.861993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.0","version":"59'18","reported_seq":48,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.543655+0000","last_change":"2024-09-11T07:16:11.082566+0000","last_active":"2024-09-11T07:16:23.543655+0000","last_peered":"2024-09-11T07:16:23.543655+0000","last_clean":"2024-09-11T07:16:23.543655+0000","last_became_active":"2024-09-11T07:16:11.082317+0000","last_became_peered":"2024-09-11T07:16:11.082317+0000","last_unstale":"2024-09-11T07:16:23.543655+0000","last_undegraded":"2024-09-11T07:16:23.543655+0000","last_fullsized":"2024-09-11T07:16:23.543655+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:09.964392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:09.964392+0000","last_clean_scrub_stamp":"2024-09-11T07:16:09.964392+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:53:07.769847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.680530+0000","last_change":"2024-09-11T07:16:08.978675+0000","last_active":"2024-09-11T07:16:23.680530+0000","last_peered":"2024-09-11T07:16:23.680530+0000","last_clean":"2024-09-11T07:16:23.680530+0000","last_became_active":"2024-09-11T07:16:08.978387+0000","last_became_peered":"2024-09-11T07:16:08.978387+0000","last_unstale":"2024-09-11T07:16:23.680530+0000","last_undegraded":"2024-09-11T07:16:23.680530+0000","last_fullsized":"2024-09-11T07:16:23.680530+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_clean_scrub_stamp":"2024-09-11T07:16:07.961130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:22:02.061377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.545798+0000","last_change":"2024-09-11T07:16:13.009928+0000","last_active":"2024-09-11T07:16:23.545798+0000","last_peered":"2024-09-11T07:16:23.545798+0000","last_clean":"2024-09-11T07:16:23.545798+0000","last_became_active":"2024-09-11T07:16:13.009629+0000","last_became_peered":"2024-09-11T07:16:13.009629+0000","last_unstale":"2024-09-11T07:16:23.545798+0000","last_undegraded":"2024-09-11T07:16:23.545798+0000","last_fullsized":"2024-09-11T07:16:23.545798+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:11.983377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:11.983377+0000","last_clean_scrub_stamp":"2024-09-11T07:16:11.983377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:11:43.925721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.546063+0000","last_change":"2024-09-11T07:16:15.022809+0000","last_active":"2024-09-11T07:16:23.546063+0000","last_peered":"2024-09-11T07:16:23.546063+0000","last_clean":"2024-09-11T07:16:23.546063+0000","last_became_active":"2024-09-11T07:16:15.022493+0000","last_became_peered":"2024-09-11T07:16:15.022493+0000","last_unstale":"2024-09-11T07:16:23.546063+0000","last_undegraded":"2024-09-11T07:16:23.546063+0000","last_fullsized":"2024-09-11T07:16:23.546063+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:14.001856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:14.001856+0000","last_clean_scrub_stamp":"2024-09-11T07:16:14.001856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:39:11.479184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3","version":"59'19","reported_seq":52,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.696710+0000","last_change":"2024-09-11T07:16:11.081280+0000","last_active":"2024-09-11T07:16:23.696710+0000","last_peered":"2024-09-11T07:16:23.696710+0000","last_clean":"2024-09-11T07:16:23.696710+0000","last_became_active":"2024-09-11T07:16:11.081000+0000","last_became_peered":"2024-09-11T07:16:11.081000+0000","last_unstale":"2024-09-11T07:16:23.696710+0000","last_undegraded":"2024-09-11T07:16:23.696710+0000","last_fullsized":"2024-09-11T07:16:23.696710+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:09.964392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:09.964392+0000","last_clean_scrub_stamp":"2024-09-11T07:16:09.964392+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:10:53.207095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"52'1","reported_seq":27,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.698618+0000","last_change":"2024-09-11T07:16:08.984724+0000","last_active":"2024-09-11T07:16:23.698618+0000","last_peered":"2024-09-11T07:16:23.698618+0000","last_clean":"2024-09-11T07:16:23.698618+0000","last_became_active":"2024-09-11T07:16:08.984252+0000","last_became_peered":"2024-09-11T07:16:08.984252+0000","last_unstale":"2024-09-11T07:16:23.698618+0000","last_undegraded":"2024-09-11T07:16:23.698618+0000","last_fullsized":"2024-09-11T07:16:23.698618+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_clean_scrub_stamp":"2024-09-11T07:16:07.961130+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:42:04.038275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.697707+0000","last_change":"2024-09-11T07:16:13.008303+0000","last_active":"2024-09-11T07:16:23.697707+0000","last_peered":"2024-09-11T07:16:23.697707+0000","last_clean":"2024-09-11T07:16:23.697707+0000","last_became_active":"2024-09-11T07:16:13.007783+0000","last_became_peered":"2024-09-11T07:16:13.007783+0000","last_unstale":"2024-09-11T07:16:23.697707+0000","last_undegraded":"2024-09-11T07:16:23.697707+0000","last_fullsized":"2024-09-11T07:16:23.697707+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:11.983377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:11.983377+0000","last_clean_scrub_stamp":"2024-09-11T07:16:11.983377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:51:32.559781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.679430+0000","last_change":"2024-09-11T07:16:15.569828+0000","last_active":"2024-09-11T07:16:23.679430+0000","last_peered":"2024-09-11T07:16:23.679430+0000","last_clean":"2024-09-11T07:16:23.679430+0000","last_became_active":"2024-09-11T07:16:15.569483+0000","last_became_peered":"2024-09-11T07:16:15.569483+0000","last_unstale":"2024-09-11T07:16:23.679430+0000","last_undegraded":"2024-09-11T07:16:23.679430+0000","last_fullsized":"2024-09-11T07:16:23.679430+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:14.001856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:14.001856+0000","last_clean_scrub_stamp":"2024-09-11T07:16:14.001856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:16:10.795980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"59'10","reported_seq":31,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.680395+0000","last_change":"2024-09-11T07:16:10.992137+0000","last_active":"2024-09-11T07:16:23.680395+0000","last_peered":"2024-09-11T07:16:23.680395+0000","last_clean":"2024-09-11T07:16:23.680395+0000","last_became_active":"2024-09-11T07:16:10.990479+0000","last_became_peered":"2024-09-11T07:16:10.990479+0000","last_unstale":"2024-09-11T07:16:23.680395+0000","last_undegraded":"2024-09-11T07:16:23.680395+0000","last_fullsized":"2024-09-11T07:16:23.680395+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:09.964392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:09.964392+0000","last_clean_scrub_stamp":"2024-09-11T07:16:09.964392+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:06:30.534438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.680171+0000","last_change":"2024-09-11T07:16:08.976919+0000","last_active":"2024-09-11T07:16:23.680171+0000","last_peered":"2024-09-11T07:16:23.680171+0000","last_clean":"2024-09-11T07:16:23.680171+0000","last_became_active":"2024-09-11T07:16:08.976645+0000","last_became_peered":"2024-09-11T07:16:08.976645+0000","last_unstale":"2024-09-11T07:16:23.680171+0000","last_undegraded":"2024-09-11T07:16:23.680171+0000","last_fullsized":"2024-09-11T07:16:23.680171+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_clean_scrub_stamp":"2024-09-11T07:16:07.961130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:08:04.733864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"59'8","reported_seq":38,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:17:22.169801+0000","last_change":"2024-09-11T07:16:13.013368+0000","last_active":"2024-09-11T07:17:22.169801+0000","last_peered":"2024-09-11T07:17:22.169801+0000","last_clean":"2024-09-11T07:17:22.169801+0000","last_became_active":"2024-09-11T07:16:13.012797+0000","last_became_peered":"2024-09-11T07:16:13.012797+0000","last_unstale":"2024-09-11T07:17:22.169801+0000","last_undegraded":"2024-09-11T07:17:22.169801+0000","last_fullsized":"2024-09-11T07:17:22.169801+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:11.983377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:11.983377+0000","last_clean_scrub_stamp":"2024-09-11T07:16:11.983377+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:26:17.888213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.695814+0000","last_change":"2024-09-11T07:16:15.026438+0000","last_active":"2024-09-11T07:16:23.695814+0000","last_peered":"2024-09-11T07:16:23.695814+0000","last_clean":"2024-09-11T07:16:23.695814+0000","last_became_active":"2024-09-11T07:16:15.025971+0000","last_became_peered":"2024-09-11T07:16:15.025971+0000","last_unstale":"2024-09-11T07:16:23.695814+0000","last_undegraded":"2024-09-11T07:16:23.695814+0000","last_fullsized":"2024-09-11T07:16:23.695814+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:14.001856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:14.001856+0000","last_clean_scrub_stamp":"2024-09-11T07:16:14.001856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:39:13.747754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"59'28","reported_seq":79,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:17:17.169460+0000","last_change":"2024-09-11T07:16:10.990091+0000","last_active":"2024-09-11T07:17:17.169460+0000","last_peered":"2024-09-11T07:17:17.169460+0000","last_clean":"2024-09-11T07:17:17.169460+0000","last_became_active":"2024-09-11T07:16:10.989436+0000","last_became_peered":"2024-09-11T07:16:10.989436+0000","last_unstale":"2024-09-11T07:17:17.169460+0000","last_undegraded":"2024-09-11T07:17:17.169460+0000","last_fullsized":"2024-09-11T07:17:17.169460+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:09.964392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:09.964392+0000","last_clean_scrub_stamp":"2024-09-11T07:16:09.964392+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:51:08.310596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.698636+0000","last_change":"2024-09-11T07:16:08.978809+0000","last_active":"2024-09-11T07:16:23.698636+0000","last_peered":"2024-09-11T07:16:23.698636+0000","last_clean":"2024-09-11T07:16:23.698636+0000","last_became_active":"2024-09-11T07:16:08.978528+0000","last_became_peered":"2024-09-11T07:16:08.978528+0000","last_unstale":"2024-09-11T07:16:23.698636+0000","last_undegraded":"2024-09-11T07:16:23.698636+0000","last_fullsized":"2024-09-11T07:16:23.698636+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_clean_scrub_stamp":"2024-09-11T07:16:07.961130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:22:10.777140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.696400+0000","last_change":"2024-09-11T07:16:13.013998+0000","last_active":"2024-09-11T07:16:23.696400+0000","last_peered":"2024-09-11T07:16:23.696400+0000","last_clean":"2024-09-11T07:16:23.696400+0000","last_became_active":"2024-09-11T07:16:13.013744+0000","last_became_peered":"2024-09-11T07:16:13.013744+0000","last_unstale":"2024-09-11T07:16:23.696400+0000","last_undegraded":"2024-09-11T07:16:23.696400+0000","last_fullsized":"2024-09-11T07:16:23.696400+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:11.983377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:11.983377+0000","last_clean_scrub_stamp":"2024-09-11T07:16:11.983377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:19:54.892465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.544634+0000","last_change":"2024-09-11T07:16:15.027135+0000","last_active":"2024-09-11T07:16:23.544634+0000","last_peered":"2024-09-11T07:16:23.544634+0000","last_clean":"2024-09-11T07:16:23.544634+0000","last_became_active":"2024-09-11T07:16:15.026709+0000","last_became_peered":"2024-09-11T07:16:15.026709+0000","last_unstale":"2024-09-11T07:16:23.544634+0000","last_undegraded":"2024-09-11T07:16:23.544634+0000","last_fullsized":"2024-09-11T07:16:23.544634+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:14.001856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:14.001856+0000","last_clean_scrub_stamp":"2024-09-11T07:16:14.001856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:14:42.680069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"59'13","reported_seq":43,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.680558+0000","last_change":"2024-09-11T07:16:10.996497+0000","last_active":"2024-09-11T07:16:23.680558+0000","last_peered":"2024-09-11T07:16:23.680558+0000","last_clean":"2024-09-11T07:16:23.680558+0000","last_became_active":"2024-09-11T07:16:10.996212+0000","last_became_peered":"2024-09-11T07:16:10.996212+0000","last_unstale":"2024-09-11T07:16:23.680558+0000","last_undegraded":"2024-09-11T07:16:23.680558+0000","last_fullsized":"2024-09-11T07:16:23.680558+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:09.964392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:09.964392+0000","last_clean_scrub_stamp":"2024-09-11T07:16:09.964392+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:39:41.676138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"59'5","reported_seq":95,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:17:23.152493+0000","last_change":"2024-09-11T07:16:13.005775+0000","last_active":"2024-09-11T07:17:23.152493+0000","last_peered":"2024-09-11T07:17:23.152493+0000","last_clean":"2024-09-11T07:17:23.152493+0000","last_became_active":"2024-09-11T07:16:08.979258+0000","last_became_peered":"2024-09-11T07:16:08.979258+0000","last_unstale":"2024-09-11T07:17:23.152493+0000","last_undegraded":"2024-09-11T07:17:23.152493+0000","last_fullsized":"2024-09-11T07:17:23.152493+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_clean_scrub_stamp":"2024-09-11T07:16:07.961130+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:38:14.432540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052719500000000005,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":69,"num_read_kb":64,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.545772+0000","last_change":"2024-09-11T07:16:08.975315+0000","last_active":"2024-09-11T07:16:23.545772+0000","last_peered":"2024-09-11T07:16:23.545772+0000","last_clean":"2024-09-11T07:16:23.545772+0000","last_became_active":"2024-09-11T07:16:08.975046+0000","last_became_peered":"2024-09-11T07:16:08.975046+0000","last_unstale":"2024-09-11T07:16:23.545772+0000","last_undegraded":"2024-09-11T07:16:23.545772+0000","last_fullsized":"2024-09-11T07:16:23.545772+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_clean_scrub_stamp":"2024-09-11T07:16:07.961130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:25:50.357403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.546056+0000","last_change":"2024-09-11T07:16:13.005939+0000","last_active":"2024-09-11T07:16:23.546056+0000","last_peered":"2024-09-11T07:16:23.546056+0000","last_clean":"2024-09-11T07:16:23.546056+0000","last_became_active":"2024-09-11T07:16:13.005634+0000","last_became_peered":"2024-09-11T07:16:13.005634+0000","last_unstale":"2024-09-11T07:16:23.546056+0000","last_undegraded":"2024-09-11T07:16:23.546056+0000","last_fullsized":"2024-09-11T07:16:23.546056+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:11.983377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:11.983377+0000","last_clean_scrub_stamp":"2024-09-11T07:16:11.983377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:05:35.892098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.696415+0000","last_change":"2024-09-11T07:16:15.570119+0000","last_active":"2024-09-11T07:16:23.696415+0000","last_peered":"2024-09-11T07:16:23.696415+0000","last_clean":"2024-09-11T07:16:23.696415+0000","last_became_active":"2024-09-11T07:16:15.569653+0000","last_became_peered":"2024-09-11T07:16:15.569653+0000","last_unstale":"2024-09-11T07:16:23.696415+0000","last_undegraded":"2024-09-11T07:16:23.696415+0000","last_fullsized":"2024-09-11T07:16:23.696415+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:14.001856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:14.001856+0000","last_clean_scrub_stamp":"2024-09-11T07:16:14.001856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:39:14.264606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"59'12","reported_seq":34,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.696710+0000","last_change":"2024-09-11T07:16:10.994427+0000","last_active":"2024-09-11T07:16:23.696710+0000","last_peered":"2024-09-11T07:16:23.696710+0000","last_clean":"2024-09-11T07:16:23.696710+0000","last_became_active":"2024-09-11T07:16:10.994116+0000","last_became_peered":"2024-09-11T07:16:10.994116+0000","last_unstale":"2024-09-11T07:16:23.696710+0000","last_undegraded":"2024-09-11T07:16:23.696710+0000","last_fullsized":"2024-09-11T07:16:23.696710+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:09.964392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:09.964392+0000","last_clean_scrub_stamp":"2024-09-11T07:16:09.964392+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:08:16.497313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"54'1","reported_seq":24,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.696327+0000","last_change":"2024-09-11T07:16:13.001573+0000","last_active":"2024-09-11T07:16:23.696327+0000","last_peered":"2024-09-11T07:16:23.696327+0000","last_clean":"2024-09-11T07:16:23.696327+0000","last_became_active":"2024-09-11T07:16:08.984849+0000","last_became_peered":"2024-09-11T07:16:08.984849+0000","last_unstale":"2024-09-11T07:16:23.696327+0000","last_undegraded":"2024-09-11T07:16:23.696327+0000","last_fullsized":"2024-09-11T07:16:23.696327+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_clean_scrub_stamp":"2024-09-11T07:16:07.961130+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:45:24.535362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013614079999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.696896+0000","last_change":"2024-09-11T07:16:08.983320+0000","last_active":"2024-09-11T07:16:23.696896+0000","last_peered":"2024-09-11T07:16:23.696896+0000","last_clean":"2024-09-11T07:16:23.696896+0000","last_became_active":"2024-09-11T07:16:08.982894+0000","last_became_peered":"2024-09-11T07:16:08.982894+0000","last_unstale":"2024-09-11T07:16:23.696896+0000","last_undegraded":"2024-09-11T07:16:23.696896+0000","last_fullsized":"2024-09-11T07:16:23.696896+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_clean_scrub_stamp":"2024-09-11T07:16:07.961130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:08:53.654473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.700279+0000","last_change":"2024-09-11T07:16:13.011556+0000","last_active":"2024-09-11T07:16:23.700279+0000","last_peered":"2024-09-11T07:16:23.700279+0000","last_clean":"2024-09-11T07:16:23.700279+0000","last_became_active":"2024-09-11T07:16:13.011254+0000","last_became_peered":"2024-09-11T07:16:13.011254+0000","last_unstale":"2024-09-11T07:16:23.700279+0000","last_undegraded":"2024-09-11T07:16:23.700279+0000","last_fullsized":"2024-09-11T07:16:23.700279+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:11.983377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:11.983377+0000","last_clean_scrub_stamp":"2024-09-11T07:16:11.983377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:19:49.285338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.679933+0000","last_change":"2024-09-11T07:16:15.026604+0000","last_active":"2024-09-11T07:16:23.679933+0000","last_peered":"2024-09-11T07:16:23.679933+0000","last_clean":"2024-09-11T07:16:23.679933+0000","last_became_active":"2024-09-11T07:16:15.026236+0000","last_became_peered":"2024-09-11T07:16:15.026236+0000","last_unstale":"2024-09-11T07:16:23.679933+0000","last_undegraded":"2024-09-11T07:16:23.679933+0000","last_fullsized":"2024-09-11T07:16:23.679933+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:14.001856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:14.001856+0000","last_clean_scrub_stamp":"2024-09-11T07:16:14.001856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:19:40.999706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"59'14","reported_seq":38,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.697262+0000","last_change":"2024-09-11T07:16:11.082694+0000","last_active":"2024-09-11T07:16:23.697262+0000","last_peered":"2024-09-11T07:16:23.697262+0000","last_clean":"2024-09-11T07:16:23.697262+0000","last_became_active":"2024-09-11T07:16:11.082381+0000","last_became_peered":"2024-09-11T07:16:11.082381+0000","last_unstale":"2024-09-11T07:16:23.697262+0000","last_undegraded":"2024-09-11T07:16:23.697262+0000","last_fullsized":"2024-09-11T07:16:23.697262+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:09.964392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:09.964392+0000","last_clean_scrub_stamp":"2024-09-11T07:16:09.964392+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:34.377717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"56'2","reported_seq":25,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.544056+0000","last_change":"2024-09-11T07:16:12.998901+0000","last_active":"2024-09-11T07:16:23.544056+0000","last_peered":"2024-09-11T07:16:23.544056+0000","last_clean":"2024-09-11T07:16:23.544056+0000","last_became_active":"2024-09-11T07:16:08.982307+0000","last_became_peered":"2024-09-11T07:16:08.982307+0000","last_unstale":"2024-09-11T07:16:23.544056+0000","last_undegraded":"2024-09-11T07:16:23.544056+0000","last_fullsized":"2024-09-11T07:16:23.544056+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_clean_scrub_stamp":"2024-09-11T07:16:07.961130+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:23:08.572218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000888977,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.699879+0000","last_change":"2024-09-11T07:16:08.978137+0000","last_active":"2024-09-11T07:16:23.699879+0000","last_peered":"2024-09-11T07:16:23.699879+0000","last_clean":"2024-09-11T07:16:23.699879+0000","last_became_active":"2024-09-11T07:16:08.977871+0000","last_became_peered":"2024-09-11T07:16:08.977871+0000","last_unstale":"2024-09-11T07:16:23.699879+0000","last_undegraded":"2024-09-11T07:16:23.699879+0000","last_fullsized":"2024-09-11T07:16:23.699879+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_clean_scrub_stamp":"2024-09-11T07:16:07.961130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:45:21.480371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"61'79","reported_seq":52,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:16:45.567631+0000","last_change":"2024-09-11T07:15:58.887909+0000","last_active":"2024-09-11T07:16:45.567631+0000","last_peered":"2024-09-11T07:16:45.567631+0000","last_clean":"2024-09-11T07:16:45.567631+0000","last_became_active":"2024-09-11T07:15:58.879318+0000","last_became_peered":"2024-09-11T07:15:58.879318+0000","last_unstale":"2024-09-11T07:16:45.567631+0000","last_undegraded":"2024-09-11T07:16:45.567631+0000","last_fullsized":"2024-09-11T07:16:45.567631+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:14:04.444392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:14:04.444392+0000","last_clean_scrub_stamp":"2024-09-11T07:14:04.444392+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:26:56.844140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.698195+0000","last_change":"2024-09-11T07:16:13.007044+0000","last_active":"2024-09-11T07:16:23.698195+0000","last_peered":"2024-09-11T07:16:23.698195+0000","last_clean":"2024-09-11T07:16:23.698195+0000","last_became_active":"2024-09-11T07:16:13.006091+0000","last_became_peered":"2024-09-11T07:16:13.006091+0000","last_unstale":"2024-09-11T07:16:23.698195+0000","last_undegraded":"2024-09-11T07:16:23.698195+0000","last_fullsized":"2024-09-11T07:16:23.698195+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:11.983377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:11.983377+0000","last_clean_scrub_stamp":"2024-09-11T07:16:11.983377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:58:22.085406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.700413+0000","last_change":"2024-09-11T07:16:15.027684+0000","last_active":"2024-09-11T07:16:23.700413+0000","last_peered":"2024-09-11T07:16:23.700413+0000","last_clean":"2024-09-11T07:16:23.700413+0000","last_became_active":"2024-09-11T07:16:15.027039+0000","last_became_peered":"2024-09-11T07:16:15.027039+0000","last_unstale":"2024-09-11T07:16:23.700413+0000","last_undegraded":"2024-09-11T07:16:23.700413+0000","last_fullsized":"2024-09-11T07:16:23.700413+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:14.001856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:14.001856+0000","last_clean_scrub_stamp":"2024-09-11T07:16:14.001856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:36:34.430096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"59'19","reported_seq":47,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.697847+0000","last_change":"2024-09-11T07:16:11.081420+0000","last_active":"2024-09-11T07:16:23.697847+0000","last_peered":"2024-09-11T07:16:23.697847+0000","last_clean":"2024-09-11T07:16:23.697847+0000","last_became_active":"2024-09-11T07:16:11.081112+0000","last_became_peered":"2024-09-11T07:16:11.081112+0000","last_unstale":"2024-09-11T07:16:23.697847+0000","last_undegraded":"2024-09-11T07:16:23.697847+0000","last_fullsized":"2024-09-11T07:16:23.697847+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:09.964392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:09.964392+0000","last_clean_scrub_stamp":"2024-09-11T07:16:09.964392+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:53:14.717541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.546253+0000","last_change":"2024-09-11T07:16:08.974450+0000","last_active":"2024-09-11T07:16:23.546253+0000","last_peered":"2024-09-11T07:16:23.546253+0000","last_clean":"2024-09-11T07:16:23.546253+0000","last_became_active":"2024-09-11T07:16:08.974151+0000","last_became_peered":"2024-09-11T07:16:08.974151+0000","last_unstale":"2024-09-11T07:16:23.546253+0000","last_undegraded":"2024-09-11T07:16:23.546253+0000","last_fullsized":"2024-09-11T07:16:23.546253+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_clean_scrub_stamp":"2024-09-11T07:16:07.961130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:40:49.090138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.545847+0000","last_change":"2024-09-11T07:16:13.009433+0000","last_active":"2024-09-11T07:16:23.545847+0000","last_peered":"2024-09-11T07:16:23.545847+0000","last_clean":"2024-09-11T07:16:23.545847+0000","last_became_active":"2024-09-11T07:16:13.008979+0000","last_became_peered":"2024-09-11T07:16:13.008979+0000","last_unstale":"2024-09-11T07:16:23.545847+0000","last_undegraded":"2024-09-11T07:16:23.545847+0000","last_fullsized":"2024-09-11T07:16:23.545847+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:11.983377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:11.983377+0000","last_clean_scrub_stamp":"2024-09-11T07:16:11.983377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:31:35.505296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.696972+0000","last_change":"2024-09-11T07:16:15.032933+0000","last_active":"2024-09-11T07:16:23.696972+0000","last_peered":"2024-09-11T07:16:23.696972+0000","last_clean":"2024-09-11T07:16:23.696972+0000","last_became_active":"2024-09-11T07:16:15.032534+0000","last_became_peered":"2024-09-11T07:16:15.032534+0000","last_unstale":"2024-09-11T07:16:23.696972+0000","last_undegraded":"2024-09-11T07:16:23.696972+0000","last_fullsized":"2024-09-11T07:16:23.696972+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:14.001856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:14.001856+0000","last_clean_scrub_stamp":"2024-09-11T07:16:14.001856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:45:48.995408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"59'9","reported_seq":32,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.696678+0000","last_change":"2024-09-11T07:16:10.995313+0000","last_active":"2024-09-11T07:16:23.696678+0000","last_peered":"2024-09-11T07:16:23.696678+0000","last_clean":"2024-09-11T07:16:23.696678+0000","last_became_active":"2024-09-11T07:16:10.995009+0000","last_became_peered":"2024-09-11T07:16:10.995009+0000","last_unstale":"2024-09-11T07:16:23.696678+0000","last_undegraded":"2024-09-11T07:16:23.696678+0000","last_fullsized":"2024-09-11T07:16:23.696678+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:09.964392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:09.964392+0000","last_clean_scrub_stamp":"2024-09-11T07:16:09.964392+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:52:00.925463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.680498+0000","last_change":"2024-09-11T07:16:08.986579+0000","last_active":"2024-09-11T07:16:23.680498+0000","last_peered":"2024-09-11T07:16:23.680498+0000","last_clean":"2024-09-11T07:16:23.680498+0000","last_became_active":"2024-09-11T07:16:08.986226+0000","last_became_peered":"2024-09-11T07:16:08.986226+0000","last_unstale":"2024-09-11T07:16:23.680498+0000","last_undegraded":"2024-09-11T07:16:23.680498+0000","last_fullsized":"2024-09-11T07:16:23.680498+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_clean_scrub_stamp":"2024-09-11T07:16:07.961130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:56:55.800637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.545850+0000","last_change":"2024-09-11T07:16:13.003534+0000","last_active":"2024-09-11T07:16:23.545850+0000","last_peered":"2024-09-11T07:16:23.545850+0000","last_clean":"2024-09-11T07:16:23.545850+0000","last_became_active":"2024-09-11T07:16:13.003262+0000","last_became_peered":"2024-09-11T07:16:13.003262+0000","last_unstale":"2024-09-11T07:16:23.545850+0000","last_undegraded":"2024-09-11T07:16:23.545850+0000","last_fullsized":"2024-09-11T07:16:23.545850+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:11.983377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:11.983377+0000","last_clean_scrub_stamp":"2024-09-11T07:16:11.983377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:30:35.612516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.696108+0000","last_change":"2024-09-11T07:16:15.028742+0000","last_active":"2024-09-11T07:16:23.696108+0000","last_peered":"2024-09-11T07:16:23.696108+0000","last_clean":"2024-09-11T07:16:23.696108+0000","last_became_active":"2024-09-11T07:16:15.028429+0000","last_became_peered":"2024-09-11T07:16:15.028429+0000","last_unstale":"2024-09-11T07:16:23.696108+0000","last_undegraded":"2024-09-11T07:16:23.696108+0000","last_fullsized":"2024-09-11T07:16:23.696108+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:14.001856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:14.001856+0000","last_clean_scrub_stamp":"2024-09-11T07:16:14.001856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:06:59.941670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"59'15","reported_seq":41,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.699120+0000","last_change":"2024-09-11T07:16:11.081415+0000","last_active":"2024-09-11T07:16:23.699120+0000","last_peered":"2024-09-11T07:16:23.699120+0000","last_clean":"2024-09-11T07:16:23.699120+0000","last_became_active":"2024-09-11T07:16:11.081110+0000","last_became_peered":"2024-09-11T07:16:11.081110+0000","last_unstale":"2024-09-11T07:16:23.699120+0000","last_undegraded":"2024-09-11T07:16:23.699120+0000","last_fullsized":"2024-09-11T07:16:23.699120+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:09.964392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:09.964392+0000","last_clean_scrub_stamp":"2024-09-11T07:16:09.964392+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:59:13.240959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.546157+0000","last_change":"2024-09-11T07:16:08.974091+0000","last_active":"2024-09-11T07:16:23.546157+0000","last_peered":"2024-09-11T07:16:23.546157+0000","last_clean":"2024-09-11T07:16:23.546157+0000","last_became_active":"2024-09-11T07:16:08.973728+0000","last_became_peered":"2024-09-11T07:16:08.973728+0000","last_unstale":"2024-09-11T07:16:23.546157+0000","last_undegraded":"2024-09-11T07:16:23.546157+0000","last_fullsized":"2024-09-11T07:16:23.546157+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_clean_scrub_stamp":"2024-09-11T07:16:07.961130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:35:20.075350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"59'8","reported_seq":38,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:17:22.169881+0000","last_change":"2024-09-11T07:16:13.012746+0000","last_active":"2024-09-11T07:17:22.169881+0000","last_peered":"2024-09-11T07:17:22.169881+0000","last_clean":"2024-09-11T07:17:22.169881+0000","last_became_active":"2024-09-11T07:16:13.012464+0000","last_became_peered":"2024-09-11T07:16:13.012464+0000","last_unstale":"2024-09-11T07:17:22.169881+0000","last_undegraded":"2024-09-11T07:17:22.169881+0000","last_fullsized":"2024-09-11T07:17:22.169881+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:11.983377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:11.983377+0000","last_clean_scrub_stamp":"2024-09-11T07:16:11.983377+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:23:45.643689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.698339+0000","last_change":"2024-09-11T07:16:15.569648+0000","last_active":"2024-09-11T07:16:23.698339+0000","last_peered":"2024-09-11T07:16:23.698339+0000","last_clean":"2024-09-11T07:16:23.698339+0000","last_became_active":"2024-09-11T07:16:15.569333+0000","last_became_peered":"2024-09-11T07:16:15.569333+0000","last_unstale":"2024-09-11T07:16:23.698339+0000","last_undegraded":"2024-09-11T07:16:23.698339+0000","last_fullsized":"2024-09-11T07:16:23.698339+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:14.001856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:14.001856+0000","last_clean_scrub_stamp":"2024-09-11T07:16:14.001856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:33:20.254599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"59'12","reported_seq":39,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.545768+0000","last_change":"2024-09-11T07:16:10.987743+0000","last_active":"2024-09-11T07:16:23.545768+0000","last_peered":"2024-09-11T07:16:23.545768+0000","last_clean":"2024-09-11T07:16:23.545768+0000","last_became_active":"2024-09-11T07:16:10.987313+0000","last_became_peered":"2024-09-11T07:16:10.987313+0000","last_unstale":"2024-09-11T07:16:23.545768+0000","last_undegraded":"2024-09-11T07:16:23.545768+0000","last_fullsized":"2024-09-11T07:16:23.545768+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:09.964392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:09.964392+0000","last_clean_scrub_stamp":"2024-09-11T07:16:09.964392+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:26.793507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"52'2","reported_seq":35,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.546369+0000","last_change":"2024-09-11T07:16:08.974186+0000","last_active":"2024-09-11T07:16:23.546369+0000","last_peered":"2024-09-11T07:16:23.546369+0000","last_clean":"2024-09-11T07:16:23.546369+0000","last_became_active":"2024-09-11T07:16:08.973880+0000","last_became_peered":"2024-09-11T07:16:08.973880+0000","last_unstale":"2024-09-11T07:16:23.546369+0000","last_undegraded":"2024-09-11T07:16:23.546369+0000","last_fullsized":"2024-09-11T07:16:23.546369+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_clean_scrub_stamp":"2024-09-11T07:16:07.961130+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:43:40.564307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.545675+0000","last_change":"2024-09-11T07:16:13.009140+0000","last_active":"2024-09-11T07:16:23.545675+0000","last_peered":"2024-09-11T07:16:23.545675+0000","last_clean":"2024-09-11T07:16:23.545675+0000","last_became_active":"2024-09-11T07:16:13.008879+0000","last_became_peered":"2024-09-11T07:16:13.008879+0000","last_unstale":"2024-09-11T07:16:23.545675+0000","last_undegraded":"2024-09-11T07:16:23.545675+0000","last_fullsized":"2024-09-11T07:16:23.545675+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:11.983377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:11.983377+0000","last_clean_scrub_stamp":"2024-09-11T07:16:11.983377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:34:29.556391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.543894+0000","last_change":"2024-09-11T07:16:15.027987+0000","last_active":"2024-09-11T07:16:23.543894+0000","last_peered":"2024-09-11T07:16:23.543894+0000","last_clean":"2024-09-11T07:16:23.543894+0000","last_became_active":"2024-09-11T07:16:15.027708+0000","last_became_peered":"2024-09-11T07:16:15.027708+0000","last_unstale":"2024-09-11T07:16:23.543894+0000","last_undegraded":"2024-09-11T07:16:23.543894+0000","last_fullsized":"2024-09-11T07:16:23.543894+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:14.001856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:14.001856+0000","last_clean_scrub_stamp":"2024-09-11T07:16:14.001856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:56:24.582132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"59'9","reported_seq":32,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.696616+0000","last_change":"2024-09-11T07:16:11.083008+0000","last_active":"2024-09-11T07:16:23.696616+0000","last_peered":"2024-09-11T07:16:23.696616+0000","last_clean":"2024-09-11T07:16:23.696616+0000","last_became_active":"2024-09-11T07:16:11.082720+0000","last_became_peered":"2024-09-11T07:16:11.082720+0000","last_unstale":"2024-09-11T07:16:23.696616+0000","last_undegraded":"2024-09-11T07:16:23.696616+0000","last_fullsized":"2024-09-11T07:16:23.696616+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:09.964392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:09.964392+0000","last_clean_scrub_stamp":"2024-09-11T07:16:09.964392+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:20:19.732631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.545772+0000","last_change":"2024-09-11T07:16:08.974988+0000","last_active":"2024-09-11T07:16:23.545772+0000","last_peered":"2024-09-11T07:16:23.545772+0000","last_clean":"2024-09-11T07:16:23.545772+0000","last_became_active":"2024-09-11T07:16:08.974744+0000","last_became_peered":"2024-09-11T07:16:08.974744+0000","last_unstale":"2024-09-11T07:16:23.545772+0000","last_undegraded":"2024-09-11T07:16:23.545772+0000","last_fullsized":"2024-09-11T07:16:23.545772+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_clean_scrub_stamp":"2024-09-11T07:16:07.961130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:21:27.696556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.544957+0000","last_change":"2024-09-11T07:16:13.009819+0000","last_active":"2024-09-11T07:16:23.544957+0000","last_peered":"2024-09-11T07:16:23.544957+0000","last_clean":"2024-09-11T07:16:23.544957+0000","last_became_active":"2024-09-11T07:16:13.009408+0000","last_became_peered":"2024-09-11T07:16:13.009408+0000","last_unstale":"2024-09-11T07:16:23.544957+0000","last_undegraded":"2024-09-11T07:16:23.544957+0000","last_fullsized":"2024-09-11T07:16:23.544957+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:11.983377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:11.983377+0000","last_clean_scrub_stamp":"2024-09-11T07:16:11.983377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:09:00.776717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.546034+0000","last_change":"2024-09-11T07:16:15.030338+0000","last_active":"2024-09-11T07:16:23.546034+0000","last_peered":"2024-09-11T07:16:23.546034+0000","last_clean":"2024-09-11T07:16:23.546034+0000","last_became_active":"2024-09-11T07:16:15.030034+0000","last_became_peered":"2024-09-11T07:16:15.030034+0000","last_unstale":"2024-09-11T07:16:23.546034+0000","last_undegraded":"2024-09-11T07:16:23.546034+0000","last_fullsized":"2024-09-11T07:16:23.546034+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:14.001856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:14.001856+0000","last_clean_scrub_stamp":"2024-09-11T07:16:14.001856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:47:43.000121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.698746+0000","last_change":"2024-09-11T07:16:08.975333+0000","last_active":"2024-09-11T07:16:23.698746+0000","last_peered":"2024-09-11T07:16:23.698746+0000","last_clean":"2024-09-11T07:16:23.698746+0000","last_became_active":"2024-09-11T07:16:08.975069+0000","last_became_peered":"2024-09-11T07:16:08.975069+0000","last_unstale":"2024-09-11T07:16:23.698746+0000","last_undegraded":"2024-09-11T07:16:23.698746+0000","last_fullsized":"2024-09-11T07:16:23.698746+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_clean_scrub_stamp":"2024-09-11T07:16:07.961130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:55:47.019869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"59'6","reported_seq":25,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.545046+0000","last_change":"2024-09-11T07:16:11.082609+0000","last_active":"2024-09-11T07:16:23.545046+0000","last_peered":"2024-09-11T07:16:23.545046+0000","last_clean":"2024-09-11T07:16:23.545046+0000","last_became_active":"2024-09-11T07:16:11.082343+0000","last_became_peered":"2024-09-11T07:16:11.082343+0000","last_unstale":"2024-09-11T07:16:23.545046+0000","last_undegraded":"2024-09-11T07:16:23.545046+0000","last_fullsized":"2024-09-11T07:16:23.545046+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:09.964392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:09.964392+0000","last_clean_scrub_stamp":"2024-09-11T07:16:09.964392+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:04:53.558163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.700226+0000","last_change":"2024-09-11T07:16:13.012413+0000","last_active":"2024-09-11T07:16:23.700226+0000","last_peered":"2024-09-11T07:16:23.700226+0000","last_clean":"2024-09-11T07:16:23.700226+0000","last_became_active":"2024-09-11T07:16:13.012058+0000","last_became_peered":"2024-09-11T07:16:13.012058+0000","last_unstale":"2024-09-11T07:16:23.700226+0000","last_undegraded":"2024-09-11T07:16:23.700226+0000","last_fullsized":"2024-09-11T07:16:23.700226+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:11.983377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:11.983377+0000","last_clean_scrub_stamp":"2024-09-11T07:16:11.983377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:56:31.879206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.696413+0000","last_change":"2024-09-11T07:16:15.571203+0000","last_active":"2024-09-11T07:16:23.696413+0000","last_peered":"2024-09-11T07:16:23.696413+0000","last_clean":"2024-09-11T07:16:23.696413+0000","last_became_active":"2024-09-11T07:16:15.570861+0000","last_became_peered":"2024-09-11T07:16:15.570861+0000","last_unstale":"2024-09-11T07:16:23.696413+0000","last_undegraded":"2024-09-11T07:16:23.696413+0000","last_fullsized":"2024-09-11T07:16:23.696413+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:14.001856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:14.001856+0000","last_clean_scrub_stamp":"2024-09-11T07:16:14.001856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:16:54.277674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"59'10","reported_seq":31,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.544451+0000","last_change":"2024-09-11T07:16:11.082543+0000","last_active":"2024-09-11T07:16:23.544451+0000","last_peered":"2024-09-11T07:16:23.544451+0000","last_clean":"2024-09-11T07:16:23.544451+0000","last_became_active":"2024-09-11T07:16:11.082261+0000","last_became_peered":"2024-09-11T07:16:11.082261+0000","last_unstale":"2024-09-11T07:16:23.544451+0000","last_undegraded":"2024-09-11T07:16:23.544451+0000","last_fullsized":"2024-09-11T07:16:23.544451+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:09.964392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:09.964392+0000","last_clean_scrub_stamp":"2024-09-11T07:16:09.964392+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:42:04.524886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.698553+0000","last_change":"2024-09-11T07:16:08.979347+0000","last_active":"2024-09-11T07:16:23.698553+0000","last_peered":"2024-09-11T07:16:23.698553+0000","last_clean":"2024-09-11T07:16:23.698553+0000","last_became_active":"2024-09-11T07:16:08.979081+0000","last_became_peered":"2024-09-11T07:16:08.979081+0000","last_unstale":"2024-09-11T07:16:23.698553+0000","last_undegraded":"2024-09-11T07:16:23.698553+0000","last_fullsized":"2024-09-11T07:16:23.698553+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_clean_scrub_stamp":"2024-09-11T07:16:07.961130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:42:45.466507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"59'8","reported_seq":37,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:17:17.169644+0000","last_change":"2024-09-11T07:16:13.009752+0000","last_active":"2024-09-11T07:17:17.169644+0000","last_peered":"2024-09-11T07:17:17.169644+0000","last_clean":"2024-09-11T07:17:17.169644+0000","last_became_active":"2024-09-11T07:16:13.009466+0000","last_became_peered":"2024-09-11T07:16:13.009466+0000","last_unstale":"2024-09-11T07:17:17.169644+0000","last_undegraded":"2024-09-11T07:17:17.169644+0000","last_fullsized":"2024-09-11T07:17:17.169644+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:11.983377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:11.983377+0000","last_clean_scrub_stamp":"2024-09-11T07:16:11.983377+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:34:56.569719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.696615+0000","last_change":"2024-09-11T07:16:15.030638+0000","last_active":"2024-09-11T07:16:23.696615+0000","last_peered":"2024-09-11T07:16:23.696615+0000","last_clean":"2024-09-11T07:16:23.696615+0000","last_became_active":"2024-09-11T07:16:15.030385+0000","last_became_peered":"2024-09-11T07:16:15.030385+0000","last_unstale":"2024-09-11T07:16:23.696615+0000","last_undegraded":"2024-09-11T07:16:23.696615+0000","last_fullsized":"2024-09-11T07:16:23.696615+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:14.001856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:14.001856+0000","last_clean_scrub_stamp":"2024-09-11T07:16:14.001856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:24:30.631482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"59'9","reported_seq":32,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.700633+0000","last_change":"2024-09-11T07:16:11.083226+0000","last_active":"2024-09-11T07:16:23.700633+0000","last_peered":"2024-09-11T07:16:23.700633+0000","last_clean":"2024-09-11T07:16:23.700633+0000","last_became_active":"2024-09-11T07:16:11.082666+0000","last_became_peered":"2024-09-11T07:16:11.082666+0000","last_unstale":"2024-09-11T07:16:23.700633+0000","last_undegraded":"2024-09-11T07:16:23.700633+0000","last_fullsized":"2024-09-11T07:16:23.700633+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:09.964392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:09.964392+0000","last_clean_scrub_stamp":"2024-09-11T07:16:09.964392+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:02:59.164722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.696780+0000","last_change":"2024-09-11T07:16:08.982466+0000","last_active":"2024-09-11T07:16:23.696780+0000","last_peered":"2024-09-11T07:16:23.696780+0000","last_clean":"2024-09-11T07:16:23.696780+0000","last_became_active":"2024-09-11T07:16:08.981585+0000","last_became_peered":"2024-09-11T07:16:08.981585+0000","last_unstale":"2024-09-11T07:16:23.696780+0000","last_undegraded":"2024-09-11T07:16:23.696780+0000","last_fullsized":"2024-09-11T07:16:23.696780+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_clean_scrub_stamp":"2024-09-11T07:16:07.961130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:42:18.050745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"59'8","reported_seq":38,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:17:22.169892+0000","last_change":"2024-09-11T07:16:13.001869+0000","last_active":"2024-09-11T07:17:22.169892+0000","last_peered":"2024-09-11T07:17:22.169892+0000","last_clean":"2024-09-11T07:17:22.169892+0000","last_became_active":"2024-09-11T07:16:13.001592+0000","last_became_peered":"2024-09-11T07:16:13.001592+0000","last_unstale":"2024-09-11T07:17:22.169892+0000","last_undegraded":"2024-09-11T07:17:22.169892+0000","last_fullsized":"2024-09-11T07:17:22.169892+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:11.983377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:11.983377+0000","last_clean_scrub_stamp":"2024-09-11T07:16:11.983377+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:08:24.689489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.546125+0000","last_change":"2024-09-11T07:16:15.026652+0000","last_active":"2024-09-11T07:16:23.546125+0000","last_peered":"2024-09-11T07:16:23.546125+0000","last_clean":"2024-09-11T07:16:23.546125+0000","last_became_active":"2024-09-11T07:16:15.026319+0000","last_became_peered":"2024-09-11T07:16:15.026319+0000","last_unstale":"2024-09-11T07:16:23.546125+0000","last_undegraded":"2024-09-11T07:16:23.546125+0000","last_fullsized":"2024-09-11T07:16:23.546125+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:14.001856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:14.001856+0000","last_clean_scrub_stamp":"2024-09-11T07:16:14.001856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:21:33.234447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"59'9","reported_seq":32,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.680065+0000","last_change":"2024-09-11T07:16:10.992302+0000","last_active":"2024-09-11T07:16:23.680065+0000","last_peered":"2024-09-11T07:16:23.680065+0000","last_clean":"2024-09-11T07:16:23.680065+0000","last_became_active":"2024-09-11T07:16:10.991836+0000","last_became_peered":"2024-09-11T07:16:10.991836+0000","last_unstale":"2024-09-11T07:16:23.680065+0000","last_undegraded":"2024-09-11T07:16:23.680065+0000","last_fullsized":"2024-09-11T07:16:23.680065+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:09.964392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:09.964392+0000","last_clean_scrub_stamp":"2024-09-11T07:16:09.964392+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:52:51.945977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.697828+0000","last_change":"2024-09-11T07:16:08.973032+0000","last_active":"2024-09-11T07:16:23.697828+0000","last_peered":"2024-09-11T07:16:23.697828+0000","last_clean":"2024-09-11T07:16:23.697828+0000","last_became_active":"2024-09-11T07:16:08.972619+0000","last_became_peered":"2024-09-11T07:16:08.972619+0000","last_unstale":"2024-09-11T07:16:23.697828+0000","last_undegraded":"2024-09-11T07:16:23.697828+0000","last_fullsized":"2024-09-11T07:16:23.697828+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_clean_scrub_stamp":"2024-09-11T07:16:07.961130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:13:56.241952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.543937+0000","last_change":"2024-09-11T07:16:13.009384+0000","last_active":"2024-09-11T07:16:23.543937+0000","last_peered":"2024-09-11T07:16:23.543937+0000","last_clean":"2024-09-11T07:16:23.543937+0000","last_became_active":"2024-09-11T07:16:13.008989+0000","last_became_peered":"2024-09-11T07:16:13.008989+0000","last_unstale":"2024-09-11T07:16:23.543937+0000","last_undegraded":"2024-09-11T07:16:23.543937+0000","last_fullsized":"2024-09-11T07:16:23.543937+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:11.983377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:11.983377+0000","last_clean_scrub_stamp":"2024-09-11T07:16:11.983377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:13:19.486173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.695814+0000","last_change":"2024-09-11T07:16:15.029843+0000","last_active":"2024-09-11T07:16:23.695814+0000","last_peered":"2024-09-11T07:16:23.695814+0000","last_clean":"2024-09-11T07:16:23.695814+0000","last_became_active":"2024-09-11T07:16:15.029462+0000","last_became_peered":"2024-09-11T07:16:15.029462+0000","last_unstale":"2024-09-11T07:16:23.695814+0000","last_undegraded":"2024-09-11T07:16:23.695814+0000","last_fullsized":"2024-09-11T07:16:23.695814+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:14.001856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:14.001856+0000","last_clean_scrub_stamp":"2024-09-11T07:16:14.001856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:37:34.145403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"59'11","reported_seq":35,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.545962+0000","last_change":"2024-09-11T07:16:10.990641+0000","last_active":"2024-09-11T07:16:23.545962+0000","last_peered":"2024-09-11T07:16:23.545962+0000","last_clean":"2024-09-11T07:16:23.545962+0000","last_became_active":"2024-09-11T07:16:10.989609+0000","last_became_peered":"2024-09-11T07:16:10.989609+0000","last_unstale":"2024-09-11T07:16:23.545962+0000","last_undegraded":"2024-09-11T07:16:23.545962+0000","last_fullsized":"2024-09-11T07:16:23.545962+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:09.964392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:09.964392+0000","last_clean_scrub_stamp":"2024-09-11T07:16:09.964392+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:53:16.552881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.680456+0000","last_change":"2024-09-11T07:16:08.977057+0000","last_active":"2024-09-11T07:16:23.680456+0000","last_peered":"2024-09-11T07:16:23.680456+0000","last_clean":"2024-09-11T07:16:23.680456+0000","last_became_active":"2024-09-11T07:16:08.976772+0000","last_became_peered":"2024-09-11T07:16:08.976772+0000","last_unstale":"2024-09-11T07:16:23.680456+0000","last_undegraded":"2024-09-11T07:16:23.680456+0000","last_fullsized":"2024-09-11T07:16:23.680456+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_clean_scrub_stamp":"2024-09-11T07:16:07.961130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:11:52.283361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.679842+0000","last_change":"2024-09-11T07:16:13.011222+0000","last_active":"2024-09-11T07:16:23.679842+0000","last_peered":"2024-09-11T07:16:23.679842+0000","last_clean":"2024-09-11T07:16:23.679842+0000","last_became_active":"2024-09-11T07:16:13.010919+0000","last_became_peered":"2024-09-11T07:16:13.010919+0000","last_unstale":"2024-09-11T07:16:23.679842+0000","last_undegraded":"2024-09-11T07:16:23.679842+0000","last_fullsized":"2024-09-11T07:16:23.679842+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:11.983377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:11.983377+0000","last_clean_scrub_stamp":"2024-09-11T07:16:11.983377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:15:46.736067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.543544+0000","last_change":"2024-09-11T07:16:15.025740+0000","last_active":"2024-09-11T07:16:23.543544+0000","last_peered":"2024-09-11T07:16:23.543544+0000","last_clean":"2024-09-11T07:16:23.543544+0000","last_became_active":"2024-09-11T07:16:15.025394+0000","last_became_peered":"2024-09-11T07:16:15.025394+0000","last_unstale":"2024-09-11T07:16:23.543544+0000","last_undegraded":"2024-09-11T07:16:23.543544+0000","last_fullsized":"2024-09-11T07:16:23.543544+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:14.001856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:14.001856+0000","last_clean_scrub_stamp":"2024-09-11T07:16:14.001856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:27:46.945874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"59'4","reported_seq":22,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.543556+0000","last_change":"2024-09-11T07:16:10.986829+0000","last_active":"2024-09-11T07:16:23.543556+0000","last_peered":"2024-09-11T07:16:23.543556+0000","last_clean":"2024-09-11T07:16:23.543556+0000","last_became_active":"2024-09-11T07:16:10.986556+0000","last_became_peered":"2024-09-11T07:16:10.986556+0000","last_unstale":"2024-09-11T07:16:23.543556+0000","last_undegraded":"2024-09-11T07:16:23.543556+0000","last_fullsized":"2024-09-11T07:16:23.543556+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:09.964392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:09.964392+0000","last_clean_scrub_stamp":"2024-09-11T07:16:09.964392+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:17:15.319325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.700351+0000","last_change":"2024-09-11T07:16:08.976759+0000","last_active":"2024-09-11T07:16:23.700351+0000","last_peered":"2024-09-11T07:16:23.700351+0000","last_clean":"2024-09-11T07:16:23.700351+0000","last_became_active":"2024-09-11T07:16:08.976510+0000","last_became_peered":"2024-09-11T07:16:08.976510+0000","last_unstale":"2024-09-11T07:16:23.700351+0000","last_undegraded":"2024-09-11T07:16:23.700351+0000","last_fullsized":"2024-09-11T07:16:23.700351+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_clean_scrub_stamp":"2024-09-11T07:16:07.961130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:09:47.175924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.698531+0000","last_change":"2024-09-11T07:16:13.015198+0000","last_active":"2024-09-11T07:16:23.698531+0000","last_peered":"2024-09-11T07:16:23.698531+0000","last_clean":"2024-09-11T07:16:23.698531+0000","last_became_active":"2024-09-11T07:16:13.014691+0000","last_became_peered":"2024-09-11T07:16:13.014691+0000","last_unstale":"2024-09-11T07:16:23.698531+0000","last_undegraded":"2024-09-11T07:16:23.698531+0000","last_fullsized":"2024-09-11T07:16:23.698531+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:11.983377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:11.983377+0000","last_clean_scrub_stamp":"2024-09-11T07:16:11.983377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:41:31.302999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"59'1","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.697438+0000","last_change":"2024-09-11T07:16:15.032852+0000","last_active":"2024-09-11T07:16:23.697438+0000","last_peered":"2024-09-11T07:16:23.697438+0000","last_clean":"2024-09-11T07:16:23.697438+0000","last_became_active":"2024-09-11T07:16:15.032569+0000","last_became_peered":"2024-09-11T07:16:15.032569+0000","last_unstale":"2024-09-11T07:16:23.697438+0000","last_undegraded":"2024-09-11T07:16:23.697438+0000","last_fullsized":"2024-09-11T07:16:23.697438+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:14.001856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:14.001856+0000","last_clean_scrub_stamp":"2024-09-11T07:16:14.001856+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:02:54.188336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"59'11","reported_seq":35,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.543700+0000","last_change":"2024-09-11T07:16:11.083002+0000","last_active":"2024-09-11T07:16:23.543700+0000","last_peered":"2024-09-11T07:16:23.543700+0000","last_clean":"2024-09-11T07:16:23.543700+0000","last_became_active":"2024-09-11T07:16:11.082731+0000","last_became_peered":"2024-09-11T07:16:11.082731+0000","last_unstale":"2024-09-11T07:16:23.543700+0000","last_undegraded":"2024-09-11T07:16:23.543700+0000","last_fullsized":"2024-09-11T07:16:23.543700+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:09.964392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:09.964392+0000","last_clean_scrub_stamp":"2024-09-11T07:16:09.964392+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:39:55.411384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.698215+0000","last_change":"2024-09-11T07:16:08.975854+0000","last_active":"2024-09-11T07:16:23.698215+0000","last_peered":"2024-09-11T07:16:23.698215+0000","last_clean":"2024-09-11T07:16:23.698215+0000","last_became_active":"2024-09-11T07:16:08.975563+0000","last_became_peered":"2024-09-11T07:16:08.975563+0000","last_unstale":"2024-09-11T07:16:23.698215+0000","last_undegraded":"2024-09-11T07:16:23.698215+0000","last_fullsized":"2024-09-11T07:16:23.698215+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_clean_scrub_stamp":"2024-09-11T07:16:07.961130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:46:33.738371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.696551+0000","last_change":"2024-09-11T07:16:13.011761+0000","last_active":"2024-09-11T07:16:23.696551+0000","last_peered":"2024-09-11T07:16:23.696551+0000","last_clean":"2024-09-11T07:16:23.696551+0000","last_became_active":"2024-09-11T07:16:13.011468+0000","last_became_peered":"2024-09-11T07:16:13.011468+0000","last_unstale":"2024-09-11T07:16:23.696551+0000","last_undegraded":"2024-09-11T07:16:23.696551+0000","last_fullsized":"2024-09-11T07:16:23.696551+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:11.983377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:11.983377+0000","last_clean_scrub_stamp":"2024-09-11T07:16:11.983377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:52:59.252238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.543795+0000","last_change":"2024-09-11T07:16:15.572137+0000","last_active":"2024-09-11T07:16:23.543795+0000","last_peered":"2024-09-11T07:16:23.543795+0000","last_clean":"2024-09-11T07:16:23.543795+0000","last_became_active":"2024-09-11T07:16:15.571732+0000","last_became_peered":"2024-09-11T07:16:15.571732+0000","last_unstale":"2024-09-11T07:16:23.543795+0000","last_undegraded":"2024-09-11T07:16:23.543795+0000","last_fullsized":"2024-09-11T07:16:23.543795+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:14.001856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:14.001856+0000","last_clean_scrub_stamp":"2024-09-11T07:16:14.001856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:31:03.983228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"59'1","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.698176+0000","last_change":"2024-09-11T07:16:15.031993+0000","last_active":"2024-09-11T07:16:23.698176+0000","last_peered":"2024-09-11T07:16:23.698176+0000","last_clean":"2024-09-11T07:16:23.698176+0000","last_became_active":"2024-09-11T07:16:15.031709+0000","last_became_peered":"2024-09-11T07:16:15.031709+0000","last_unstale":"2024-09-11T07:16:23.698176+0000","last_undegraded":"2024-09-11T07:16:23.698176+0000","last_fullsized":"2024-09-11T07:16:23.698176+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:14.001856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:14.001856+0000","last_clean_scrub_stamp":"2024-09-11T07:16:14.001856+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:01:52.061264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.699216+0000","last_change":"2024-09-11T07:16:08.975465+0000","last_active":"2024-09-11T07:16:23.699216+0000","last_peered":"2024-09-11T07:16:23.699216+0000","last_clean":"2024-09-11T07:16:23.699216+0000","last_became_active":"2024-09-11T07:16:08.975212+0000","last_became_peered":"2024-09-11T07:16:08.975212+0000","last_unstale":"2024-09-11T07:16:23.699216+0000","last_undegraded":"2024-09-11T07:16:23.699216+0000","last_fullsized":"2024-09-11T07:16:23.699216+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_clean_scrub_stamp":"2024-09-11T07:16:07.961130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:35:00.408914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"59'10","reported_seq":31,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.696517+0000","last_change":"2024-09-11T07:16:11.083005+0000","last_active":"2024-09-11T07:16:23.696517+0000","last_peered":"2024-09-11T07:16:23.696517+0000","last_clean":"2024-09-11T07:16:23.696517+0000","last_became_active":"2024-09-11T07:16:11.082373+0000","last_became_peered":"2024-09-11T07:16:11.082373+0000","last_unstale":"2024-09-11T07:16:23.696517+0000","last_undegraded":"2024-09-11T07:16:23.696517+0000","last_fullsized":"2024-09-11T07:16:23.696517+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:09.964392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:09.964392+0000","last_clean_scrub_stamp":"2024-09-11T07:16:09.964392+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:26:15.959149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"59'8","reported_seq":40,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:17:17.169620+0000","last_change":"2024-09-11T07:16:13.013994+0000","last_active":"2024-09-11T07:17:17.169620+0000","last_peered":"2024-09-11T07:17:17.169620+0000","last_clean":"2024-09-11T07:17:17.169620+0000","last_became_active":"2024-09-11T07:16:13.013642+0000","last_became_peered":"2024-09-11T07:16:13.013642+0000","last_unstale":"2024-09-11T07:17:17.169620+0000","last_undegraded":"2024-09-11T07:17:17.169620+0000","last_fullsized":"2024-09-11T07:17:17.169620+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:11.983377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:11.983377+0000","last_clean_scrub_stamp":"2024-09-11T07:16:11.983377+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:29:48.739971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.679987+0000","last_change":"2024-09-11T07:16:15.031643+0000","last_active":"2024-09-11T07:16:23.679987+0000","last_peered":"2024-09-11T07:16:23.679987+0000","last_clean":"2024-09-11T07:16:23.679987+0000","last_became_active":"2024-09-11T07:16:15.031223+0000","last_became_peered":"2024-09-11T07:16:15.031223+0000","last_unstale":"2024-09-11T07:16:23.679987+0000","last_undegraded":"2024-09-11T07:16:23.679987+0000","last_fullsized":"2024-09-11T07:16:23.679987+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:14.001856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:14.001856+0000","last_clean_scrub_stamp":"2024-09-11T07:16:14.001856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:10:01.569732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.543806+0000","last_change":"2024-09-11T07:16:08.983839+0000","last_active":"2024-09-11T07:16:23.543806+0000","last_peered":"2024-09-11T07:16:23.543806+0000","last_clean":"2024-09-11T07:16:23.543806+0000","last_became_active":"2024-09-11T07:16:08.983507+0000","last_became_peered":"2024-09-11T07:16:08.983507+0000","last_unstale":"2024-09-11T07:16:23.543806+0000","last_undegraded":"2024-09-11T07:16:23.543806+0000","last_fullsized":"2024-09-11T07:16:23.543806+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_clean_scrub_stamp":"2024-09-11T07:16:07.961130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:19:31.746340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"59'11","reported_seq":35,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.698137+0000","last_change":"2024-09-11T07:16:10.991086+0000","last_active":"2024-09-11T07:16:23.698137+0000","last_peered":"2024-09-11T07:16:23.698137+0000","last_clean":"2024-09-11T07:16:23.698137+0000","last_became_active":"2024-09-11T07:16:10.990727+0000","last_became_peered":"2024-09-11T07:16:10.990727+0000","last_unstale":"2024-09-11T07:16:23.698137+0000","last_undegraded":"2024-09-11T07:16:23.698137+0000","last_fullsized":"2024-09-11T07:16:23.698137+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:09.964392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:09.964392+0000","last_clean_scrub_stamp":"2024-09-11T07:16:09.964392+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:56:58.109637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.696422+0000","last_change":"2024-09-11T07:16:13.011193+0000","last_active":"2024-09-11T07:16:23.696422+0000","last_peered":"2024-09-11T07:16:23.696422+0000","last_clean":"2024-09-11T07:16:23.696422+0000","last_became_active":"2024-09-11T07:16:13.010864+0000","last_became_peered":"2024-09-11T07:16:13.010864+0000","last_unstale":"2024-09-11T07:16:23.696422+0000","last_undegraded":"2024-09-11T07:16:23.696422+0000","last_fullsized":"2024-09-11T07:16:23.696422+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:11.983377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:11.983377+0000","last_clean_scrub_stamp":"2024-09-11T07:16:11.983377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:01:16.821011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.546098+0000","last_change":"2024-09-11T07:16:15.570339+0000","last_active":"2024-09-11T07:16:23.546098+0000","last_peered":"2024-09-11T07:16:23.546098+0000","last_clean":"2024-09-11T07:16:23.546098+0000","last_became_active":"2024-09-11T07:16:15.570042+0000","last_became_peered":"2024-09-11T07:16:15.570042+0000","last_unstale":"2024-09-11T07:16:23.546098+0000","last_undegraded":"2024-09-11T07:16:23.546098+0000","last_fullsized":"2024-09-11T07:16:23.546098+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:14.001856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:14.001856+0000","last_clean_scrub_stamp":"2024-09-11T07:16:14.001856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:43:13.748820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.697976+0000","last_change":"2024-09-11T07:16:08.975111+0000","last_active":"2024-09-11T07:16:23.697976+0000","last_peered":"2024-09-11T07:16:23.697976+0000","last_clean":"2024-09-11T07:16:23.697976+0000","last_became_active":"2024-09-11T07:16:08.974838+0000","last_became_peered":"2024-09-11T07:16:08.974838+0000","last_unstale":"2024-09-11T07:16:23.697976+0000","last_undegraded":"2024-09-11T07:16:23.697976+0000","last_fullsized":"2024-09-11T07:16:23.697976+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_clean_scrub_stamp":"2024-09-11T07:16:07.961130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:56:43.594262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1c","version":"59'15","reported_seq":41,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.546156+0000","last_change":"2024-09-11T07:16:10.991183+0000","last_active":"2024-09-11T07:16:23.546156+0000","last_peered":"2024-09-11T07:16:23.546156+0000","last_clean":"2024-09-11T07:16:23.546156+0000","last_became_active":"2024-09-11T07:16:10.990850+0000","last_became_peered":"2024-09-11T07:16:10.990850+0000","last_unstale":"2024-09-11T07:16:23.546156+0000","last_undegraded":"2024-09-11T07:16:23.546156+0000","last_fullsized":"2024-09-11T07:16:23.546156+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:09.964392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:09.964392+0000","last_clean_scrub_stamp":"2024-09-11T07:16:09.964392+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:44:28.768280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.680288+0000","last_change":"2024-09-11T07:16:13.014219+0000","last_active":"2024-09-11T07:16:23.680288+0000","last_peered":"2024-09-11T07:16:23.680288+0000","last_clean":"2024-09-11T07:16:23.680288+0000","last_became_active":"2024-09-11T07:16:13.013874+0000","last_became_peered":"2024-09-11T07:16:13.013874+0000","last_unstale":"2024-09-11T07:16:23.680288+0000","last_undegraded":"2024-09-11T07:16:23.680288+0000","last_fullsized":"2024-09-11T07:16:23.680288+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:11.983377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:11.983377+0000","last_clean_scrub_stamp":"2024-09-11T07:16:11.983377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:38:44.453198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.544870+0000","last_change":"2024-09-11T07:16:15.572158+0000","last_active":"2024-09-11T07:16:23.544870+0000","last_peered":"2024-09-11T07:16:23.544870+0000","last_clean":"2024-09-11T07:16:23.544870+0000","last_became_active":"2024-09-11T07:16:15.571809+0000","last_became_peered":"2024-09-11T07:16:15.571809+0000","last_unstale":"2024-09-11T07:16:23.544870+0000","last_undegraded":"2024-09-11T07:16:23.544870+0000","last_fullsized":"2024-09-11T07:16:23.544870+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:14.001856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:14.001856+0000","last_clean_scrub_stamp":"2024-09-11T07:16:14.001856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:50:04.733788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.544146+0000","last_change":"2024-09-11T07:16:08.975787+0000","last_active":"2024-09-11T07:16:23.544146+0000","last_peered":"2024-09-11T07:16:23.544146+0000","last_clean":"2024-09-11T07:16:23.544146+0000","last_became_active":"2024-09-11T07:16:08.975491+0000","last_became_peered":"2024-09-11T07:16:08.975491+0000","last_unstale":"2024-09-11T07:16:23.544146+0000","last_undegraded":"2024-09-11T07:16:23.544146+0000","last_fullsized":"2024-09-11T07:16:23.544146+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_clean_scrub_stamp":"2024-09-11T07:16:07.961130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:06:20.732700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"59'12","reported_seq":39,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.544534+0000","last_change":"2024-09-11T07:16:10.992964+0000","last_active":"2024-09-11T07:16:23.544534+0000","last_peered":"2024-09-11T07:16:23.544534+0000","last_clean":"2024-09-11T07:16:23.544534+0000","last_became_active":"2024-09-11T07:16:10.992693+0000","last_became_peered":"2024-09-11T07:16:10.992693+0000","last_unstale":"2024-09-11T07:16:23.544534+0000","last_undegraded":"2024-09-11T07:16:23.544534+0000","last_fullsized":"2024-09-11T07:16:23.544534+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:09.964392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:09.964392+0000","last_clean_scrub_stamp":"2024-09-11T07:16:09.964392+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:56:02.847562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.546185+0000","last_change":"2024-09-11T07:16:13.006047+0000","last_active":"2024-09-11T07:16:23.546185+0000","last_peered":"2024-09-11T07:16:23.546185+0000","last_clean":"2024-09-11T07:16:23.546185+0000","last_became_active":"2024-09-11T07:16:13.004006+0000","last_became_peered":"2024-09-11T07:16:13.004006+0000","last_unstale":"2024-09-11T07:16:23.546185+0000","last_undegraded":"2024-09-11T07:16:23.546185+0000","last_fullsized":"2024-09-11T07:16:23.546185+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:11.983377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:11.983377+0000","last_clean_scrub_stamp":"2024-09-11T07:16:11.983377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:09:31.446253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":391,"ondisk_log_size":391,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":69,"num_read_kb":64,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1328,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":29,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":48,"seq":206158430229,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298540,"kb_used_data":1188,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446612,"statfs":{"total":95995035648,"available":95689330688,"internally_reserved":0,"allocated":1216512,"data_stored":823879,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:16:59 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.377},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"last":0.49299999999999999}]},{"osd":1,"last update":"Wed Sep 11 07:16:59 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"last":0.40400000000000003},{"interface":"front","average":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":17.690999999999999,"5min":17.690999999999999,"15min":17.690999999999999},"last":0.41099999999999998}]},{"osd":2,"last update":"Wed Sep 11 07:16:59 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.43099999999999999}]},{"osd":3,"last update":"Wed Sep 11 07:16:59 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.624,"5min":0.624,"15min":0.624},"last":0.86399999999999999}]},{"osd":4,"last update":"Wed Sep 11 07:16:59 2024","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.32700000000000001}]},{"osd":5,"last update":"Wed Sep 11 07:16:59 2024","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.62,"5min":0.62,"15min":0.62},"last":0.56499999999999995}]},{"osd":6,"last update":"Wed Sep 11 07:16:59 2024","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.505}]}]},{"osd":6,"up_from":42,"seq":180388626457,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298512,"kb_used_data":1156,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446640,"statfs":{"total":95995035648,"available":95689359360,"internally_reserved":0,"allocated":1183744,"data_stored":821727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:16:34 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.623}]},{"osd":1,"last update":"Wed Sep 11 07:16:34 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.49299999999999999}]},{"osd":2,"last update":"Wed Sep 11 07:16:34 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.66100000000000003}]},{"osd":3,"last update":"Wed Sep 11 07:16:34 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.53800000000000003}]},{"osd":4,"last update":"Wed Sep 11 07:16:34 2024","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.38,"15min":0.38},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"last":0.36699999999999999}]},{"osd":5,"last update":"Wed Sep 11 07:16:34 2024","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"last":0.314},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"last":0.41799999999999998}]},{"osd":7,"last update":"Wed Sep 11 07:16:57 2024","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.33600000000000002}]}]},{"osd":5,"up_from":36,"seq":154618822685,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297936,"kb_used_data":580,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447216,"statfs":{"total":95995035648,"available":95689949184,"internally_reserved":0,"allocated":593920,"data_stored":231610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:17:12 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.52100000000000002,"15min":0.52200000000000002},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.497,"5min":0.52500000000000002,"15min":0.53000000000000003},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.66200000000000003,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.437}]},{"osd":1,"last update":"Wed Sep 11 07:17:12 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.53000000000000003,"15min":0.53300000000000003},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.53400000000000003,"15min":0.53800000000000003},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.70199999999999996,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.50800000000000001}]},{"osd":2,"last update":"Wed Sep 11 07:17:12 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.51600000000000001,"15min":0.51800000000000002},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.72599999999999998,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.56299999999999994,"15min":0.56499999999999995},"min":{"1min":0.39400000000000002,"5min":0.374,"15min":0.374},"max":{"1min":0.69299999999999995,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.38100000000000001}]},{"osd":3,"last update":"Wed Sep 11 07:17:12 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.52300000000000002,"15min":0.52100000000000002},"min":{"1min":0.40899999999999997,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.54700000000000004,"15min":0.54400000000000004},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.45000000000000001}]},{"osd":4,"last update":"Wed Sep 11 07:17:12 2024","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.35599999999999998,"15min":0.34799999999999998},"min":{"1min":0.307,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.38400000000000001,"15min":0.38},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.40100000000000002}]},{"osd":6,"last update":"Wed Sep 11 07:16:35 2024","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.38,"15min":0.38},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"last":0.47399999999999998}]},{"osd":7,"last update":"Wed Sep 11 07:17:00 2024","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.48399999999999999}]}]},{"osd":4,"up_from":30,"seq":128849018914,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297964,"kb_used_data":608,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447188,"statfs":{"total":95995035648,"available":95689920512,"internally_reserved":0,"allocated":622592,"data_stored":233000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:16:50 2024","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.54400000000000004,"15min":0.54900000000000004},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.74299999999999999,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":17.521999999999998},{"interface":"front","average":{"1min":0.503,"5min":0.54000000000000004,"15min":0.54600000000000004},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.71999999999999997,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.60899999999999999}]},{"osd":1,"last update":"Wed Sep 11 07:16:50 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.54200000000000004,"15min":0.55000000000000004},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.70199999999999996,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":17.542000000000002},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.55400000000000005,"15min":0.56000000000000005},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.68000000000000005,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.57499999999999996}]},{"osd":2,"last update":"Wed Sep 11 07:16:50 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.54500000000000004,"15min":0.55300000000000005},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.71399999999999997,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":17.550000000000001},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.53300000000000003,"15min":0.53700000000000003},"min":{"1min":0.36299999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.69799999999999995,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.63500000000000001}]},{"osd":3,"last update":"Wed Sep 11 07:16:50 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.58799999999999997,"15min":0.59799999999999998},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.755,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.58299999999999996,"15min":0.58899999999999997},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.77300000000000002,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":17.481000000000002}]},{"osd":5,"last update":"Wed Sep 11 07:17:13 2024","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.36799999999999999,"15min":0.35899999999999999},"min":{"1min":0.19800000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.437,"5min":0.39100000000000001,"15min":0.38300000000000001},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.55600000000000005}]},{"osd":6,"last update":"Wed Sep 11 07:16:37 2024","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.371,"15min":0.371},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.628,"5min":0.628,"15min":0.628},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"last":0.46700000000000003}]},{"osd":7,"last update":"Wed Sep 11 07:16:57 2024","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.439,"5min":0.439,"15min":0.439},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.59399999999999997}]}]},{"osd":3,"up_from":25,"seq":107374182439,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297976,"kb_used_data":620,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447176,"statfs":{"total":95995035648,"available":95689908224,"internally_reserved":0,"allocated":634880,"data_stored":232826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:16:32 2024","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.39500000000000002,"15min":0.39800000000000002},"min":{"1min":0.27100000000000002,"5min":0.246,"15min":0.246},"max":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.40999999999999998,"15min":0.41099999999999998},"min":{"1min":0.20799999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"last":0.34200000000000003}]},{"osd":1,"last update":"Wed Sep 11 07:16:32 2024","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.40500000000000003,"15min":0.40899999999999997},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":0.51800000000000002,"5min":0.57599999999999996,"15min":0.57599999999999996},"last":0.30599999999999999},{"interface":"front","average":{"1min":0.434,"5min":0.44,"15min":0.441},"min":{"1min":0.312,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"last":0.35699999999999998}]},{"osd":2,"last update":"Wed Sep 11 07:16:32 2024","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.46800000000000003,"15min":0.47199999999999998},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.61499999999999999,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.436,"5min":0.46400000000000002,"15min":0.46800000000000003},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.55400000000000005,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.29299999999999998}]},{"osd":4,"last update":"Wed Sep 11 07:16:50 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.61499999999999999,"15min":0.625},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.68799999999999994,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.63300000000000001,"15min":0.64700000000000002},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.73099999999999998,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.41499999999999998}]},{"osd":5,"last update":"Wed Sep 11 07:17:12 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.624,"15min":0.63400000000000001},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.79800000000000004,"5min":1.075,"15min":1.075},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.61299999999999999,"15min":0.621},"min":{"1min":0.39300000000000002,"5min":0.374,"15min":0.374},"max":{"1min":0.68799999999999994,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.497}]},{"osd":6,"last update":"Wed Sep 11 07:16:36 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.41099999999999998}]},{"osd":7,"last update":"Wed Sep 11 07:16:57 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.53400000000000003}]}]},{"osd":2,"up_from":18,"seq":77309411370,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297940,"kb_used_data":584,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447212,"statfs":{"total":95995035648,"available":95689945088,"internally_reserved":0,"allocated":598016,"data_stored":232116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:17:12 2024","interfaces":[{"interface":"back","average":{"1min":0.32500000000000001,"5min":0.33800000000000002,"15min":0.33600000000000002},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.59099999999999997,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.32800000000000001,"5min":0.34000000000000002,"15min":0.33200000000000002},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.51300000000000001,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.23999999999999999}]},{"osd":1,"last update":"Wed Sep 11 07:17:12 2024","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.35399999999999998,"15min":0.34399999999999997},"min":{"1min":0.20699999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.28499999999999998},{"interface":"front","average":{"1min":0.34899999999999998,"5min":0.36599999999999999,"15min":0.35899999999999999},"min":{"1min":0.26500000000000001,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.57399999999999995,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.29899999999999999}]},{"osd":3,"last update":"Wed Sep 11 07:16:33 2024","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.441,"15min":0.44500000000000001},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.53000000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.27000000000000002},{"interface":"front","average":{"1min":0.40000000000000002,"5min":0.43099999999999999,"15min":0.436},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"last":0.33400000000000002}]},{"osd":4,"last update":"Wed Sep 11 07:16:51 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.56999999999999995,"15min":0.57299999999999995},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.3220000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.56999999999999995,"15min":0.56699999999999995},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":0.46500000000000002}]},{"osd":5,"last update":"Wed Sep 11 07:17:12 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.56899999999999995,"15min":0.57699999999999996},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.56299999999999994,"15min":0.56899999999999995},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.878,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.36899999999999999}]},{"osd":6,"last update":"Wed Sep 11 07:16:34 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.504,"5min":1.504,"15min":1.504},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.40400000000000003}]},{"osd":7,"last update":"Wed Sep 11 07:16:57 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.443},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.248,"5min":1.248,"15min":1.248},"last":0.42999999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574896,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297988,"kb_used_data":632,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447164,"statfs":{"total":95995035648,"available":95689895936,"internally_reserved":0,"allocated":647168,"data_stored":233321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:16:47 2024","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.30499999999999999,"15min":0.27500000000000002},"min":{"1min":0.221,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"last":0.307},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.33700000000000002,"15min":0.307},"min":{"1min":0.27800000000000002,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"last":0.34300000000000003}]},{"osd":2,"last update":"Wed Sep 11 07:17:07 2024","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.38700000000000001,"15min":0.377},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"last":0.27100000000000002},{"interface":"front","average":{"1min":0.39900000000000002,"5min":0.40799999999999997,"15min":0.40400000000000003},"min":{"1min":0.248,"5min":0.245,"15min":0.245},"max":{"1min":0.56100000000000005,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.37}]},{"osd":3,"last update":"Wed Sep 11 07:16:34 2024","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.42099999999999999,"15min":0.41799999999999998},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.433,"15min":0.42899999999999999},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.628,"5min":0.628,"15min":0.628},"last":0.38500000000000001}]},{"osd":4,"last update":"Wed Sep 11 07:16:54 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.59199999999999997,"15min":0.59499999999999997},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.78300000000000003,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.432},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.58399999999999996,"15min":0.59099999999999997},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.70699999999999996,"5min":1.022,"15min":1.022},"last":0.51100000000000001}]},{"osd":5,"last update":"Wed Sep 11 07:17:18 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.628,"15min":0.63200000000000001},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.91000000000000003,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.626,"15min":0.63200000000000001},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.77200000000000002,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.48899999999999999}]},{"osd":6,"last update":"Wed Sep 11 07:16:34 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.4450000000000001,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.41899999999999998}]},{"osd":7,"last update":"Wed Sep 11 07:16:58 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.44700000000000001}]}]},{"osd":0,"up_from":8,"seq":34359738420,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298532,"kb_used_data":1180,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446620,"statfs":{"total":95995035648,"available":95689338880,"internally_reserved":0,"allocated":1208320,"data_stored":823201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 11 07:16:48 2024","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.33800000000000002,"15min":0.32000000000000001},"min":{"1min":0.251,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.57499999999999996,"5min":0.60899999999999999,"15min":0.60899999999999999},"last":0.26200000000000001},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.35099999999999998,"15min":0.32400000000000001},"min":{"1min":0.24299999999999999,"5min":0.215,"15min":0.215},"max":{"1min":0.67900000000000005,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.40699999999999997}]},{"osd":2,"last update":"Wed Sep 11 07:17:07 2024","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.378,"15min":0.36799999999999999},"min":{"1min":0.20699999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.313},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.379,"15min":0.374},"min":{"1min":0.25900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.48499999999999999,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.30199999999999999}]},{"osd":3,"last update":"Wed Sep 11 07:17:28 2024","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.42899999999999999,"15min":0.433},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.61599999999999999,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.29499999999999998},{"interface":"front","average":{"1min":0.37,"5min":0.42999999999999999,"15min":0.436},"min":{"1min":0.28999999999999998,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.501,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.35899999999999999}]},{"osd":4,"last update":"Wed Sep 11 07:16:55 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.624,"15min":0.64000000000000001},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.84699999999999998,"5min":1.048,"15min":1.048},"last":0.436},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.60899999999999999,"15min":0.625},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.70899999999999996,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.48099999999999998}]},{"osd":5,"last update":"Wed Sep 11 07:17:17 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.61399999999999999,"15min":0.623},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.875,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.64300000000000002,"15min":0.65700000000000003},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.78200000000000003,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":0.50700000000000001}]},{"osd":6,"last update":"Wed Sep 11 07:16:36 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.571,"5min":1.571,"15min":1.571},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.391,"5min":1.391,"15min":1.391},"last":0.58899999999999997}]},{"osd":7,"last update":"Wed Sep 11 07:17:00 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.41999999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":2338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T07:17:31.805 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid a81cffe6-700c-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-11T07:17:32.047 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:17:32.284 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:17:32 smithi196 ceph-mon[30773]: from='client.14724 v1:172.21.15.71:0/3253012527' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T07:17:32.330 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:17:32 smithi071 ceph-mon[27013]: from='client.14724 v1:172.21.15.71:0/3253012527' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T07:17:32.330 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:17:32 smithi071 ceph-mon[37727]: from='client.14724 v1:172.21.15.71:0/3253012527' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T07:17:33.047 INFO:journalctl@ceph.alertmanager.a.smithi071.stdout:Sep 11 07:17:32 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-alertmanager-a[70009]: ts=2024-09-11T07:17:32.892Z caller=cluster.go:698 level=info component=cluster msg="gossip settled; proceeding" elapsed=10.004274238s 2024-09-11T07:17:33.284 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:17:33 smithi196 ceph-mon[30773]: pgmap v27: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:17:33.338 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:17:33 smithi071 ceph-mon[27013]: pgmap v27: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:17:33.338 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:17:33 smithi071 ceph-mon[37727]: pgmap v27: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:17:33.698 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T07:17:33.700 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2024-09-11T07:17:33.785 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:33 smithi196 systemd[1]: Starting Ceph grafana.a for a81cffe6-700c-11ef-bcea-c7b262605968... 2024-09-11T07:17:34.104 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:33 smithi196 podman[61619]: 2024-09-11 07:17:33.690693886 +0000 UTC m=+0.014581702 image pull 954c08fa618887fe9a6fe572ae541197c77e625b05f4193df7898f5ac649ada9 quay.io/ceph/ceph-grafana:9.4.7 2024-09-11T07:17:34.104 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:33 smithi196 podman[61619]: 2024-09-11 07:17:33.808658026 +0000 UTC m=+0.132545843 container create 15a51a14b3cc3ab89dbd5c2a734cf136f8c198cc90a4f9ec764fca3f8e0e478b (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a, version=8.5, vendor=Red Hat, Inc., description=Ceph Grafana Container, name=ubi8, io.k8s.display-name=Red Hat Universal Base Image 8, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, io.buildah.version=1.32.0, io.openshift.expose-services=, release=239.1651231664, io.openshift.tags=base rhel8, com.redhat.component=ubi8-container, maintainer=Paul Cuzner , vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, build-date=2022-04-29T11:48:11.055897, summary=Grafana Container configured for Ceph mgr/dashboard integration, architecture=x86_64, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., distribution-scope=public, vcs-type=git, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI) 2024-09-11T07:17:34.104 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 podman[61619]: 2024-09-11 07:17:34.019578954 +0000 UTC m=+0.343466770 container init 15a51a14b3cc3ab89dbd5c2a734cf136f8c198cc90a4f9ec764fca3f8e0e478b (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, io.buildah.version=1.32.0, com.redhat.component=ubi8-container, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, io.openshift.tags=base rhel8, name=ubi8, io.k8s.display-name=Red Hat Universal Base Image 8, maintainer=Paul Cuzner , architecture=x86_64, release=239.1651231664, distribution-scope=public, summary=Grafana Container configured for Ceph mgr/dashboard integration, vendor=Red Hat, Inc., vcs-type=git, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, build-date=2022-04-29T11:48:11.055897, io.openshift.expose-services=, version=8.5, description=Ceph Grafana Container, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56) 2024-09-11T07:17:34.104 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 podman[61619]: 2024-09-11 07:17:34.022659455 +0000 UTC m=+0.346547274 container start 15a51a14b3cc3ab89dbd5c2a734cf136f8c198cc90a4f9ec764fca3f8e0e478b (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a, io.buildah.version=1.32.0, distribution-scope=public, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, io.openshift.tags=base rhel8, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, com.redhat.component=ubi8-container, release=239.1651231664, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, io.k8s.display-name=Red Hat Universal Base Image 8, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, architecture=x86_64, summary=Grafana Container configured for Ceph mgr/dashboard integration, io.openshift.expose-services=, vcs-type=git, build-date=2022-04-29T11:48:11.055897, version=8.5, description=Ceph Grafana Container, vendor=Red Hat, Inc., io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., name=ubi8, maintainer=Paul Cuzner ) 2024-09-11T07:17:34.104 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 bash[61619]: 15a51a14b3cc3ab89dbd5c2a734cf136f8c198cc90a4f9ec764fca3f8e0e478b 2024-09-11T07:17:34.278 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":28,"stamp":"2024-09-11T07:17:33.532035+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596188,"num_objects":196,"num_object_clones":0,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":196,"num_whiteouts":0,"num_read":927,"num_read_kb":784,"num_write":543,"num_write_kb":1671,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":548,"ondisk_log_size":548,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":2385388,"kb_used_data":6548,"kb_used_omap":13,"kb_used_meta":2378738,"kb_avail":747575828,"statfs":{"total":767960285184,"available":765517647872,"internally_reserved":0,"allocated":6705152,"data_stored":3631680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2435828416},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003184"},"pg_stats":[{"pgid":"6.1b","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.543739+0000","last_change":"2024-09-11T07:16:15.572595+0000","last_active":"2024-09-11T07:16:23.543739+0000","last_peered":"2024-09-11T07:16:23.543739+0000","last_clean":"2024-09-11T07:16:23.543739+0000","last_became_active":"2024-09-11T07:16:15.572183+0000","last_became_peered":"2024-09-11T07:16:15.572183+0000","last_unstale":"2024-09-11T07:16:23.543739+0000","last_undegraded":"2024-09-11T07:16:23.543739+0000","last_fullsized":"2024-09-11T07:16:23.543739+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:14.001856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:14.001856+0000","last_clean_scrub_stamp":"2024-09-11T07:16:14.001856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:02:23.907135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.696481+0000","last_change":"2024-09-11T07:16:08.985569+0000","last_active":"2024-09-11T07:16:23.696481+0000","last_peered":"2024-09-11T07:16:23.696481+0000","last_clean":"2024-09-11T07:16:23.696481+0000","last_became_active":"2024-09-11T07:16:08.984051+0000","last_became_peered":"2024-09-11T07:16:08.984051+0000","last_unstale":"2024-09-11T07:16:23.696481+0000","last_undegraded":"2024-09-11T07:16:23.696481+0000","last_fullsized":"2024-09-11T07:16:23.696481+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_clean_scrub_stamp":"2024-09-11T07:16:07.961130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:58:34.882566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"59'15","reported_seq":41,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.543865+0000","last_change":"2024-09-11T07:16:10.984677+0000","last_active":"2024-09-11T07:16:23.543865+0000","last_peered":"2024-09-11T07:16:23.543865+0000","last_clean":"2024-09-11T07:16:23.543865+0000","last_became_active":"2024-09-11T07:16:10.984359+0000","last_became_peered":"2024-09-11T07:16:10.984359+0000","last_unstale":"2024-09-11T07:16:23.543865+0000","last_undegraded":"2024-09-11T07:16:23.543865+0000","last_fullsized":"2024-09-11T07:16:23.543865+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:09.964392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:09.964392+0000","last_clean_scrub_stamp":"2024-09-11T07:16:09.964392+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:11:45.801540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.545701+0000","last_change":"2024-09-11T07:16:13.012630+0000","last_active":"2024-09-11T07:16:23.545701+0000","last_peered":"2024-09-11T07:16:23.545701+0000","last_clean":"2024-09-11T07:16:23.545701+0000","last_became_active":"2024-09-11T07:16:13.012330+0000","last_became_peered":"2024-09-11T07:16:13.012330+0000","last_unstale":"2024-09-11T07:16:23.545701+0000","last_undegraded":"2024-09-11T07:16:23.545701+0000","last_fullsized":"2024-09-11T07:16:23.545701+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:11.983377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:11.983377+0000","last_clean_scrub_stamp":"2024-09-11T07:16:11.983377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:42:50.740744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.544732+0000","last_change":"2024-09-11T07:16:08.979458+0000","last_active":"2024-09-11T07:16:23.544732+0000","last_peered":"2024-09-11T07:16:23.544732+0000","last_clean":"2024-09-11T07:16:23.544732+0000","last_became_active":"2024-09-11T07:16:08.979189+0000","last_became_peered":"2024-09-11T07:16:08.979189+0000","last_unstale":"2024-09-11T07:16:23.544732+0000","last_undegraded":"2024-09-11T07:16:23.544732+0000","last_fullsized":"2024-09-11T07:16:23.544732+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_clean_scrub_stamp":"2024-09-11T07:16:07.961130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:30:35.223490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"59'9","reported_seq":32,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.545708+0000","last_change":"2024-09-11T07:16:10.986505+0000","last_active":"2024-09-11T07:16:23.545708+0000","last_peered":"2024-09-11T07:16:23.545708+0000","last_clean":"2024-09-11T07:16:23.545708+0000","last_became_active":"2024-09-11T07:16:10.986133+0000","last_became_peered":"2024-09-11T07:16:10.986133+0000","last_unstale":"2024-09-11T07:16:23.545708+0000","last_undegraded":"2024-09-11T07:16:23.545708+0000","last_fullsized":"2024-09-11T07:16:23.545708+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:09.964392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:09.964392+0000","last_clean_scrub_stamp":"2024-09-11T07:16:09.964392+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:56:35.589886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.679635+0000","last_change":"2024-09-11T07:16:13.011740+0000","last_active":"2024-09-11T07:16:23.679635+0000","last_peered":"2024-09-11T07:16:23.679635+0000","last_clean":"2024-09-11T07:16:23.679635+0000","last_became_active":"2024-09-11T07:16:13.011464+0000","last_became_peered":"2024-09-11T07:16:13.011464+0000","last_unstale":"2024-09-11T07:16:23.679635+0000","last_undegraded":"2024-09-11T07:16:23.679635+0000","last_fullsized":"2024-09-11T07:16:23.679635+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:11.983377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:11.983377+0000","last_clean_scrub_stamp":"2024-09-11T07:16:11.983377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:20:12.606718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.546341+0000","last_change":"2024-09-11T07:16:15.024715+0000","last_active":"2024-09-11T07:16:23.546341+0000","last_peered":"2024-09-11T07:16:23.546341+0000","last_clean":"2024-09-11T07:16:23.546341+0000","last_became_active":"2024-09-11T07:16:15.024374+0000","last_became_peered":"2024-09-11T07:16:15.024374+0000","last_unstale":"2024-09-11T07:16:23.546341+0000","last_undegraded":"2024-09-11T07:16:23.546341+0000","last_fullsized":"2024-09-11T07:16:23.546341+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:14.001856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:14.001856+0000","last_clean_scrub_stamp":"2024-09-11T07:16:14.001856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:09:01.580201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.698547+0000","last_change":"2024-09-11T07:16:08.984792+0000","last_active":"2024-09-11T07:16:23.698547+0000","last_peered":"2024-09-11T07:16:23.698547+0000","last_clean":"2024-09-11T07:16:23.698547+0000","last_became_active":"2024-09-11T07:16:08.984218+0000","last_became_peered":"2024-09-11T07:16:08.984218+0000","last_unstale":"2024-09-11T07:16:23.698547+0000","last_undegraded":"2024-09-11T07:16:23.698547+0000","last_fullsized":"2024-09-11T07:16:23.698547+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_clean_scrub_stamp":"2024-09-11T07:16:07.961130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:58:17.686345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"59'5","reported_seq":26,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.545955+0000","last_change":"2024-09-11T07:16:10.990055+0000","last_active":"2024-09-11T07:16:23.545955+0000","last_peered":"2024-09-11T07:16:23.545955+0000","last_clean":"2024-09-11T07:16:23.545955+0000","last_became_active":"2024-09-11T07:16:10.989767+0000","last_became_peered":"2024-09-11T07:16:10.989767+0000","last_unstale":"2024-09-11T07:16:23.545955+0000","last_undegraded":"2024-09-11T07:16:23.545955+0000","last_fullsized":"2024-09-11T07:16:23.545955+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:09.964392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:09.964392+0000","last_clean_scrub_stamp":"2024-09-11T07:16:09.964392+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:49:30.478807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.697656+0000","last_change":"2024-09-11T07:16:13.008474+0000","last_active":"2024-09-11T07:16:23.697656+0000","last_peered":"2024-09-11T07:16:23.697656+0000","last_clean":"2024-09-11T07:16:23.697656+0000","last_became_active":"2024-09-11T07:16:13.008166+0000","last_became_peered":"2024-09-11T07:16:13.008166+0000","last_unstale":"2024-09-11T07:16:23.697656+0000","last_undegraded":"2024-09-11T07:16:23.697656+0000","last_fullsized":"2024-09-11T07:16:23.697656+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:11.983377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:11.983377+0000","last_clean_scrub_stamp":"2024-09-11T07:16:11.983377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:13:54.682507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.699431+0000","last_change":"2024-09-11T07:16:15.027351+0000","last_active":"2024-09-11T07:16:23.699431+0000","last_peered":"2024-09-11T07:16:23.699431+0000","last_clean":"2024-09-11T07:16:23.699431+0000","last_became_active":"2024-09-11T07:16:15.027004+0000","last_became_peered":"2024-09-11T07:16:15.027004+0000","last_unstale":"2024-09-11T07:16:23.699431+0000","last_undegraded":"2024-09-11T07:16:23.699431+0000","last_fullsized":"2024-09-11T07:16:23.699431+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:14.001856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:14.001856+0000","last_clean_scrub_stamp":"2024-09-11T07:16:14.001856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:22:45.696181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.698382+0000","last_change":"2024-09-11T07:16:08.981918+0000","last_active":"2024-09-11T07:16:23.698382+0000","last_peered":"2024-09-11T07:16:23.698382+0000","last_clean":"2024-09-11T07:16:23.698382+0000","last_became_active":"2024-09-11T07:16:08.981666+0000","last_became_peered":"2024-09-11T07:16:08.981666+0000","last_unstale":"2024-09-11T07:16:23.698382+0000","last_undegraded":"2024-09-11T07:16:23.698382+0000","last_fullsized":"2024-09-11T07:16:23.698382+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_clean_scrub_stamp":"2024-09-11T07:16:07.961130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:10:54.181537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"59'9","reported_seq":32,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.546141+0000","last_change":"2024-09-11T07:16:10.986357+0000","last_active":"2024-09-11T07:16:23.546141+0000","last_peered":"2024-09-11T07:16:23.546141+0000","last_clean":"2024-09-11T07:16:23.546141+0000","last_became_active":"2024-09-11T07:16:10.985996+0000","last_became_peered":"2024-09-11T07:16:10.985996+0000","last_unstale":"2024-09-11T07:16:23.546141+0000","last_undegraded":"2024-09-11T07:16:23.546141+0000","last_fullsized":"2024-09-11T07:16:23.546141+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:09.964392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:09.964392+0000","last_clean_scrub_stamp":"2024-09-11T07:16:09.964392+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:46:08.788947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.699640+0000","last_change":"2024-09-11T07:16:13.010847+0000","last_active":"2024-09-11T07:16:23.699640+0000","last_peered":"2024-09-11T07:16:23.699640+0000","last_clean":"2024-09-11T07:16:23.699640+0000","last_became_active":"2024-09-11T07:16:13.010563+0000","last_became_peered":"2024-09-11T07:16:13.010563+0000","last_unstale":"2024-09-11T07:16:23.699640+0000","last_undegraded":"2024-09-11T07:16:23.699640+0000","last_fullsized":"2024-09-11T07:16:23.699640+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:11.983377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:11.983377+0000","last_clean_scrub_stamp":"2024-09-11T07:16:11.983377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:02:34.695236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.695948+0000","last_change":"2024-09-11T07:16:15.029725+0000","last_active":"2024-09-11T07:16:23.695948+0000","last_peered":"2024-09-11T07:16:23.695948+0000","last_clean":"2024-09-11T07:16:23.695948+0000","last_became_active":"2024-09-11T07:16:15.029081+0000","last_became_peered":"2024-09-11T07:16:15.029081+0000","last_unstale":"2024-09-11T07:16:23.695948+0000","last_undegraded":"2024-09-11T07:16:23.695948+0000","last_fullsized":"2024-09-11T07:16:23.695948+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:14.001856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:14.001856+0000","last_clean_scrub_stamp":"2024-09-11T07:16:14.001856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:40:49.141340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.d","version":"59'17","reported_seq":44,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.546434+0000","last_change":"2024-09-11T07:16:10.990415+0000","last_active":"2024-09-11T07:16:23.546434+0000","last_peered":"2024-09-11T07:16:23.546434+0000","last_clean":"2024-09-11T07:16:23.546434+0000","last_became_active":"2024-09-11T07:16:10.989736+0000","last_became_peered":"2024-09-11T07:16:10.989736+0000","last_unstale":"2024-09-11T07:16:23.546434+0000","last_undegraded":"2024-09-11T07:16:23.546434+0000","last_fullsized":"2024-09-11T07:16:23.546434+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:09.964392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:09.964392+0000","last_clean_scrub_stamp":"2024-09-11T07:16:09.964392+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:01:09.015482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.698120+0000","last_change":"2024-09-11T07:16:08.981326+0000","last_active":"2024-09-11T07:16:23.698120+0000","last_peered":"2024-09-11T07:16:23.698120+0000","last_clean":"2024-09-11T07:16:23.698120+0000","last_became_active":"2024-09-11T07:16:08.981071+0000","last_became_peered":"2024-09-11T07:16:08.981071+0000","last_unstale":"2024-09-11T07:16:23.698120+0000","last_undegraded":"2024-09-11T07:16:23.698120+0000","last_fullsized":"2024-09-11T07:16:23.698120+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_clean_scrub_stamp":"2024-09-11T07:16:07.961130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:04:53.812989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.680070+0000","last_change":"2024-09-11T07:16:13.014168+0000","last_active":"2024-09-11T07:16:23.680070+0000","last_peered":"2024-09-11T07:16:23.680070+0000","last_clean":"2024-09-11T07:16:23.680070+0000","last_became_active":"2024-09-11T07:16:13.013895+0000","last_became_peered":"2024-09-11T07:16:13.013895+0000","last_unstale":"2024-09-11T07:16:23.680070+0000","last_undegraded":"2024-09-11T07:16:23.680070+0000","last_fullsized":"2024-09-11T07:16:23.680070+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:11.983377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:11.983377+0000","last_clean_scrub_stamp":"2024-09-11T07:16:11.983377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:28:12.498619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.546057+0000","last_change":"2024-09-11T07:16:15.020311+0000","last_active":"2024-09-11T07:16:23.546057+0000","last_peered":"2024-09-11T07:16:23.546057+0000","last_clean":"2024-09-11T07:16:23.546057+0000","last_became_active":"2024-09-11T07:16:15.020019+0000","last_became_peered":"2024-09-11T07:16:15.020019+0000","last_unstale":"2024-09-11T07:16:23.546057+0000","last_undegraded":"2024-09-11T07:16:23.546057+0000","last_fullsized":"2024-09-11T07:16:23.546057+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:14.001856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:14.001856+0000","last_clean_scrub_stamp":"2024-09-11T07:16:14.001856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:33:25.419358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"59'10","reported_seq":31,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.546265+0000","last_change":"2024-09-11T07:16:10.984319+0000","last_active":"2024-09-11T07:16:23.546265+0000","last_peered":"2024-09-11T07:16:23.546265+0000","last_clean":"2024-09-11T07:16:23.546265+0000","last_became_active":"2024-09-11T07:16:10.983958+0000","last_became_peered":"2024-09-11T07:16:10.983958+0000","last_unstale":"2024-09-11T07:16:23.546265+0000","last_undegraded":"2024-09-11T07:16:23.546265+0000","last_fullsized":"2024-09-11T07:16:23.546265+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:09.964392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:09.964392+0000","last_clean_scrub_stamp":"2024-09-11T07:16:09.964392+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:21:43.233451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"52'1","reported_seq":25,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.680530+0000","last_change":"2024-09-11T07:16:08.986406+0000","last_active":"2024-09-11T07:16:23.680530+0000","last_peered":"2024-09-11T07:16:23.680530+0000","last_clean":"2024-09-11T07:16:23.680530+0000","last_became_active":"2024-09-11T07:16:08.986138+0000","last_became_peered":"2024-09-11T07:16:08.986138+0000","last_unstale":"2024-09-11T07:16:23.680530+0000","last_undegraded":"2024-09-11T07:16:23.680530+0000","last_fullsized":"2024-09-11T07:16:23.680530+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_clean_scrub_stamp":"2024-09-11T07:16:07.961130+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:27:29.521041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"59'8","reported_seq":38,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:17:22.169810+0000","last_change":"2024-09-11T07:16:13.006522+0000","last_active":"2024-09-11T07:17:22.169810+0000","last_peered":"2024-09-11T07:17:22.169810+0000","last_clean":"2024-09-11T07:17:22.169810+0000","last_became_active":"2024-09-11T07:16:13.006248+0000","last_became_peered":"2024-09-11T07:16:13.006248+0000","last_unstale":"2024-09-11T07:17:22.169810+0000","last_undegraded":"2024-09-11T07:17:22.169810+0000","last_fullsized":"2024-09-11T07:17:22.169810+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:11.983377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:11.983377+0000","last_clean_scrub_stamp":"2024-09-11T07:16:11.983377+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:04:49.074899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.546013+0000","last_change":"2024-09-11T07:16:15.023222+0000","last_active":"2024-09-11T07:16:23.546013+0000","last_peered":"2024-09-11T07:16:23.546013+0000","last_clean":"2024-09-11T07:16:23.546013+0000","last_became_active":"2024-09-11T07:16:15.022930+0000","last_became_peered":"2024-09-11T07:16:15.022930+0000","last_unstale":"2024-09-11T07:16:23.546013+0000","last_undegraded":"2024-09-11T07:16:23.546013+0000","last_fullsized":"2024-09-11T07:16:23.546013+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:14.001856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:14.001856+0000","last_clean_scrub_stamp":"2024-09-11T07:16:14.001856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:10:45.341658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"59'15","reported_seq":41,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.680532+0000","last_change":"2024-09-11T07:16:10.995433+0000","last_active":"2024-09-11T07:16:23.680532+0000","last_peered":"2024-09-11T07:16:23.680532+0000","last_clean":"2024-09-11T07:16:23.680532+0000","last_became_active":"2024-09-11T07:16:10.994998+0000","last_became_peered":"2024-09-11T07:16:10.994998+0000","last_unstale":"2024-09-11T07:16:23.680532+0000","last_undegraded":"2024-09-11T07:16:23.680532+0000","last_fullsized":"2024-09-11T07:16:23.680532+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:09.964392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:09.964392+0000","last_clean_scrub_stamp":"2024-09-11T07:16:09.964392+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:21:17.126355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.679759+0000","last_change":"2024-09-11T07:16:08.985774+0000","last_active":"2024-09-11T07:16:23.679759+0000","last_peered":"2024-09-11T07:16:23.679759+0000","last_clean":"2024-09-11T07:16:23.679759+0000","last_became_active":"2024-09-11T07:16:08.985504+0000","last_became_peered":"2024-09-11T07:16:08.985504+0000","last_unstale":"2024-09-11T07:16:23.679759+0000","last_undegraded":"2024-09-11T07:16:23.679759+0000","last_fullsized":"2024-09-11T07:16:23.679759+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_clean_scrub_stamp":"2024-09-11T07:16:07.961130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:35:01.959999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"59'8","reported_seq":38,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:17:22.169900+0000","last_change":"2024-09-11T07:16:13.014053+0000","last_active":"2024-09-11T07:17:22.169900+0000","last_peered":"2024-09-11T07:17:22.169900+0000","last_clean":"2024-09-11T07:17:22.169900+0000","last_became_active":"2024-09-11T07:16:13.013714+0000","last_became_peered":"2024-09-11T07:16:13.013714+0000","last_unstale":"2024-09-11T07:17:22.169900+0000","last_undegraded":"2024-09-11T07:17:22.169900+0000","last_fullsized":"2024-09-11T07:17:22.169900+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:11.983377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:11.983377+0000","last_clean_scrub_stamp":"2024-09-11T07:16:11.983377+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:16:35.951931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.700500+0000","last_change":"2024-09-11T07:16:15.029085+0000","last_active":"2024-09-11T07:16:23.700500+0000","last_peered":"2024-09-11T07:16:23.700500+0000","last_clean":"2024-09-11T07:16:23.700500+0000","last_became_active":"2024-09-11T07:16:15.028772+0000","last_became_peered":"2024-09-11T07:16:15.028772+0000","last_unstale":"2024-09-11T07:16:23.700500+0000","last_undegraded":"2024-09-11T07:16:23.700500+0000","last_fullsized":"2024-09-11T07:16:23.700500+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:14.001856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:14.001856+0000","last_clean_scrub_stamp":"2024-09-11T07:16:14.001856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:05:48.972728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"59'11","reported_seq":35,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.546278+0000","last_change":"2024-09-11T07:16:10.990461+0000","last_active":"2024-09-11T07:16:23.546278+0000","last_peered":"2024-09-11T07:16:23.546278+0000","last_clean":"2024-09-11T07:16:23.546278+0000","last_became_active":"2024-09-11T07:16:10.989633+0000","last_became_peered":"2024-09-11T07:16:10.989633+0000","last_unstale":"2024-09-11T07:16:23.546278+0000","last_undegraded":"2024-09-11T07:16:23.546278+0000","last_fullsized":"2024-09-11T07:16:23.546278+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:09.964392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:09.964392+0000","last_clean_scrub_stamp":"2024-09-11T07:16:09.964392+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:17:17.718687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.696827+0000","last_change":"2024-09-11T07:16:08.983152+0000","last_active":"2024-09-11T07:16:23.696827+0000","last_peered":"2024-09-11T07:16:23.696827+0000","last_clean":"2024-09-11T07:16:23.696827+0000","last_became_active":"2024-09-11T07:16:08.982544+0000","last_became_peered":"2024-09-11T07:16:08.982544+0000","last_unstale":"2024-09-11T07:16:23.696827+0000","last_undegraded":"2024-09-11T07:16:23.696827+0000","last_fullsized":"2024-09-11T07:16:23.696827+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_clean_scrub_stamp":"2024-09-11T07:16:07.961130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:37:51.386475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.700209+0000","last_change":"2024-09-11T07:16:13.009722+0000","last_active":"2024-09-11T07:16:23.700209+0000","last_peered":"2024-09-11T07:16:23.700209+0000","last_clean":"2024-09-11T07:16:23.700209+0000","last_became_active":"2024-09-11T07:16:13.009403+0000","last_became_peered":"2024-09-11T07:16:13.009403+0000","last_unstale":"2024-09-11T07:16:23.700209+0000","last_undegraded":"2024-09-11T07:16:23.700209+0000","last_fullsized":"2024-09-11T07:16:23.700209+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:11.983377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:11.983377+0000","last_clean_scrub_stamp":"2024-09-11T07:16:11.983377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:53:22.629089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.544196+0000","last_change":"2024-09-11T07:16:15.569670+0000","last_active":"2024-09-11T07:16:23.544196+0000","last_peered":"2024-09-11T07:16:23.544196+0000","last_clean":"2024-09-11T07:16:23.544196+0000","last_became_active":"2024-09-11T07:16:15.569280+0000","last_became_peered":"2024-09-11T07:16:15.569280+0000","last_unstale":"2024-09-11T07:16:23.544196+0000","last_undegraded":"2024-09-11T07:16:23.544196+0000","last_fullsized":"2024-09-11T07:16:23.544196+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:14.001856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:14.001856+0000","last_clean_scrub_stamp":"2024-09-11T07:16:14.001856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:52:12.822802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"59'14","reported_seq":37,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.545851+0000","last_change":"2024-09-11T07:16:10.984372+0000","last_active":"2024-09-11T07:16:23.545851+0000","last_peered":"2024-09-11T07:16:23.545851+0000","last_clean":"2024-09-11T07:16:23.545851+0000","last_became_active":"2024-09-11T07:16:10.983949+0000","last_became_peered":"2024-09-11T07:16:10.983949+0000","last_unstale":"2024-09-11T07:16:23.545851+0000","last_undegraded":"2024-09-11T07:16:23.545851+0000","last_fullsized":"2024-09-11T07:16:23.545851+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:09.964392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:09.964392+0000","last_clean_scrub_stamp":"2024-09-11T07:16:09.964392+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:21:10.318092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.698279+0000","last_change":"2024-09-11T07:16:08.976869+0000","last_active":"2024-09-11T07:16:23.698279+0000","last_peered":"2024-09-11T07:16:23.698279+0000","last_clean":"2024-09-11T07:16:23.698279+0000","last_became_active":"2024-09-11T07:16:08.976593+0000","last_became_peered":"2024-09-11T07:16:08.976593+0000","last_unstale":"2024-09-11T07:16:23.698279+0000","last_undegraded":"2024-09-11T07:16:23.698279+0000","last_fullsized":"2024-09-11T07:16:23.698279+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_clean_scrub_stamp":"2024-09-11T07:16:07.961130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:23:50.495081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"59'8","reported_seq":41,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:17:22.169796+0000","last_change":"2024-09-11T07:16:13.011985+0000","last_active":"2024-09-11T07:17:22.169796+0000","last_peered":"2024-09-11T07:17:22.169796+0000","last_clean":"2024-09-11T07:17:22.169796+0000","last_became_active":"2024-09-11T07:16:13.011691+0000","last_became_peered":"2024-09-11T07:16:13.011691+0000","last_unstale":"2024-09-11T07:17:22.169796+0000","last_undegraded":"2024-09-11T07:17:22.169796+0000","last_fullsized":"2024-09-11T07:17:22.169796+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:11.983377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:11.983377+0000","last_clean_scrub_stamp":"2024-09-11T07:16:11.983377+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:28:31.156724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.696407+0000","last_change":"2024-09-11T07:16:15.571695+0000","last_active":"2024-09-11T07:16:23.696407+0000","last_peered":"2024-09-11T07:16:23.696407+0000","last_clean":"2024-09-11T07:16:23.696407+0000","last_became_active":"2024-09-11T07:16:15.571404+0000","last_became_peered":"2024-09-11T07:16:15.571404+0000","last_unstale":"2024-09-11T07:16:23.696407+0000","last_undegraded":"2024-09-11T07:16:23.696407+0000","last_fullsized":"2024-09-11T07:16:23.696407+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:14.001856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:14.001856+0000","last_clean_scrub_stamp":"2024-09-11T07:16:14.001856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:18:33.861993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.0","version":"59'18","reported_seq":48,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.543655+0000","last_change":"2024-09-11T07:16:11.082566+0000","last_active":"2024-09-11T07:16:23.543655+0000","last_peered":"2024-09-11T07:16:23.543655+0000","last_clean":"2024-09-11T07:16:23.543655+0000","last_became_active":"2024-09-11T07:16:11.082317+0000","last_became_peered":"2024-09-11T07:16:11.082317+0000","last_unstale":"2024-09-11T07:16:23.543655+0000","last_undegraded":"2024-09-11T07:16:23.543655+0000","last_fullsized":"2024-09-11T07:16:23.543655+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:09.964392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:09.964392+0000","last_clean_scrub_stamp":"2024-09-11T07:16:09.964392+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:53:07.769847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.680530+0000","last_change":"2024-09-11T07:16:08.978675+0000","last_active":"2024-09-11T07:16:23.680530+0000","last_peered":"2024-09-11T07:16:23.680530+0000","last_clean":"2024-09-11T07:16:23.680530+0000","last_became_active":"2024-09-11T07:16:08.978387+0000","last_became_peered":"2024-09-11T07:16:08.978387+0000","last_unstale":"2024-09-11T07:16:23.680530+0000","last_undegraded":"2024-09-11T07:16:23.680530+0000","last_fullsized":"2024-09-11T07:16:23.680530+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_clean_scrub_stamp":"2024-09-11T07:16:07.961130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:22:02.061377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.545798+0000","last_change":"2024-09-11T07:16:13.009928+0000","last_active":"2024-09-11T07:16:23.545798+0000","last_peered":"2024-09-11T07:16:23.545798+0000","last_clean":"2024-09-11T07:16:23.545798+0000","last_became_active":"2024-09-11T07:16:13.009629+0000","last_became_peered":"2024-09-11T07:16:13.009629+0000","last_unstale":"2024-09-11T07:16:23.545798+0000","last_undegraded":"2024-09-11T07:16:23.545798+0000","last_fullsized":"2024-09-11T07:16:23.545798+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:11.983377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:11.983377+0000","last_clean_scrub_stamp":"2024-09-11T07:16:11.983377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:11:43.925721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.546063+0000","last_change":"2024-09-11T07:16:15.022809+0000","last_active":"2024-09-11T07:16:23.546063+0000","last_peered":"2024-09-11T07:16:23.546063+0000","last_clean":"2024-09-11T07:16:23.546063+0000","last_became_active":"2024-09-11T07:16:15.022493+0000","last_became_peered":"2024-09-11T07:16:15.022493+0000","last_unstale":"2024-09-11T07:16:23.546063+0000","last_undegraded":"2024-09-11T07:16:23.546063+0000","last_fullsized":"2024-09-11T07:16:23.546063+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:14.001856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:14.001856+0000","last_clean_scrub_stamp":"2024-09-11T07:16:14.001856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:39:11.479184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3","version":"59'19","reported_seq":52,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.696710+0000","last_change":"2024-09-11T07:16:11.081280+0000","last_active":"2024-09-11T07:16:23.696710+0000","last_peered":"2024-09-11T07:16:23.696710+0000","last_clean":"2024-09-11T07:16:23.696710+0000","last_became_active":"2024-09-11T07:16:11.081000+0000","last_became_peered":"2024-09-11T07:16:11.081000+0000","last_unstale":"2024-09-11T07:16:23.696710+0000","last_undegraded":"2024-09-11T07:16:23.696710+0000","last_fullsized":"2024-09-11T07:16:23.696710+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:09.964392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:09.964392+0000","last_clean_scrub_stamp":"2024-09-11T07:16:09.964392+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:10:53.207095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"52'1","reported_seq":27,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.698618+0000","last_change":"2024-09-11T07:16:08.984724+0000","last_active":"2024-09-11T07:16:23.698618+0000","last_peered":"2024-09-11T07:16:23.698618+0000","last_clean":"2024-09-11T07:16:23.698618+0000","last_became_active":"2024-09-11T07:16:08.984252+0000","last_became_peered":"2024-09-11T07:16:08.984252+0000","last_unstale":"2024-09-11T07:16:23.698618+0000","last_undegraded":"2024-09-11T07:16:23.698618+0000","last_fullsized":"2024-09-11T07:16:23.698618+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_clean_scrub_stamp":"2024-09-11T07:16:07.961130+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:42:04.038275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.697707+0000","last_change":"2024-09-11T07:16:13.008303+0000","last_active":"2024-09-11T07:16:23.697707+0000","last_peered":"2024-09-11T07:16:23.697707+0000","last_clean":"2024-09-11T07:16:23.697707+0000","last_became_active":"2024-09-11T07:16:13.007783+0000","last_became_peered":"2024-09-11T07:16:13.007783+0000","last_unstale":"2024-09-11T07:16:23.697707+0000","last_undegraded":"2024-09-11T07:16:23.697707+0000","last_fullsized":"2024-09-11T07:16:23.697707+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:11.983377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:11.983377+0000","last_clean_scrub_stamp":"2024-09-11T07:16:11.983377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:51:32.559781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.679430+0000","last_change":"2024-09-11T07:16:15.569828+0000","last_active":"2024-09-11T07:16:23.679430+0000","last_peered":"2024-09-11T07:16:23.679430+0000","last_clean":"2024-09-11T07:16:23.679430+0000","last_became_active":"2024-09-11T07:16:15.569483+0000","last_became_peered":"2024-09-11T07:16:15.569483+0000","last_unstale":"2024-09-11T07:16:23.679430+0000","last_undegraded":"2024-09-11T07:16:23.679430+0000","last_fullsized":"2024-09-11T07:16:23.679430+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:14.001856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:14.001856+0000","last_clean_scrub_stamp":"2024-09-11T07:16:14.001856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:16:10.795980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"59'10","reported_seq":31,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.680395+0000","last_change":"2024-09-11T07:16:10.992137+0000","last_active":"2024-09-11T07:16:23.680395+0000","last_peered":"2024-09-11T07:16:23.680395+0000","last_clean":"2024-09-11T07:16:23.680395+0000","last_became_active":"2024-09-11T07:16:10.990479+0000","last_became_peered":"2024-09-11T07:16:10.990479+0000","last_unstale":"2024-09-11T07:16:23.680395+0000","last_undegraded":"2024-09-11T07:16:23.680395+0000","last_fullsized":"2024-09-11T07:16:23.680395+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:09.964392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:09.964392+0000","last_clean_scrub_stamp":"2024-09-11T07:16:09.964392+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:06:30.534438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.680171+0000","last_change":"2024-09-11T07:16:08.976919+0000","last_active":"2024-09-11T07:16:23.680171+0000","last_peered":"2024-09-11T07:16:23.680171+0000","last_clean":"2024-09-11T07:16:23.680171+0000","last_became_active":"2024-09-11T07:16:08.976645+0000","last_became_peered":"2024-09-11T07:16:08.976645+0000","last_unstale":"2024-09-11T07:16:23.680171+0000","last_undegraded":"2024-09-11T07:16:23.680171+0000","last_fullsized":"2024-09-11T07:16:23.680171+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_clean_scrub_stamp":"2024-09-11T07:16:07.961130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:08:04.733864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"59'8","reported_seq":38,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:17:22.169801+0000","last_change":"2024-09-11T07:16:13.013368+0000","last_active":"2024-09-11T07:17:22.169801+0000","last_peered":"2024-09-11T07:17:22.169801+0000","last_clean":"2024-09-11T07:17:22.169801+0000","last_became_active":"2024-09-11T07:16:13.012797+0000","last_became_peered":"2024-09-11T07:16:13.012797+0000","last_unstale":"2024-09-11T07:17:22.169801+0000","last_undegraded":"2024-09-11T07:17:22.169801+0000","last_fullsized":"2024-09-11T07:17:22.169801+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:11.983377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:11.983377+0000","last_clean_scrub_stamp":"2024-09-11T07:16:11.983377+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:26:17.888213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.695814+0000","last_change":"2024-09-11T07:16:15.026438+0000","last_active":"2024-09-11T07:16:23.695814+0000","last_peered":"2024-09-11T07:16:23.695814+0000","last_clean":"2024-09-11T07:16:23.695814+0000","last_became_active":"2024-09-11T07:16:15.025971+0000","last_became_peered":"2024-09-11T07:16:15.025971+0000","last_unstale":"2024-09-11T07:16:23.695814+0000","last_undegraded":"2024-09-11T07:16:23.695814+0000","last_fullsized":"2024-09-11T07:16:23.695814+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:14.001856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:14.001856+0000","last_clean_scrub_stamp":"2024-09-11T07:16:14.001856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:39:13.747754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"59'28","reported_seq":80,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:17:22.169839+0000","last_change":"2024-09-11T07:16:10.990091+0000","last_active":"2024-09-11T07:17:22.169839+0000","last_peered":"2024-09-11T07:17:22.169839+0000","last_clean":"2024-09-11T07:17:22.169839+0000","last_became_active":"2024-09-11T07:16:10.989436+0000","last_became_peered":"2024-09-11T07:16:10.989436+0000","last_unstale":"2024-09-11T07:17:22.169839+0000","last_undegraded":"2024-09-11T07:17:22.169839+0000","last_fullsized":"2024-09-11T07:17:22.169839+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:09.964392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:09.964392+0000","last_clean_scrub_stamp":"2024-09-11T07:16:09.964392+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:51:08.310596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.698636+0000","last_change":"2024-09-11T07:16:08.978809+0000","last_active":"2024-09-11T07:16:23.698636+0000","last_peered":"2024-09-11T07:16:23.698636+0000","last_clean":"2024-09-11T07:16:23.698636+0000","last_became_active":"2024-09-11T07:16:08.978528+0000","last_became_peered":"2024-09-11T07:16:08.978528+0000","last_unstale":"2024-09-11T07:16:23.698636+0000","last_undegraded":"2024-09-11T07:16:23.698636+0000","last_fullsized":"2024-09-11T07:16:23.698636+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_clean_scrub_stamp":"2024-09-11T07:16:07.961130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:22:10.777140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.696400+0000","last_change":"2024-09-11T07:16:13.013998+0000","last_active":"2024-09-11T07:16:23.696400+0000","last_peered":"2024-09-11T07:16:23.696400+0000","last_clean":"2024-09-11T07:16:23.696400+0000","last_became_active":"2024-09-11T07:16:13.013744+0000","last_became_peered":"2024-09-11T07:16:13.013744+0000","last_unstale":"2024-09-11T07:16:23.696400+0000","last_undegraded":"2024-09-11T07:16:23.696400+0000","last_fullsized":"2024-09-11T07:16:23.696400+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:11.983377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:11.983377+0000","last_clean_scrub_stamp":"2024-09-11T07:16:11.983377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:19:54.892465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.544634+0000","last_change":"2024-09-11T07:16:15.027135+0000","last_active":"2024-09-11T07:16:23.544634+0000","last_peered":"2024-09-11T07:16:23.544634+0000","last_clean":"2024-09-11T07:16:23.544634+0000","last_became_active":"2024-09-11T07:16:15.026709+0000","last_became_peered":"2024-09-11T07:16:15.026709+0000","last_unstale":"2024-09-11T07:16:23.544634+0000","last_undegraded":"2024-09-11T07:16:23.544634+0000","last_fullsized":"2024-09-11T07:16:23.544634+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:14.001856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:14.001856+0000","last_clean_scrub_stamp":"2024-09-11T07:16:14.001856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:14:42.680069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"59'13","reported_seq":43,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.680558+0000","last_change":"2024-09-11T07:16:10.996497+0000","last_active":"2024-09-11T07:16:23.680558+0000","last_peered":"2024-09-11T07:16:23.680558+0000","last_clean":"2024-09-11T07:16:23.680558+0000","last_became_active":"2024-09-11T07:16:10.996212+0000","last_became_peered":"2024-09-11T07:16:10.996212+0000","last_unstale":"2024-09-11T07:16:23.680558+0000","last_undegraded":"2024-09-11T07:16:23.680558+0000","last_fullsized":"2024-09-11T07:16:23.680558+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:09.964392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:09.964392+0000","last_clean_scrub_stamp":"2024-09-11T07:16:09.964392+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:39:41.676138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"59'5","reported_seq":100,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:17:28.157605+0000","last_change":"2024-09-11T07:16:13.005775+0000","last_active":"2024-09-11T07:17:28.157605+0000","last_peered":"2024-09-11T07:17:28.157605+0000","last_clean":"2024-09-11T07:17:28.157605+0000","last_became_active":"2024-09-11T07:16:08.979258+0000","last_became_peered":"2024-09-11T07:16:08.979258+0000","last_unstale":"2024-09-11T07:17:28.157605+0000","last_undegraded":"2024-09-11T07:17:28.157605+0000","last_fullsized":"2024-09-11T07:17:28.157605+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_clean_scrub_stamp":"2024-09-11T07:16:07.961130+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:38:14.432540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052719500000000005,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":74,"num_read_kb":69,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.545772+0000","last_change":"2024-09-11T07:16:08.975315+0000","last_active":"2024-09-11T07:16:23.545772+0000","last_peered":"2024-09-11T07:16:23.545772+0000","last_clean":"2024-09-11T07:16:23.545772+0000","last_became_active":"2024-09-11T07:16:08.975046+0000","last_became_peered":"2024-09-11T07:16:08.975046+0000","last_unstale":"2024-09-11T07:16:23.545772+0000","last_undegraded":"2024-09-11T07:16:23.545772+0000","last_fullsized":"2024-09-11T07:16:23.545772+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_clean_scrub_stamp":"2024-09-11T07:16:07.961130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:25:50.357403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.546056+0000","last_change":"2024-09-11T07:16:13.005939+0000","last_active":"2024-09-11T07:16:23.546056+0000","last_peered":"2024-09-11T07:16:23.546056+0000","last_clean":"2024-09-11T07:16:23.546056+0000","last_became_active":"2024-09-11T07:16:13.005634+0000","last_became_peered":"2024-09-11T07:16:13.005634+0000","last_unstale":"2024-09-11T07:16:23.546056+0000","last_undegraded":"2024-09-11T07:16:23.546056+0000","last_fullsized":"2024-09-11T07:16:23.546056+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:11.983377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:11.983377+0000","last_clean_scrub_stamp":"2024-09-11T07:16:11.983377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:05:35.892098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.696415+0000","last_change":"2024-09-11T07:16:15.570119+0000","last_active":"2024-09-11T07:16:23.696415+0000","last_peered":"2024-09-11T07:16:23.696415+0000","last_clean":"2024-09-11T07:16:23.696415+0000","last_became_active":"2024-09-11T07:16:15.569653+0000","last_became_peered":"2024-09-11T07:16:15.569653+0000","last_unstale":"2024-09-11T07:16:23.696415+0000","last_undegraded":"2024-09-11T07:16:23.696415+0000","last_fullsized":"2024-09-11T07:16:23.696415+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:14.001856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:14.001856+0000","last_clean_scrub_stamp":"2024-09-11T07:16:14.001856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:39:14.264606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"59'12","reported_seq":34,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.696710+0000","last_change":"2024-09-11T07:16:10.994427+0000","last_active":"2024-09-11T07:16:23.696710+0000","last_peered":"2024-09-11T07:16:23.696710+0000","last_clean":"2024-09-11T07:16:23.696710+0000","last_became_active":"2024-09-11T07:16:10.994116+0000","last_became_peered":"2024-09-11T07:16:10.994116+0000","last_unstale":"2024-09-11T07:16:23.696710+0000","last_undegraded":"2024-09-11T07:16:23.696710+0000","last_fullsized":"2024-09-11T07:16:23.696710+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:09.964392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:09.964392+0000","last_clean_scrub_stamp":"2024-09-11T07:16:09.964392+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:08:16.497313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"54'1","reported_seq":24,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.696327+0000","last_change":"2024-09-11T07:16:13.001573+0000","last_active":"2024-09-11T07:16:23.696327+0000","last_peered":"2024-09-11T07:16:23.696327+0000","last_clean":"2024-09-11T07:16:23.696327+0000","last_became_active":"2024-09-11T07:16:08.984849+0000","last_became_peered":"2024-09-11T07:16:08.984849+0000","last_unstale":"2024-09-11T07:16:23.696327+0000","last_undegraded":"2024-09-11T07:16:23.696327+0000","last_fullsized":"2024-09-11T07:16:23.696327+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_clean_scrub_stamp":"2024-09-11T07:16:07.961130+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:45:24.535362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013614079999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.696896+0000","last_change":"2024-09-11T07:16:08.983320+0000","last_active":"2024-09-11T07:16:23.696896+0000","last_peered":"2024-09-11T07:16:23.696896+0000","last_clean":"2024-09-11T07:16:23.696896+0000","last_became_active":"2024-09-11T07:16:08.982894+0000","last_became_peered":"2024-09-11T07:16:08.982894+0000","last_unstale":"2024-09-11T07:16:23.696896+0000","last_undegraded":"2024-09-11T07:16:23.696896+0000","last_fullsized":"2024-09-11T07:16:23.696896+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_clean_scrub_stamp":"2024-09-11T07:16:07.961130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:08:53.654473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.700279+0000","last_change":"2024-09-11T07:16:13.011556+0000","last_active":"2024-09-11T07:16:23.700279+0000","last_peered":"2024-09-11T07:16:23.700279+0000","last_clean":"2024-09-11T07:16:23.700279+0000","last_became_active":"2024-09-11T07:16:13.011254+0000","last_became_peered":"2024-09-11T07:16:13.011254+0000","last_unstale":"2024-09-11T07:16:23.700279+0000","last_undegraded":"2024-09-11T07:16:23.700279+0000","last_fullsized":"2024-09-11T07:16:23.700279+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:11.983377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:11.983377+0000","last_clean_scrub_stamp":"2024-09-11T07:16:11.983377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:19:49.285338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.679933+0000","last_change":"2024-09-11T07:16:15.026604+0000","last_active":"2024-09-11T07:16:23.679933+0000","last_peered":"2024-09-11T07:16:23.679933+0000","last_clean":"2024-09-11T07:16:23.679933+0000","last_became_active":"2024-09-11T07:16:15.026236+0000","last_became_peered":"2024-09-11T07:16:15.026236+0000","last_unstale":"2024-09-11T07:16:23.679933+0000","last_undegraded":"2024-09-11T07:16:23.679933+0000","last_fullsized":"2024-09-11T07:16:23.679933+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:14.001856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:14.001856+0000","last_clean_scrub_stamp":"2024-09-11T07:16:14.001856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:19:40.999706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"59'14","reported_seq":38,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.697262+0000","last_change":"2024-09-11T07:16:11.082694+0000","last_active":"2024-09-11T07:16:23.697262+0000","last_peered":"2024-09-11T07:16:23.697262+0000","last_clean":"2024-09-11T07:16:23.697262+0000","last_became_active":"2024-09-11T07:16:11.082381+0000","last_became_peered":"2024-09-11T07:16:11.082381+0000","last_unstale":"2024-09-11T07:16:23.697262+0000","last_undegraded":"2024-09-11T07:16:23.697262+0000","last_fullsized":"2024-09-11T07:16:23.697262+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:09.964392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:09.964392+0000","last_clean_scrub_stamp":"2024-09-11T07:16:09.964392+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:21:34.377717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"56'2","reported_seq":25,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.544056+0000","last_change":"2024-09-11T07:16:12.998901+0000","last_active":"2024-09-11T07:16:23.544056+0000","last_peered":"2024-09-11T07:16:23.544056+0000","last_clean":"2024-09-11T07:16:23.544056+0000","last_became_active":"2024-09-11T07:16:08.982307+0000","last_became_peered":"2024-09-11T07:16:08.982307+0000","last_unstale":"2024-09-11T07:16:23.544056+0000","last_undegraded":"2024-09-11T07:16:23.544056+0000","last_fullsized":"2024-09-11T07:16:23.544056+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_clean_scrub_stamp":"2024-09-11T07:16:07.961130+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:23:08.572218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000888977,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.699879+0000","last_change":"2024-09-11T07:16:08.978137+0000","last_active":"2024-09-11T07:16:23.699879+0000","last_peered":"2024-09-11T07:16:23.699879+0000","last_clean":"2024-09-11T07:16:23.699879+0000","last_became_active":"2024-09-11T07:16:08.977871+0000","last_became_peered":"2024-09-11T07:16:08.977871+0000","last_unstale":"2024-09-11T07:16:23.699879+0000","last_undegraded":"2024-09-11T07:16:23.699879+0000","last_fullsized":"2024-09-11T07:16:23.699879+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_clean_scrub_stamp":"2024-09-11T07:16:07.961130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:45:21.480371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"61'79","reported_seq":52,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:16:45.567631+0000","last_change":"2024-09-11T07:15:58.887909+0000","last_active":"2024-09-11T07:16:45.567631+0000","last_peered":"2024-09-11T07:16:45.567631+0000","last_clean":"2024-09-11T07:16:45.567631+0000","last_became_active":"2024-09-11T07:15:58.879318+0000","last_became_peered":"2024-09-11T07:15:58.879318+0000","last_unstale":"2024-09-11T07:16:45.567631+0000","last_undegraded":"2024-09-11T07:16:45.567631+0000","last_fullsized":"2024-09-11T07:16:45.567631+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:14:04.444392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:14:04.444392+0000","last_clean_scrub_stamp":"2024-09-11T07:14:04.444392+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:26:56.844140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.698195+0000","last_change":"2024-09-11T07:16:13.007044+0000","last_active":"2024-09-11T07:16:23.698195+0000","last_peered":"2024-09-11T07:16:23.698195+0000","last_clean":"2024-09-11T07:16:23.698195+0000","last_became_active":"2024-09-11T07:16:13.006091+0000","last_became_peered":"2024-09-11T07:16:13.006091+0000","last_unstale":"2024-09-11T07:16:23.698195+0000","last_undegraded":"2024-09-11T07:16:23.698195+0000","last_fullsized":"2024-09-11T07:16:23.698195+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:11.983377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:11.983377+0000","last_clean_scrub_stamp":"2024-09-11T07:16:11.983377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:58:22.085406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.700413+0000","last_change":"2024-09-11T07:16:15.027684+0000","last_active":"2024-09-11T07:16:23.700413+0000","last_peered":"2024-09-11T07:16:23.700413+0000","last_clean":"2024-09-11T07:16:23.700413+0000","last_became_active":"2024-09-11T07:16:15.027039+0000","last_became_peered":"2024-09-11T07:16:15.027039+0000","last_unstale":"2024-09-11T07:16:23.700413+0000","last_undegraded":"2024-09-11T07:16:23.700413+0000","last_fullsized":"2024-09-11T07:16:23.700413+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:14.001856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:14.001856+0000","last_clean_scrub_stamp":"2024-09-11T07:16:14.001856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:36:34.430096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"59'19","reported_seq":47,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.697847+0000","last_change":"2024-09-11T07:16:11.081420+0000","last_active":"2024-09-11T07:16:23.697847+0000","last_peered":"2024-09-11T07:16:23.697847+0000","last_clean":"2024-09-11T07:16:23.697847+0000","last_became_active":"2024-09-11T07:16:11.081112+0000","last_became_peered":"2024-09-11T07:16:11.081112+0000","last_unstale":"2024-09-11T07:16:23.697847+0000","last_undegraded":"2024-09-11T07:16:23.697847+0000","last_fullsized":"2024-09-11T07:16:23.697847+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:09.964392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:09.964392+0000","last_clean_scrub_stamp":"2024-09-11T07:16:09.964392+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:53:14.717541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.546253+0000","last_change":"2024-09-11T07:16:08.974450+0000","last_active":"2024-09-11T07:16:23.546253+0000","last_peered":"2024-09-11T07:16:23.546253+0000","last_clean":"2024-09-11T07:16:23.546253+0000","last_became_active":"2024-09-11T07:16:08.974151+0000","last_became_peered":"2024-09-11T07:16:08.974151+0000","last_unstale":"2024-09-11T07:16:23.546253+0000","last_undegraded":"2024-09-11T07:16:23.546253+0000","last_fullsized":"2024-09-11T07:16:23.546253+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_clean_scrub_stamp":"2024-09-11T07:16:07.961130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:40:49.090138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.545847+0000","last_change":"2024-09-11T07:16:13.009433+0000","last_active":"2024-09-11T07:16:23.545847+0000","last_peered":"2024-09-11T07:16:23.545847+0000","last_clean":"2024-09-11T07:16:23.545847+0000","last_became_active":"2024-09-11T07:16:13.008979+0000","last_became_peered":"2024-09-11T07:16:13.008979+0000","last_unstale":"2024-09-11T07:16:23.545847+0000","last_undegraded":"2024-09-11T07:16:23.545847+0000","last_fullsized":"2024-09-11T07:16:23.545847+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:11.983377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:11.983377+0000","last_clean_scrub_stamp":"2024-09-11T07:16:11.983377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:31:35.505296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.696972+0000","last_change":"2024-09-11T07:16:15.032933+0000","last_active":"2024-09-11T07:16:23.696972+0000","last_peered":"2024-09-11T07:16:23.696972+0000","last_clean":"2024-09-11T07:16:23.696972+0000","last_became_active":"2024-09-11T07:16:15.032534+0000","last_became_peered":"2024-09-11T07:16:15.032534+0000","last_unstale":"2024-09-11T07:16:23.696972+0000","last_undegraded":"2024-09-11T07:16:23.696972+0000","last_fullsized":"2024-09-11T07:16:23.696972+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:14.001856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:14.001856+0000","last_clean_scrub_stamp":"2024-09-11T07:16:14.001856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:45:48.995408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"59'9","reported_seq":32,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.696678+0000","last_change":"2024-09-11T07:16:10.995313+0000","last_active":"2024-09-11T07:16:23.696678+0000","last_peered":"2024-09-11T07:16:23.696678+0000","last_clean":"2024-09-11T07:16:23.696678+0000","last_became_active":"2024-09-11T07:16:10.995009+0000","last_became_peered":"2024-09-11T07:16:10.995009+0000","last_unstale":"2024-09-11T07:16:23.696678+0000","last_undegraded":"2024-09-11T07:16:23.696678+0000","last_fullsized":"2024-09-11T07:16:23.696678+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:09.964392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:09.964392+0000","last_clean_scrub_stamp":"2024-09-11T07:16:09.964392+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:52:00.925463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.680498+0000","last_change":"2024-09-11T07:16:08.986579+0000","last_active":"2024-09-11T07:16:23.680498+0000","last_peered":"2024-09-11T07:16:23.680498+0000","last_clean":"2024-09-11T07:16:23.680498+0000","last_became_active":"2024-09-11T07:16:08.986226+0000","last_became_peered":"2024-09-11T07:16:08.986226+0000","last_unstale":"2024-09-11T07:16:23.680498+0000","last_undegraded":"2024-09-11T07:16:23.680498+0000","last_fullsized":"2024-09-11T07:16:23.680498+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_clean_scrub_stamp":"2024-09-11T07:16:07.961130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:56:55.800637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.545850+0000","last_change":"2024-09-11T07:16:13.003534+0000","last_active":"2024-09-11T07:16:23.545850+0000","last_peered":"2024-09-11T07:16:23.545850+0000","last_clean":"2024-09-11T07:16:23.545850+0000","last_became_active":"2024-09-11T07:16:13.003262+0000","last_became_peered":"2024-09-11T07:16:13.003262+0000","last_unstale":"2024-09-11T07:16:23.545850+0000","last_undegraded":"2024-09-11T07:16:23.545850+0000","last_fullsized":"2024-09-11T07:16:23.545850+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:11.983377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:11.983377+0000","last_clean_scrub_stamp":"2024-09-11T07:16:11.983377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:30:35.612516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.696108+0000","last_change":"2024-09-11T07:16:15.028742+0000","last_active":"2024-09-11T07:16:23.696108+0000","last_peered":"2024-09-11T07:16:23.696108+0000","last_clean":"2024-09-11T07:16:23.696108+0000","last_became_active":"2024-09-11T07:16:15.028429+0000","last_became_peered":"2024-09-11T07:16:15.028429+0000","last_unstale":"2024-09-11T07:16:23.696108+0000","last_undegraded":"2024-09-11T07:16:23.696108+0000","last_fullsized":"2024-09-11T07:16:23.696108+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:14.001856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:14.001856+0000","last_clean_scrub_stamp":"2024-09-11T07:16:14.001856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:06:59.941670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"59'15","reported_seq":41,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.699120+0000","last_change":"2024-09-11T07:16:11.081415+0000","last_active":"2024-09-11T07:16:23.699120+0000","last_peered":"2024-09-11T07:16:23.699120+0000","last_clean":"2024-09-11T07:16:23.699120+0000","last_became_active":"2024-09-11T07:16:11.081110+0000","last_became_peered":"2024-09-11T07:16:11.081110+0000","last_unstale":"2024-09-11T07:16:23.699120+0000","last_undegraded":"2024-09-11T07:16:23.699120+0000","last_fullsized":"2024-09-11T07:16:23.699120+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:09.964392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:09.964392+0000","last_clean_scrub_stamp":"2024-09-11T07:16:09.964392+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:59:13.240959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.546157+0000","last_change":"2024-09-11T07:16:08.974091+0000","last_active":"2024-09-11T07:16:23.546157+0000","last_peered":"2024-09-11T07:16:23.546157+0000","last_clean":"2024-09-11T07:16:23.546157+0000","last_became_active":"2024-09-11T07:16:08.973728+0000","last_became_peered":"2024-09-11T07:16:08.973728+0000","last_unstale":"2024-09-11T07:16:23.546157+0000","last_undegraded":"2024-09-11T07:16:23.546157+0000","last_fullsized":"2024-09-11T07:16:23.546157+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_clean_scrub_stamp":"2024-09-11T07:16:07.961130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:35:20.075350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"59'8","reported_seq":38,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:17:22.169881+0000","last_change":"2024-09-11T07:16:13.012746+0000","last_active":"2024-09-11T07:17:22.169881+0000","last_peered":"2024-09-11T07:17:22.169881+0000","last_clean":"2024-09-11T07:17:22.169881+0000","last_became_active":"2024-09-11T07:16:13.012464+0000","last_became_peered":"2024-09-11T07:16:13.012464+0000","last_unstale":"2024-09-11T07:17:22.169881+0000","last_undegraded":"2024-09-11T07:17:22.169881+0000","last_fullsized":"2024-09-11T07:17:22.169881+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:11.983377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:11.983377+0000","last_clean_scrub_stamp":"2024-09-11T07:16:11.983377+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:23:45.643689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.698339+0000","last_change":"2024-09-11T07:16:15.569648+0000","last_active":"2024-09-11T07:16:23.698339+0000","last_peered":"2024-09-11T07:16:23.698339+0000","last_clean":"2024-09-11T07:16:23.698339+0000","last_became_active":"2024-09-11T07:16:15.569333+0000","last_became_peered":"2024-09-11T07:16:15.569333+0000","last_unstale":"2024-09-11T07:16:23.698339+0000","last_undegraded":"2024-09-11T07:16:23.698339+0000","last_fullsized":"2024-09-11T07:16:23.698339+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:14.001856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:14.001856+0000","last_clean_scrub_stamp":"2024-09-11T07:16:14.001856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:33:20.254599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"59'12","reported_seq":39,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.545768+0000","last_change":"2024-09-11T07:16:10.987743+0000","last_active":"2024-09-11T07:16:23.545768+0000","last_peered":"2024-09-11T07:16:23.545768+0000","last_clean":"2024-09-11T07:16:23.545768+0000","last_became_active":"2024-09-11T07:16:10.987313+0000","last_became_peered":"2024-09-11T07:16:10.987313+0000","last_unstale":"2024-09-11T07:16:23.545768+0000","last_undegraded":"2024-09-11T07:16:23.545768+0000","last_fullsized":"2024-09-11T07:16:23.545768+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:09.964392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:09.964392+0000","last_clean_scrub_stamp":"2024-09-11T07:16:09.964392+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:26.793507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"52'2","reported_seq":35,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.546369+0000","last_change":"2024-09-11T07:16:08.974186+0000","last_active":"2024-09-11T07:16:23.546369+0000","last_peered":"2024-09-11T07:16:23.546369+0000","last_clean":"2024-09-11T07:16:23.546369+0000","last_became_active":"2024-09-11T07:16:08.973880+0000","last_became_peered":"2024-09-11T07:16:08.973880+0000","last_unstale":"2024-09-11T07:16:23.546369+0000","last_undegraded":"2024-09-11T07:16:23.546369+0000","last_fullsized":"2024-09-11T07:16:23.546369+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_clean_scrub_stamp":"2024-09-11T07:16:07.961130+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:43:40.564307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.545675+0000","last_change":"2024-09-11T07:16:13.009140+0000","last_active":"2024-09-11T07:16:23.545675+0000","last_peered":"2024-09-11T07:16:23.545675+0000","last_clean":"2024-09-11T07:16:23.545675+0000","last_became_active":"2024-09-11T07:16:13.008879+0000","last_became_peered":"2024-09-11T07:16:13.008879+0000","last_unstale":"2024-09-11T07:16:23.545675+0000","last_undegraded":"2024-09-11T07:16:23.545675+0000","last_fullsized":"2024-09-11T07:16:23.545675+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:11.983377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:11.983377+0000","last_clean_scrub_stamp":"2024-09-11T07:16:11.983377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:34:29.556391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.543894+0000","last_change":"2024-09-11T07:16:15.027987+0000","last_active":"2024-09-11T07:16:23.543894+0000","last_peered":"2024-09-11T07:16:23.543894+0000","last_clean":"2024-09-11T07:16:23.543894+0000","last_became_active":"2024-09-11T07:16:15.027708+0000","last_became_peered":"2024-09-11T07:16:15.027708+0000","last_unstale":"2024-09-11T07:16:23.543894+0000","last_undegraded":"2024-09-11T07:16:23.543894+0000","last_fullsized":"2024-09-11T07:16:23.543894+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:14.001856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:14.001856+0000","last_clean_scrub_stamp":"2024-09-11T07:16:14.001856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:56:24.582132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"59'9","reported_seq":32,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.696616+0000","last_change":"2024-09-11T07:16:11.083008+0000","last_active":"2024-09-11T07:16:23.696616+0000","last_peered":"2024-09-11T07:16:23.696616+0000","last_clean":"2024-09-11T07:16:23.696616+0000","last_became_active":"2024-09-11T07:16:11.082720+0000","last_became_peered":"2024-09-11T07:16:11.082720+0000","last_unstale":"2024-09-11T07:16:23.696616+0000","last_undegraded":"2024-09-11T07:16:23.696616+0000","last_fullsized":"2024-09-11T07:16:23.696616+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:09.964392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:09.964392+0000","last_clean_scrub_stamp":"2024-09-11T07:16:09.964392+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:20:19.732631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.545772+0000","last_change":"2024-09-11T07:16:08.974988+0000","last_active":"2024-09-11T07:16:23.545772+0000","last_peered":"2024-09-11T07:16:23.545772+0000","last_clean":"2024-09-11T07:16:23.545772+0000","last_became_active":"2024-09-11T07:16:08.974744+0000","last_became_peered":"2024-09-11T07:16:08.974744+0000","last_unstale":"2024-09-11T07:16:23.545772+0000","last_undegraded":"2024-09-11T07:16:23.545772+0000","last_fullsized":"2024-09-11T07:16:23.545772+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_clean_scrub_stamp":"2024-09-11T07:16:07.961130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:21:27.696556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.544957+0000","last_change":"2024-09-11T07:16:13.009819+0000","last_active":"2024-09-11T07:16:23.544957+0000","last_peered":"2024-09-11T07:16:23.544957+0000","last_clean":"2024-09-11T07:16:23.544957+0000","last_became_active":"2024-09-11T07:16:13.009408+0000","last_became_peered":"2024-09-11T07:16:13.009408+0000","last_unstale":"2024-09-11T07:16:23.544957+0000","last_undegraded":"2024-09-11T07:16:23.544957+0000","last_fullsized":"2024-09-11T07:16:23.544957+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:11.983377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:11.983377+0000","last_clean_scrub_stamp":"2024-09-11T07:16:11.983377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:09:00.776717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.546034+0000","last_change":"2024-09-11T07:16:15.030338+0000","last_active":"2024-09-11T07:16:23.546034+0000","last_peered":"2024-09-11T07:16:23.546034+0000","last_clean":"2024-09-11T07:16:23.546034+0000","last_became_active":"2024-09-11T07:16:15.030034+0000","last_became_peered":"2024-09-11T07:16:15.030034+0000","last_unstale":"2024-09-11T07:16:23.546034+0000","last_undegraded":"2024-09-11T07:16:23.546034+0000","last_fullsized":"2024-09-11T07:16:23.546034+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:14.001856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:14.001856+0000","last_clean_scrub_stamp":"2024-09-11T07:16:14.001856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:47:43.000121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.698746+0000","last_change":"2024-09-11T07:16:08.975333+0000","last_active":"2024-09-11T07:16:23.698746+0000","last_peered":"2024-09-11T07:16:23.698746+0000","last_clean":"2024-09-11T07:16:23.698746+0000","last_became_active":"2024-09-11T07:16:08.975069+0000","last_became_peered":"2024-09-11T07:16:08.975069+0000","last_unstale":"2024-09-11T07:16:23.698746+0000","last_undegraded":"2024-09-11T07:16:23.698746+0000","last_fullsized":"2024-09-11T07:16:23.698746+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_clean_scrub_stamp":"2024-09-11T07:16:07.961130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:55:47.019869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"59'6","reported_seq":25,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.545046+0000","last_change":"2024-09-11T07:16:11.082609+0000","last_active":"2024-09-11T07:16:23.545046+0000","last_peered":"2024-09-11T07:16:23.545046+0000","last_clean":"2024-09-11T07:16:23.545046+0000","last_became_active":"2024-09-11T07:16:11.082343+0000","last_became_peered":"2024-09-11T07:16:11.082343+0000","last_unstale":"2024-09-11T07:16:23.545046+0000","last_undegraded":"2024-09-11T07:16:23.545046+0000","last_fullsized":"2024-09-11T07:16:23.545046+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:09.964392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:09.964392+0000","last_clean_scrub_stamp":"2024-09-11T07:16:09.964392+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:04:53.558163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.700226+0000","last_change":"2024-09-11T07:16:13.012413+0000","last_active":"2024-09-11T07:16:23.700226+0000","last_peered":"2024-09-11T07:16:23.700226+0000","last_clean":"2024-09-11T07:16:23.700226+0000","last_became_active":"2024-09-11T07:16:13.012058+0000","last_became_peered":"2024-09-11T07:16:13.012058+0000","last_unstale":"2024-09-11T07:16:23.700226+0000","last_undegraded":"2024-09-11T07:16:23.700226+0000","last_fullsized":"2024-09-11T07:16:23.700226+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:11.983377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:11.983377+0000","last_clean_scrub_stamp":"2024-09-11T07:16:11.983377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:56:31.879206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.696413+0000","last_change":"2024-09-11T07:16:15.571203+0000","last_active":"2024-09-11T07:16:23.696413+0000","last_peered":"2024-09-11T07:16:23.696413+0000","last_clean":"2024-09-11T07:16:23.696413+0000","last_became_active":"2024-09-11T07:16:15.570861+0000","last_became_peered":"2024-09-11T07:16:15.570861+0000","last_unstale":"2024-09-11T07:16:23.696413+0000","last_undegraded":"2024-09-11T07:16:23.696413+0000","last_fullsized":"2024-09-11T07:16:23.696413+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:14.001856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:14.001856+0000","last_clean_scrub_stamp":"2024-09-11T07:16:14.001856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:16:54.277674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"59'10","reported_seq":31,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.544451+0000","last_change":"2024-09-11T07:16:11.082543+0000","last_active":"2024-09-11T07:16:23.544451+0000","last_peered":"2024-09-11T07:16:23.544451+0000","last_clean":"2024-09-11T07:16:23.544451+0000","last_became_active":"2024-09-11T07:16:11.082261+0000","last_became_peered":"2024-09-11T07:16:11.082261+0000","last_unstale":"2024-09-11T07:16:23.544451+0000","last_undegraded":"2024-09-11T07:16:23.544451+0000","last_fullsized":"2024-09-11T07:16:23.544451+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:09.964392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:09.964392+0000","last_clean_scrub_stamp":"2024-09-11T07:16:09.964392+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:42:04.524886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.698553+0000","last_change":"2024-09-11T07:16:08.979347+0000","last_active":"2024-09-11T07:16:23.698553+0000","last_peered":"2024-09-11T07:16:23.698553+0000","last_clean":"2024-09-11T07:16:23.698553+0000","last_became_active":"2024-09-11T07:16:08.979081+0000","last_became_peered":"2024-09-11T07:16:08.979081+0000","last_unstale":"2024-09-11T07:16:23.698553+0000","last_undegraded":"2024-09-11T07:16:23.698553+0000","last_fullsized":"2024-09-11T07:16:23.698553+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_clean_scrub_stamp":"2024-09-11T07:16:07.961130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:42:45.466507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"59'8","reported_seq":38,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:17:22.169911+0000","last_change":"2024-09-11T07:16:13.009752+0000","last_active":"2024-09-11T07:17:22.169911+0000","last_peered":"2024-09-11T07:17:22.169911+0000","last_clean":"2024-09-11T07:17:22.169911+0000","last_became_active":"2024-09-11T07:16:13.009466+0000","last_became_peered":"2024-09-11T07:16:13.009466+0000","last_unstale":"2024-09-11T07:17:22.169911+0000","last_undegraded":"2024-09-11T07:17:22.169911+0000","last_fullsized":"2024-09-11T07:17:22.169911+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:11.983377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:11.983377+0000","last_clean_scrub_stamp":"2024-09-11T07:16:11.983377+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:34:56.569719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.696615+0000","last_change":"2024-09-11T07:16:15.030638+0000","last_active":"2024-09-11T07:16:23.696615+0000","last_peered":"2024-09-11T07:16:23.696615+0000","last_clean":"2024-09-11T07:16:23.696615+0000","last_became_active":"2024-09-11T07:16:15.030385+0000","last_became_peered":"2024-09-11T07:16:15.030385+0000","last_unstale":"2024-09-11T07:16:23.696615+0000","last_undegraded":"2024-09-11T07:16:23.696615+0000","last_fullsized":"2024-09-11T07:16:23.696615+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:14.001856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:14.001856+0000","last_clean_scrub_stamp":"2024-09-11T07:16:14.001856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:24:30.631482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"59'9","reported_seq":32,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.700633+0000","last_change":"2024-09-11T07:16:11.083226+0000","last_active":"2024-09-11T07:16:23.700633+0000","last_peered":"2024-09-11T07:16:23.700633+0000","last_clean":"2024-09-11T07:16:23.700633+0000","last_became_active":"2024-09-11T07:16:11.082666+0000","last_became_peered":"2024-09-11T07:16:11.082666+0000","last_unstale":"2024-09-11T07:16:23.700633+0000","last_undegraded":"2024-09-11T07:16:23.700633+0000","last_fullsized":"2024-09-11T07:16:23.700633+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:09.964392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:09.964392+0000","last_clean_scrub_stamp":"2024-09-11T07:16:09.964392+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:02:59.164722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.696780+0000","last_change":"2024-09-11T07:16:08.982466+0000","last_active":"2024-09-11T07:16:23.696780+0000","last_peered":"2024-09-11T07:16:23.696780+0000","last_clean":"2024-09-11T07:16:23.696780+0000","last_became_active":"2024-09-11T07:16:08.981585+0000","last_became_peered":"2024-09-11T07:16:08.981585+0000","last_unstale":"2024-09-11T07:16:23.696780+0000","last_undegraded":"2024-09-11T07:16:23.696780+0000","last_fullsized":"2024-09-11T07:16:23.696780+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_clean_scrub_stamp":"2024-09-11T07:16:07.961130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:42:18.050745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"59'8","reported_seq":38,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:17:22.169892+0000","last_change":"2024-09-11T07:16:13.001869+0000","last_active":"2024-09-11T07:17:22.169892+0000","last_peered":"2024-09-11T07:17:22.169892+0000","last_clean":"2024-09-11T07:17:22.169892+0000","last_became_active":"2024-09-11T07:16:13.001592+0000","last_became_peered":"2024-09-11T07:16:13.001592+0000","last_unstale":"2024-09-11T07:17:22.169892+0000","last_undegraded":"2024-09-11T07:17:22.169892+0000","last_fullsized":"2024-09-11T07:17:22.169892+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:11.983377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:11.983377+0000","last_clean_scrub_stamp":"2024-09-11T07:16:11.983377+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:08:24.689489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.546125+0000","last_change":"2024-09-11T07:16:15.026652+0000","last_active":"2024-09-11T07:16:23.546125+0000","last_peered":"2024-09-11T07:16:23.546125+0000","last_clean":"2024-09-11T07:16:23.546125+0000","last_became_active":"2024-09-11T07:16:15.026319+0000","last_became_peered":"2024-09-11T07:16:15.026319+0000","last_unstale":"2024-09-11T07:16:23.546125+0000","last_undegraded":"2024-09-11T07:16:23.546125+0000","last_fullsized":"2024-09-11T07:16:23.546125+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:14.001856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:14.001856+0000","last_clean_scrub_stamp":"2024-09-11T07:16:14.001856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:21:33.234447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"59'9","reported_seq":32,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.680065+0000","last_change":"2024-09-11T07:16:10.992302+0000","last_active":"2024-09-11T07:16:23.680065+0000","last_peered":"2024-09-11T07:16:23.680065+0000","last_clean":"2024-09-11T07:16:23.680065+0000","last_became_active":"2024-09-11T07:16:10.991836+0000","last_became_peered":"2024-09-11T07:16:10.991836+0000","last_unstale":"2024-09-11T07:16:23.680065+0000","last_undegraded":"2024-09-11T07:16:23.680065+0000","last_fullsized":"2024-09-11T07:16:23.680065+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:09.964392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:09.964392+0000","last_clean_scrub_stamp":"2024-09-11T07:16:09.964392+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:52:51.945977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.697828+0000","last_change":"2024-09-11T07:16:08.973032+0000","last_active":"2024-09-11T07:16:23.697828+0000","last_peered":"2024-09-11T07:16:23.697828+0000","last_clean":"2024-09-11T07:16:23.697828+0000","last_became_active":"2024-09-11T07:16:08.972619+0000","last_became_peered":"2024-09-11T07:16:08.972619+0000","last_unstale":"2024-09-11T07:16:23.697828+0000","last_undegraded":"2024-09-11T07:16:23.697828+0000","last_fullsized":"2024-09-11T07:16:23.697828+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_clean_scrub_stamp":"2024-09-11T07:16:07.961130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:13:56.241952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.543937+0000","last_change":"2024-09-11T07:16:13.009384+0000","last_active":"2024-09-11T07:16:23.543937+0000","last_peered":"2024-09-11T07:16:23.543937+0000","last_clean":"2024-09-11T07:16:23.543937+0000","last_became_active":"2024-09-11T07:16:13.008989+0000","last_became_peered":"2024-09-11T07:16:13.008989+0000","last_unstale":"2024-09-11T07:16:23.543937+0000","last_undegraded":"2024-09-11T07:16:23.543937+0000","last_fullsized":"2024-09-11T07:16:23.543937+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:11.983377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:11.983377+0000","last_clean_scrub_stamp":"2024-09-11T07:16:11.983377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:13:19.486173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.695814+0000","last_change":"2024-09-11T07:16:15.029843+0000","last_active":"2024-09-11T07:16:23.695814+0000","last_peered":"2024-09-11T07:16:23.695814+0000","last_clean":"2024-09-11T07:16:23.695814+0000","last_became_active":"2024-09-11T07:16:15.029462+0000","last_became_peered":"2024-09-11T07:16:15.029462+0000","last_unstale":"2024-09-11T07:16:23.695814+0000","last_undegraded":"2024-09-11T07:16:23.695814+0000","last_fullsized":"2024-09-11T07:16:23.695814+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:14.001856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:14.001856+0000","last_clean_scrub_stamp":"2024-09-11T07:16:14.001856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:37:34.145403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"59'11","reported_seq":35,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.545962+0000","last_change":"2024-09-11T07:16:10.990641+0000","last_active":"2024-09-11T07:16:23.545962+0000","last_peered":"2024-09-11T07:16:23.545962+0000","last_clean":"2024-09-11T07:16:23.545962+0000","last_became_active":"2024-09-11T07:16:10.989609+0000","last_became_peered":"2024-09-11T07:16:10.989609+0000","last_unstale":"2024-09-11T07:16:23.545962+0000","last_undegraded":"2024-09-11T07:16:23.545962+0000","last_fullsized":"2024-09-11T07:16:23.545962+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:09.964392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:09.964392+0000","last_clean_scrub_stamp":"2024-09-11T07:16:09.964392+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:53:16.552881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.680456+0000","last_change":"2024-09-11T07:16:08.977057+0000","last_active":"2024-09-11T07:16:23.680456+0000","last_peered":"2024-09-11T07:16:23.680456+0000","last_clean":"2024-09-11T07:16:23.680456+0000","last_became_active":"2024-09-11T07:16:08.976772+0000","last_became_peered":"2024-09-11T07:16:08.976772+0000","last_unstale":"2024-09-11T07:16:23.680456+0000","last_undegraded":"2024-09-11T07:16:23.680456+0000","last_fullsized":"2024-09-11T07:16:23.680456+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_clean_scrub_stamp":"2024-09-11T07:16:07.961130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:11:52.283361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.679842+0000","last_change":"2024-09-11T07:16:13.011222+0000","last_active":"2024-09-11T07:16:23.679842+0000","last_peered":"2024-09-11T07:16:23.679842+0000","last_clean":"2024-09-11T07:16:23.679842+0000","last_became_active":"2024-09-11T07:16:13.010919+0000","last_became_peered":"2024-09-11T07:16:13.010919+0000","last_unstale":"2024-09-11T07:16:23.679842+0000","last_undegraded":"2024-09-11T07:16:23.679842+0000","last_fullsized":"2024-09-11T07:16:23.679842+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:11.983377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:11.983377+0000","last_clean_scrub_stamp":"2024-09-11T07:16:11.983377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:15:46.736067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.543544+0000","last_change":"2024-09-11T07:16:15.025740+0000","last_active":"2024-09-11T07:16:23.543544+0000","last_peered":"2024-09-11T07:16:23.543544+0000","last_clean":"2024-09-11T07:16:23.543544+0000","last_became_active":"2024-09-11T07:16:15.025394+0000","last_became_peered":"2024-09-11T07:16:15.025394+0000","last_unstale":"2024-09-11T07:16:23.543544+0000","last_undegraded":"2024-09-11T07:16:23.543544+0000","last_fullsized":"2024-09-11T07:16:23.543544+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:14.001856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:14.001856+0000","last_clean_scrub_stamp":"2024-09-11T07:16:14.001856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:27:46.945874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"59'4","reported_seq":22,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.543556+0000","last_change":"2024-09-11T07:16:10.986829+0000","last_active":"2024-09-11T07:16:23.543556+0000","last_peered":"2024-09-11T07:16:23.543556+0000","last_clean":"2024-09-11T07:16:23.543556+0000","last_became_active":"2024-09-11T07:16:10.986556+0000","last_became_peered":"2024-09-11T07:16:10.986556+0000","last_unstale":"2024-09-11T07:16:23.543556+0000","last_undegraded":"2024-09-11T07:16:23.543556+0000","last_fullsized":"2024-09-11T07:16:23.543556+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:09.964392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:09.964392+0000","last_clean_scrub_stamp":"2024-09-11T07:16:09.964392+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:17:15.319325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.700351+0000","last_change":"2024-09-11T07:16:08.976759+0000","last_active":"2024-09-11T07:16:23.700351+0000","last_peered":"2024-09-11T07:16:23.700351+0000","last_clean":"2024-09-11T07:16:23.700351+0000","last_became_active":"2024-09-11T07:16:08.976510+0000","last_became_peered":"2024-09-11T07:16:08.976510+0000","last_unstale":"2024-09-11T07:16:23.700351+0000","last_undegraded":"2024-09-11T07:16:23.700351+0000","last_fullsized":"2024-09-11T07:16:23.700351+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_clean_scrub_stamp":"2024-09-11T07:16:07.961130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:09:47.175924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.698531+0000","last_change":"2024-09-11T07:16:13.015198+0000","last_active":"2024-09-11T07:16:23.698531+0000","last_peered":"2024-09-11T07:16:23.698531+0000","last_clean":"2024-09-11T07:16:23.698531+0000","last_became_active":"2024-09-11T07:16:13.014691+0000","last_became_peered":"2024-09-11T07:16:13.014691+0000","last_unstale":"2024-09-11T07:16:23.698531+0000","last_undegraded":"2024-09-11T07:16:23.698531+0000","last_fullsized":"2024-09-11T07:16:23.698531+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:11.983377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:11.983377+0000","last_clean_scrub_stamp":"2024-09-11T07:16:11.983377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:41:31.302999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"59'1","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.697438+0000","last_change":"2024-09-11T07:16:15.032852+0000","last_active":"2024-09-11T07:16:23.697438+0000","last_peered":"2024-09-11T07:16:23.697438+0000","last_clean":"2024-09-11T07:16:23.697438+0000","last_became_active":"2024-09-11T07:16:15.032569+0000","last_became_peered":"2024-09-11T07:16:15.032569+0000","last_unstale":"2024-09-11T07:16:23.697438+0000","last_undegraded":"2024-09-11T07:16:23.697438+0000","last_fullsized":"2024-09-11T07:16:23.697438+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:14.001856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:14.001856+0000","last_clean_scrub_stamp":"2024-09-11T07:16:14.001856+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:02:54.188336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"59'11","reported_seq":35,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.543700+0000","last_change":"2024-09-11T07:16:11.083002+0000","last_active":"2024-09-11T07:16:23.543700+0000","last_peered":"2024-09-11T07:16:23.543700+0000","last_clean":"2024-09-11T07:16:23.543700+0000","last_became_active":"2024-09-11T07:16:11.082731+0000","last_became_peered":"2024-09-11T07:16:11.082731+0000","last_unstale":"2024-09-11T07:16:23.543700+0000","last_undegraded":"2024-09-11T07:16:23.543700+0000","last_fullsized":"2024-09-11T07:16:23.543700+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:09.964392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:09.964392+0000","last_clean_scrub_stamp":"2024-09-11T07:16:09.964392+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:39:55.411384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.698215+0000","last_change":"2024-09-11T07:16:08.975854+0000","last_active":"2024-09-11T07:16:23.698215+0000","last_peered":"2024-09-11T07:16:23.698215+0000","last_clean":"2024-09-11T07:16:23.698215+0000","last_became_active":"2024-09-11T07:16:08.975563+0000","last_became_peered":"2024-09-11T07:16:08.975563+0000","last_unstale":"2024-09-11T07:16:23.698215+0000","last_undegraded":"2024-09-11T07:16:23.698215+0000","last_fullsized":"2024-09-11T07:16:23.698215+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_clean_scrub_stamp":"2024-09-11T07:16:07.961130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:46:33.738371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.696551+0000","last_change":"2024-09-11T07:16:13.011761+0000","last_active":"2024-09-11T07:16:23.696551+0000","last_peered":"2024-09-11T07:16:23.696551+0000","last_clean":"2024-09-11T07:16:23.696551+0000","last_became_active":"2024-09-11T07:16:13.011468+0000","last_became_peered":"2024-09-11T07:16:13.011468+0000","last_unstale":"2024-09-11T07:16:23.696551+0000","last_undegraded":"2024-09-11T07:16:23.696551+0000","last_fullsized":"2024-09-11T07:16:23.696551+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:11.983377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:11.983377+0000","last_clean_scrub_stamp":"2024-09-11T07:16:11.983377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:52:59.252238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.543795+0000","last_change":"2024-09-11T07:16:15.572137+0000","last_active":"2024-09-11T07:16:23.543795+0000","last_peered":"2024-09-11T07:16:23.543795+0000","last_clean":"2024-09-11T07:16:23.543795+0000","last_became_active":"2024-09-11T07:16:15.571732+0000","last_became_peered":"2024-09-11T07:16:15.571732+0000","last_unstale":"2024-09-11T07:16:23.543795+0000","last_undegraded":"2024-09-11T07:16:23.543795+0000","last_fullsized":"2024-09-11T07:16:23.543795+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:14.001856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:14.001856+0000","last_clean_scrub_stamp":"2024-09-11T07:16:14.001856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:31:03.983228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"59'1","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.698176+0000","last_change":"2024-09-11T07:16:15.031993+0000","last_active":"2024-09-11T07:16:23.698176+0000","last_peered":"2024-09-11T07:16:23.698176+0000","last_clean":"2024-09-11T07:16:23.698176+0000","last_became_active":"2024-09-11T07:16:15.031709+0000","last_became_peered":"2024-09-11T07:16:15.031709+0000","last_unstale":"2024-09-11T07:16:23.698176+0000","last_undegraded":"2024-09-11T07:16:23.698176+0000","last_fullsized":"2024-09-11T07:16:23.698176+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:14.001856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:14.001856+0000","last_clean_scrub_stamp":"2024-09-11T07:16:14.001856+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:01:52.061264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.699216+0000","last_change":"2024-09-11T07:16:08.975465+0000","last_active":"2024-09-11T07:16:23.699216+0000","last_peered":"2024-09-11T07:16:23.699216+0000","last_clean":"2024-09-11T07:16:23.699216+0000","last_became_active":"2024-09-11T07:16:08.975212+0000","last_became_peered":"2024-09-11T07:16:08.975212+0000","last_unstale":"2024-09-11T07:16:23.699216+0000","last_undegraded":"2024-09-11T07:16:23.699216+0000","last_fullsized":"2024-09-11T07:16:23.699216+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_clean_scrub_stamp":"2024-09-11T07:16:07.961130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:35:00.408914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"59'10","reported_seq":31,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.696517+0000","last_change":"2024-09-11T07:16:11.083005+0000","last_active":"2024-09-11T07:16:23.696517+0000","last_peered":"2024-09-11T07:16:23.696517+0000","last_clean":"2024-09-11T07:16:23.696517+0000","last_became_active":"2024-09-11T07:16:11.082373+0000","last_became_peered":"2024-09-11T07:16:11.082373+0000","last_unstale":"2024-09-11T07:16:23.696517+0000","last_undegraded":"2024-09-11T07:16:23.696517+0000","last_fullsized":"2024-09-11T07:16:23.696517+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:09.964392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:09.964392+0000","last_clean_scrub_stamp":"2024-09-11T07:16:09.964392+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:26:15.959149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"59'8","reported_seq":41,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T07:17:22.169921+0000","last_change":"2024-09-11T07:16:13.013994+0000","last_active":"2024-09-11T07:17:22.169921+0000","last_peered":"2024-09-11T07:17:22.169921+0000","last_clean":"2024-09-11T07:17:22.169921+0000","last_became_active":"2024-09-11T07:16:13.013642+0000","last_became_peered":"2024-09-11T07:16:13.013642+0000","last_unstale":"2024-09-11T07:17:22.169921+0000","last_undegraded":"2024-09-11T07:17:22.169921+0000","last_fullsized":"2024-09-11T07:17:22.169921+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:11.983377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:11.983377+0000","last_clean_scrub_stamp":"2024-09-11T07:16:11.983377+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:29:48.739971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.679987+0000","last_change":"2024-09-11T07:16:15.031643+0000","last_active":"2024-09-11T07:16:23.679987+0000","last_peered":"2024-09-11T07:16:23.679987+0000","last_clean":"2024-09-11T07:16:23.679987+0000","last_became_active":"2024-09-11T07:16:15.031223+0000","last_became_peered":"2024-09-11T07:16:15.031223+0000","last_unstale":"2024-09-11T07:16:23.679987+0000","last_undegraded":"2024-09-11T07:16:23.679987+0000","last_fullsized":"2024-09-11T07:16:23.679987+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:14.001856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:14.001856+0000","last_clean_scrub_stamp":"2024-09-11T07:16:14.001856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:10:01.569732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.543806+0000","last_change":"2024-09-11T07:16:08.983839+0000","last_active":"2024-09-11T07:16:23.543806+0000","last_peered":"2024-09-11T07:16:23.543806+0000","last_clean":"2024-09-11T07:16:23.543806+0000","last_became_active":"2024-09-11T07:16:08.983507+0000","last_became_peered":"2024-09-11T07:16:08.983507+0000","last_unstale":"2024-09-11T07:16:23.543806+0000","last_undegraded":"2024-09-11T07:16:23.543806+0000","last_fullsized":"2024-09-11T07:16:23.543806+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_clean_scrub_stamp":"2024-09-11T07:16:07.961130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:19:31.746340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"59'11","reported_seq":35,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.698137+0000","last_change":"2024-09-11T07:16:10.991086+0000","last_active":"2024-09-11T07:16:23.698137+0000","last_peered":"2024-09-11T07:16:23.698137+0000","last_clean":"2024-09-11T07:16:23.698137+0000","last_became_active":"2024-09-11T07:16:10.990727+0000","last_became_peered":"2024-09-11T07:16:10.990727+0000","last_unstale":"2024-09-11T07:16:23.698137+0000","last_undegraded":"2024-09-11T07:16:23.698137+0000","last_fullsized":"2024-09-11T07:16:23.698137+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:09.964392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:09.964392+0000","last_clean_scrub_stamp":"2024-09-11T07:16:09.964392+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:56:58.109637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.696422+0000","last_change":"2024-09-11T07:16:13.011193+0000","last_active":"2024-09-11T07:16:23.696422+0000","last_peered":"2024-09-11T07:16:23.696422+0000","last_clean":"2024-09-11T07:16:23.696422+0000","last_became_active":"2024-09-11T07:16:13.010864+0000","last_became_peered":"2024-09-11T07:16:13.010864+0000","last_unstale":"2024-09-11T07:16:23.696422+0000","last_undegraded":"2024-09-11T07:16:23.696422+0000","last_fullsized":"2024-09-11T07:16:23.696422+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:11.983377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:11.983377+0000","last_clean_scrub_stamp":"2024-09-11T07:16:11.983377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:01:16.821011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.546098+0000","last_change":"2024-09-11T07:16:15.570339+0000","last_active":"2024-09-11T07:16:23.546098+0000","last_peered":"2024-09-11T07:16:23.546098+0000","last_clean":"2024-09-11T07:16:23.546098+0000","last_became_active":"2024-09-11T07:16:15.570042+0000","last_became_peered":"2024-09-11T07:16:15.570042+0000","last_unstale":"2024-09-11T07:16:23.546098+0000","last_undegraded":"2024-09-11T07:16:23.546098+0000","last_fullsized":"2024-09-11T07:16:23.546098+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:14.001856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:14.001856+0000","last_clean_scrub_stamp":"2024-09-11T07:16:14.001856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:43:13.748820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.697976+0000","last_change":"2024-09-11T07:16:08.975111+0000","last_active":"2024-09-11T07:16:23.697976+0000","last_peered":"2024-09-11T07:16:23.697976+0000","last_clean":"2024-09-11T07:16:23.697976+0000","last_became_active":"2024-09-11T07:16:08.974838+0000","last_became_peered":"2024-09-11T07:16:08.974838+0000","last_unstale":"2024-09-11T07:16:23.697976+0000","last_undegraded":"2024-09-11T07:16:23.697976+0000","last_fullsized":"2024-09-11T07:16:23.697976+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_clean_scrub_stamp":"2024-09-11T07:16:07.961130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:56:43.594262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1c","version":"59'15","reported_seq":41,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.546156+0000","last_change":"2024-09-11T07:16:10.991183+0000","last_active":"2024-09-11T07:16:23.546156+0000","last_peered":"2024-09-11T07:16:23.546156+0000","last_clean":"2024-09-11T07:16:23.546156+0000","last_became_active":"2024-09-11T07:16:10.990850+0000","last_became_peered":"2024-09-11T07:16:10.990850+0000","last_unstale":"2024-09-11T07:16:23.546156+0000","last_undegraded":"2024-09-11T07:16:23.546156+0000","last_fullsized":"2024-09-11T07:16:23.546156+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:09.964392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:09.964392+0000","last_clean_scrub_stamp":"2024-09-11T07:16:09.964392+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:44:28.768280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.680288+0000","last_change":"2024-09-11T07:16:13.014219+0000","last_active":"2024-09-11T07:16:23.680288+0000","last_peered":"2024-09-11T07:16:23.680288+0000","last_clean":"2024-09-11T07:16:23.680288+0000","last_became_active":"2024-09-11T07:16:13.013874+0000","last_became_peered":"2024-09-11T07:16:13.013874+0000","last_unstale":"2024-09-11T07:16:23.680288+0000","last_undegraded":"2024-09-11T07:16:23.680288+0000","last_fullsized":"2024-09-11T07:16:23.680288+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:11.983377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:11.983377+0000","last_clean_scrub_stamp":"2024-09-11T07:16:11.983377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:38:44.453198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.544870+0000","last_change":"2024-09-11T07:16:15.572158+0000","last_active":"2024-09-11T07:16:23.544870+0000","last_peered":"2024-09-11T07:16:23.544870+0000","last_clean":"2024-09-11T07:16:23.544870+0000","last_became_active":"2024-09-11T07:16:15.571809+0000","last_became_peered":"2024-09-11T07:16:15.571809+0000","last_unstale":"2024-09-11T07:16:23.544870+0000","last_undegraded":"2024-09-11T07:16:23.544870+0000","last_fullsized":"2024-09-11T07:16:23.544870+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:14.001856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:14.001856+0000","last_clean_scrub_stamp":"2024-09-11T07:16:14.001856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:50:04.733788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.544146+0000","last_change":"2024-09-11T07:16:08.975787+0000","last_active":"2024-09-11T07:16:23.544146+0000","last_peered":"2024-09-11T07:16:23.544146+0000","last_clean":"2024-09-11T07:16:23.544146+0000","last_became_active":"2024-09-11T07:16:08.975491+0000","last_became_peered":"2024-09-11T07:16:08.975491+0000","last_unstale":"2024-09-11T07:16:23.544146+0000","last_undegraded":"2024-09-11T07:16:23.544146+0000","last_fullsized":"2024-09-11T07:16:23.544146+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:07.961130+0000","last_clean_scrub_stamp":"2024-09-11T07:16:07.961130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:06:20.732700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"59'12","reported_seq":39,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.544534+0000","last_change":"2024-09-11T07:16:10.992964+0000","last_active":"2024-09-11T07:16:23.544534+0000","last_peered":"2024-09-11T07:16:23.544534+0000","last_clean":"2024-09-11T07:16:23.544534+0000","last_became_active":"2024-09-11T07:16:10.992693+0000","last_became_peered":"2024-09-11T07:16:10.992693+0000","last_unstale":"2024-09-11T07:16:23.544534+0000","last_undegraded":"2024-09-11T07:16:23.544534+0000","last_fullsized":"2024-09-11T07:16:23.544534+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:09.964392+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:09.964392+0000","last_clean_scrub_stamp":"2024-09-11T07:16:09.964392+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:56:02.847562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-11T07:16:23.546185+0000","last_change":"2024-09-11T07:16:13.006047+0000","last_active":"2024-09-11T07:16:23.546185+0000","last_peered":"2024-09-11T07:16:23.546185+0000","last_clean":"2024-09-11T07:16:23.546185+0000","last_became_active":"2024-09-11T07:16:13.004006+0000","last_became_peered":"2024-09-11T07:16:13.004006+0000","last_unstale":"2024-09-11T07:16:23.546185+0000","last_undegraded":"2024-09-11T07:16:23.546185+0000","last_fullsized":"2024-09-11T07:16:23.546185+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T07:16:11.983377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T07:16:11.983377+0000","last_clean_scrub_stamp":"2024-09-11T07:16:11.983377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:09:31.446253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":391,"ondisk_log_size":391,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":74,"num_read_kb":69,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1328,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":29,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":48,"seq":206158430229,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298540,"kb_used_data":1188,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446612,"statfs":{"total":95995035648,"available":95689330688,"internally_reserved":0,"allocated":1216512,"data_stored":823879,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:16:59 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.377},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"last":0.49299999999999999}]},{"osd":1,"last update":"Wed Sep 11 07:16:59 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"last":0.40400000000000003},{"interface":"front","average":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":17.690999999999999,"5min":17.690999999999999,"15min":17.690999999999999},"last":0.41099999999999998}]},{"osd":2,"last update":"Wed Sep 11 07:16:59 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.43099999999999999}]},{"osd":3,"last update":"Wed Sep 11 07:16:59 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.624,"5min":0.624,"15min":0.624},"last":0.86399999999999999}]},{"osd":4,"last update":"Wed Sep 11 07:16:59 2024","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.32700000000000001}]},{"osd":5,"last update":"Wed Sep 11 07:16:59 2024","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.62,"5min":0.62,"15min":0.62},"last":0.56499999999999995}]},{"osd":6,"last update":"Wed Sep 11 07:16:59 2024","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.505}]}]},{"osd":6,"up_from":42,"seq":180388626458,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298512,"kb_used_data":1156,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446640,"statfs":{"total":95995035648,"available":95689359360,"internally_reserved":0,"allocated":1183744,"data_stored":821727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:16:34 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.29999999999999999},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.27600000000000002}]},{"osd":1,"last update":"Wed Sep 11 07:16:34 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.35099999999999998}]},{"osd":2,"last update":"Wed Sep 11 07:16:34 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.33700000000000002}]},{"osd":3,"last update":"Wed Sep 11 07:16:34 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.437},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.26200000000000001}]},{"osd":4,"last update":"Wed Sep 11 07:16:34 2024","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.38,"15min":0.38},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"last":0.316},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"last":0.46999999999999997}]},{"osd":5,"last update":"Wed Sep 11 07:16:34 2024","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"last":0.40000000000000002}]},{"osd":7,"last update":"Wed Sep 11 07:16:57 2024","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"last":0.498},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.42599999999999999}]}]},{"osd":5,"up_from":36,"seq":154618822686,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297936,"kb_used_data":580,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447216,"statfs":{"total":95995035648,"available":95689949184,"internally_reserved":0,"allocated":593920,"data_stored":231610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:17:12 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.52100000000000002,"15min":0.52200000000000002},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.497,"5min":0.52500000000000002,"15min":0.53000000000000003},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.66200000000000003,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.33400000000000002}]},{"osd":1,"last update":"Wed Sep 11 07:17:12 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.53000000000000003,"15min":0.53300000000000003},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.53400000000000003,"15min":0.53800000000000003},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.70199999999999996,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.38800000000000001}]},{"osd":2,"last update":"Wed Sep 11 07:17:12 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.51600000000000001,"15min":0.51800000000000002},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.72599999999999998,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.56299999999999994,"15min":0.56499999999999995},"min":{"1min":0.39400000000000002,"5min":0.374,"15min":0.374},"max":{"1min":0.69299999999999995,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.625}]},{"osd":3,"last update":"Wed Sep 11 07:17:12 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.52300000000000002,"15min":0.52100000000000002},"min":{"1min":0.40899999999999997,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.54700000000000004,"15min":0.54400000000000004},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.64700000000000002}]},{"osd":4,"last update":"Wed Sep 11 07:17:12 2024","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.35599999999999998,"15min":0.34799999999999998},"min":{"1min":0.307,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.38400000000000001,"15min":0.38},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.44800000000000001}]},{"osd":6,"last update":"Wed Sep 11 07:16:35 2024","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.38,"15min":0.38},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"last":0.53300000000000003}]},{"osd":7,"last update":"Wed Sep 11 07:17:00 2024","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.69199999999999995}]}]},{"osd":4,"up_from":30,"seq":128849018915,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297964,"kb_used_data":608,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447188,"statfs":{"total":95995035648,"available":95689920512,"internally_reserved":0,"allocated":622592,"data_stored":233000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:16:50 2024","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.54400000000000004,"15min":0.54900000000000004},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.74299999999999999,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.503,"5min":0.54000000000000004,"15min":0.54600000000000004},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.71999999999999997,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.93300000000000005}]},{"osd":1,"last update":"Wed Sep 11 07:16:50 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.54200000000000004,"15min":0.55000000000000004},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.70199999999999996,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.55400000000000005,"15min":0.56000000000000005},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.68000000000000005,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.41299999999999998}]},{"osd":2,"last update":"Wed Sep 11 07:16:50 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.54500000000000004,"15min":0.55300000000000005},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.71399999999999997,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.53300000000000003,"15min":0.53700000000000003},"min":{"1min":0.36299999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.69799999999999995,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.90700000000000003}]},{"osd":3,"last update":"Wed Sep 11 07:16:50 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.58799999999999997,"15min":0.59799999999999998},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.755,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.58299999999999996,"15min":0.58899999999999997},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.77300000000000002,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.501}]},{"osd":5,"last update":"Wed Sep 11 07:17:13 2024","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.36799999999999999,"15min":0.35899999999999999},"min":{"1min":0.19800000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.437,"5min":0.39100000000000001,"15min":0.38300000000000001},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.51200000000000001}]},{"osd":6,"last update":"Wed Sep 11 07:16:37 2024","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.371,"15min":0.371},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.628,"5min":0.628,"15min":0.628},"last":0.374},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"last":0.44400000000000001}]},{"osd":7,"last update":"Wed Sep 11 07:16:57 2024","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"last":0.432},{"interface":"front","average":{"1min":0.439,"5min":0.439,"15min":0.439},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.42299999999999999}]}]},{"osd":3,"up_from":25,"seq":107374182439,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297976,"kb_used_data":620,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447176,"statfs":{"total":95995035648,"available":95689908224,"internally_reserved":0,"allocated":634880,"data_stored":232826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:16:32 2024","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.39500000000000002,"15min":0.39800000000000002},"min":{"1min":0.27100000000000002,"5min":0.246,"15min":0.246},"max":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.40999999999999998,"15min":0.41099999999999998},"min":{"1min":0.20799999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"last":0.34200000000000003}]},{"osd":1,"last update":"Wed Sep 11 07:16:32 2024","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.40500000000000003,"15min":0.40899999999999997},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":0.51800000000000002,"5min":0.57599999999999996,"15min":0.57599999999999996},"last":0.30599999999999999},{"interface":"front","average":{"1min":0.434,"5min":0.44,"15min":0.441},"min":{"1min":0.312,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"last":0.35699999999999998}]},{"osd":2,"last update":"Wed Sep 11 07:16:32 2024","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.46800000000000003,"15min":0.47199999999999998},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.61499999999999999,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.436,"5min":0.46400000000000002,"15min":0.46800000000000003},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.55400000000000005,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.29299999999999998}]},{"osd":4,"last update":"Wed Sep 11 07:16:50 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.61499999999999999,"15min":0.625},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.68799999999999994,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.63300000000000001,"15min":0.64700000000000002},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.73099999999999998,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.41499999999999998}]},{"osd":5,"last update":"Wed Sep 11 07:17:12 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.624,"15min":0.63400000000000001},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.79800000000000004,"5min":1.075,"15min":1.075},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.61299999999999999,"15min":0.621},"min":{"1min":0.39300000000000002,"5min":0.374,"15min":0.374},"max":{"1min":0.68799999999999994,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.497}]},{"osd":6,"last update":"Wed Sep 11 07:16:36 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.41099999999999998}]},{"osd":7,"last update":"Wed Sep 11 07:16:57 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.53400000000000003}]}]},{"osd":2,"up_from":18,"seq":77309411371,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297940,"kb_used_data":584,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447212,"statfs":{"total":95995035648,"available":95689945088,"internally_reserved":0,"allocated":598016,"data_stored":232116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:17:12 2024","interfaces":[{"interface":"back","average":{"1min":0.32500000000000001,"5min":0.33800000000000002,"15min":0.33600000000000002},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.59099999999999997,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.32800000000000001,"5min":0.34000000000000002,"15min":0.33200000000000002},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.51300000000000001,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.27400000000000002}]},{"osd":1,"last update":"Wed Sep 11 07:17:12 2024","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.35399999999999998,"15min":0.34399999999999997},"min":{"1min":0.20699999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.34899999999999998,"5min":0.36599999999999999,"15min":0.35899999999999999},"min":{"1min":0.26500000000000001,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.57399999999999995,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.30299999999999999}]},{"osd":3,"last update":"Wed Sep 11 07:16:33 2024","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.441,"15min":0.44500000000000001},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.53000000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.377},{"interface":"front","average":{"1min":0.40000000000000002,"5min":0.43099999999999999,"15min":0.436},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"last":0.34499999999999997}]},{"osd":4,"last update":"Wed Sep 11 07:16:51 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.56999999999999995,"15min":0.57299999999999995},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.3220000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.56999999999999995,"15min":0.56699999999999995},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":0.46000000000000002}]},{"osd":5,"last update":"Wed Sep 11 07:17:12 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.56899999999999995,"15min":0.57699999999999996},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.56299999999999994,"15min":0.56899999999999995},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.878,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.39500000000000002}]},{"osd":6,"last update":"Wed Sep 11 07:16:34 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.504,"5min":1.504,"15min":1.504},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.434}]},{"osd":7,"last update":"Wed Sep 11 07:16:57 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.248,"5min":1.248,"15min":1.248},"last":0.45000000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574897,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297988,"kb_used_data":632,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447164,"statfs":{"total":95995035648,"available":95689895936,"internally_reserved":0,"allocated":647168,"data_stored":233321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 11 07:16:47 2024","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.30499999999999999,"15min":0.27500000000000002},"min":{"1min":0.221,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"last":0.307},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.33700000000000002,"15min":0.307},"min":{"1min":0.27800000000000002,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"last":0.34300000000000003}]},{"osd":2,"last update":"Wed Sep 11 07:17:07 2024","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.38700000000000001,"15min":0.377},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"last":0.27100000000000002},{"interface":"front","average":{"1min":0.39900000000000002,"5min":0.40799999999999997,"15min":0.40400000000000003},"min":{"1min":0.248,"5min":0.245,"15min":0.245},"max":{"1min":0.56100000000000005,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.37}]},{"osd":3,"last update":"Wed Sep 11 07:16:34 2024","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.42099999999999999,"15min":0.41799999999999998},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.433,"15min":0.42899999999999999},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.628,"5min":0.628,"15min":0.628},"last":0.38500000000000001}]},{"osd":4,"last update":"Wed Sep 11 07:16:54 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.59199999999999997,"15min":0.59499999999999997},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.78300000000000003,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.432},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.58399999999999996,"15min":0.59099999999999997},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.70699999999999996,"5min":1.022,"15min":1.022},"last":0.51100000000000001}]},{"osd":5,"last update":"Wed Sep 11 07:17:18 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.628,"15min":0.63200000000000001},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.91000000000000003,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.626,"15min":0.63200000000000001},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.77200000000000002,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.48899999999999999}]},{"osd":6,"last update":"Wed Sep 11 07:16:34 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.4450000000000001,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.41899999999999998}]},{"osd":7,"last update":"Wed Sep 11 07:16:58 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.44700000000000001}]}]},{"osd":0,"up_from":8,"seq":34359738420,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298532,"kb_used_data":1180,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446620,"statfs":{"total":95995035648,"available":95689338880,"internally_reserved":0,"allocated":1208320,"data_stored":823201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 11 07:16:48 2024","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.33800000000000002,"15min":0.32000000000000001},"min":{"1min":0.251,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.57499999999999996,"5min":0.60899999999999999,"15min":0.60899999999999999},"last":0.26200000000000001},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.35099999999999998,"15min":0.32400000000000001},"min":{"1min":0.24299999999999999,"5min":0.215,"15min":0.215},"max":{"1min":0.67900000000000005,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.40699999999999997}]},{"osd":2,"last update":"Wed Sep 11 07:17:07 2024","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.378,"15min":0.36799999999999999},"min":{"1min":0.20699999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.313},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.379,"15min":0.374},"min":{"1min":0.25900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.48499999999999999,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.30199999999999999}]},{"osd":3,"last update":"Wed Sep 11 07:17:28 2024","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.42899999999999999,"15min":0.433},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.61599999999999999,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.29499999999999998},{"interface":"front","average":{"1min":0.37,"5min":0.42999999999999999,"15min":0.436},"min":{"1min":0.28999999999999998,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.501,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.35899999999999999}]},{"osd":4,"last update":"Wed Sep 11 07:16:55 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.624,"15min":0.64000000000000001},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.84699999999999998,"5min":1.048,"15min":1.048},"last":0.436},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.60899999999999999,"15min":0.625},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.70899999999999996,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.48099999999999998}]},{"osd":5,"last update":"Wed Sep 11 07:17:17 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.61399999999999999,"15min":0.623},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.875,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.64300000000000002,"15min":0.65700000000000003},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.78200000000000003,"5min":1.5429999999999999,"15min":1.5429999999999999},"last":0.50700000000000001}]},{"osd":6,"last update":"Wed Sep 11 07:16:36 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.571,"5min":1.571,"15min":1.571},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.391,"5min":1.391,"15min":1.391},"last":0.58899999999999997}]},{"osd":7,"last update":"Wed Sep 11 07:17:00 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.41999999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":2338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T07:17:34.284 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-11T07:17:34.284 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-11T07:17:34.285 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-11T07:17:34.285 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9 shell --fsid a81cffe6-700c-11ef-bcea-c7b262605968 -- ceph health --format=json 2024-09-11T07:17:34.355 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 systemd[1]: Started Ceph grafana.a for a81cffe6-700c-11ef-bcea-c7b262605968. 2024-09-11T07:17:34.355 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=settings t=2024-09-11T07:17:34.186194277Z level=info msg="Starting Grafana" version=9.4.7 commit=4add91f03d branch=HEAD compiled=2023-03-16T23:56:52Z 2024-09-11T07:17:34.355 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=settings t=2024-09-11T07:17:34.186422719Z level=warn msg="\"sentry\" frontend logging provider is deprecated and will be removed in the next major version. Use \"grafana\" provider instead." 2024-09-11T07:17:34.355 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=settings t=2024-09-11T07:17:34.186462331Z level=info msg="Config loaded from" file=/usr/share/grafana/conf/defaults.ini 2024-09-11T07:17:34.355 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=settings t=2024-09-11T07:17:34.186472094Z level=info msg="Config loaded from" file=/etc/grafana/grafana.ini 2024-09-11T07:17:34.355 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=settings t=2024-09-11T07:17:34.186481371Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_DATA=/var/lib/grafana" 2024-09-11T07:17:34.355 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=settings t=2024-09-11T07:17:34.186489918Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_LOGS=/var/log/grafana" 2024-09-11T07:17:34.356 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=settings t=2024-09-11T07:17:34.186497833Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_PLUGINS=/var/lib/grafana/plugins" 2024-09-11T07:17:34.356 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=settings t=2024-09-11T07:17:34.186505558Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_PROVISIONING=/etc/grafana/provisioning" 2024-09-11T07:17:34.356 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=settings t=2024-09-11T07:17:34.186513488Z level=info msg="Path Home" path=/usr/share/grafana 2024-09-11T07:17:34.356 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=settings t=2024-09-11T07:17:34.18652117Z level=info msg="Path Data" path=/var/lib/grafana 2024-09-11T07:17:34.356 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=settings t=2024-09-11T07:17:34.186528558Z level=info msg="Path Logs" path=/var/log/grafana 2024-09-11T07:17:34.356 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=settings t=2024-09-11T07:17:34.186535911Z level=info msg="Path Plugins" path=/var/lib/grafana/plugins 2024-09-11T07:17:34.356 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=settings t=2024-09-11T07:17:34.18654347Z level=info msg="Path Provisioning" path=/etc/grafana/provisioning 2024-09-11T07:17:34.356 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=settings t=2024-09-11T07:17:34.186551272Z level=info msg="App mode production" 2024-09-11T07:17:34.356 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=sqlstore t=2024-09-11T07:17:34.186629897Z level=info msg="Connecting to DB" dbtype=sqlite3 2024-09-11T07:17:34.356 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=sqlstore t=2024-09-11T07:17:34.186654416Z level=warn msg="SQLite database file has broader permissions than it should" path=/var/lib/grafana/grafana.db mode=-rw-r--r-- expected=-rw-r----- 2024-09-11T07:17:34.356 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.187505118Z level=info msg="Starting DB migrations" 2024-09-11T07:17:34.356 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.187597834Z level=info msg="Executing migration" id="create migration_log table" 2024-09-11T07:17:34.357 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.188114785Z level=info msg="Executing migration" id="create user table" 2024-09-11T07:17:34.357 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.188630295Z level=info msg="Executing migration" id="add unique index user.login" 2024-09-11T07:17:34.357 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.191318868Z level=info msg="Executing migration" id="add unique index user.email" 2024-09-11T07:17:34.358 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.194230745Z level=info msg="Executing migration" id="drop index UQE_user_login - v1" 2024-09-11T07:17:34.358 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.196933119Z level=info msg="Executing migration" id="drop index UQE_user_email - v1" 2024-09-11T07:17:34.358 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.199352402Z level=info msg="Executing migration" id="Rename table user to user_v1 - v1" 2024-09-11T07:17:34.358 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.202281917Z level=info msg="Executing migration" id="create user table v2" 2024-09-11T07:17:34.358 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.20269256Z level=info msg="Executing migration" id="create index UQE_user_login - v2" 2024-09-11T07:17:34.358 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.205330659Z level=info msg="Executing migration" id="create index UQE_user_email - v2" 2024-09-11T07:17:34.358 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.205810008Z level=info msg="Executing migration" id="copy data_source v1 to v2" 2024-09-11T07:17:34.358 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.208170981Z level=info msg="Executing migration" id="Drop old table user_v1" 2024-09-11T07:17:34.358 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.208807307Z level=info msg="Executing migration" id="Add column help_flags1 to user table" 2024-09-11T07:17:34.358 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.21098272Z level=info msg="Executing migration" id="Update user table charset" 2024-09-11T07:17:34.358 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.211443788Z level=info msg="Executing migration" id="Add last_seen_at column to user" 2024-09-11T07:17:34.358 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.213582359Z level=info msg="Executing migration" id="Add missing user data" 2024-09-11T07:17:34.359 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.214091097Z level=info msg="Executing migration" id="Add is_disabled column to user" 2024-09-11T07:17:34.359 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.216433098Z level=info msg="Executing migration" id="Add index user.login/user.email" 2024-09-11T07:17:34.359 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.219054579Z level=info msg="Executing migration" id="Add is_service_account column to user" 2024-09-11T07:17:34.359 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.221601787Z level=info msg="Executing migration" id="Update is_service_account column to nullable" 2024-09-11T07:17:34.359 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.224447983Z level=info msg="Executing migration" id="create temp user table v1-7" 2024-09-11T07:17:34.359 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.227107869Z level=info msg="Executing migration" id="create index IDX_temp_user_email - v1-7" 2024-09-11T07:17:34.359 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.229773444Z level=info msg="Executing migration" id="create index IDX_temp_user_org_id - v1-7" 2024-09-11T07:17:34.359 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.230280132Z level=info msg="Executing migration" id="create index IDX_temp_user_code - v1-7" 2024-09-11T07:17:34.359 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.232920833Z level=info msg="Executing migration" id="create index IDX_temp_user_status - v1-7" 2024-09-11T07:17:34.359 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.23559444Z level=info msg="Executing migration" id="Update temp_user table charset" 2024-09-11T07:17:34.359 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.238108467Z level=info msg="Executing migration" id="drop index IDX_temp_user_email - v1" 2024-09-11T07:17:34.359 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.240808252Z level=info msg="Executing migration" id="drop index IDX_temp_user_org_id - v1" 2024-09-11T07:17:34.359 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.243428816Z level=info msg="Executing migration" id="drop index IDX_temp_user_code - v1" 2024-09-11T07:17:34.360 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.246270729Z level=info msg="Executing migration" id="drop index IDX_temp_user_status - v1" 2024-09-11T07:17:34.360 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.248749577Z level=info msg="Executing migration" id="Rename table temp_user to temp_user_tmp_qwerty - v1" 2024-09-11T07:17:34.360 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.249603613Z level=info msg="Executing migration" id="create temp_user v2" 2024-09-11T07:17:34.360 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.25038955Z level=info msg="Executing migration" id="create index IDX_temp_user_email - v2" 2024-09-11T07:17:34.360 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.251382806Z level=info msg="Executing migration" id="create index IDX_temp_user_org_id - v2" 2024-09-11T07:17:34.360 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.252201269Z level=info msg="Executing migration" id="create index IDX_temp_user_code - v2" 2024-09-11T07:17:34.360 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.252915183Z level=info msg="Executing migration" id="create index IDX_temp_user_status - v2" 2024-09-11T07:17:34.360 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.253775872Z level=info msg="Executing migration" id="copy temp_user v1 to v2" 2024-09-11T07:17:34.360 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.254549437Z level=info msg="Executing migration" id="drop temp_user_tmp_qwerty" 2024-09-11T07:17:34.360 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.255371091Z level=info msg="Executing migration" id="Set created for temp users that will otherwise prematurely expire" 2024-09-11T07:17:34.360 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.25626634Z level=info msg="Executing migration" id="create star table" 2024-09-11T07:17:34.360 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.256643136Z level=info msg="Executing migration" id="add unique index star.user_id_dashboard_id" 2024-09-11T07:17:34.361 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.257467446Z level=info msg="Executing migration" id="create org table v1" 2024-09-11T07:17:34.361 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.258292432Z level=info msg="Executing migration" id="create index UQE_org_name - v1" 2024-09-11T07:17:34.361 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.259276589Z level=info msg="Executing migration" id="create org_user table v1" 2024-09-11T07:17:34.361 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.259916403Z level=info msg="Executing migration" id="create index IDX_org_user_org_id - v1" 2024-09-11T07:17:34.361 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.260830097Z level=info msg="Executing migration" id="create index UQE_org_user_org_id_user_id - v1" 2024-09-11T07:17:34.361 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.261468057Z level=info msg="Executing migration" id="create index IDX_org_user_user_id - v1" 2024-09-11T07:17:34.361 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.261864795Z level=info msg="Executing migration" id="Update org table charset" 2024-09-11T07:17:34.361 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.262793048Z level=info msg="Executing migration" id="Update org_user table charset" 2024-09-11T07:17:34.361 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.263101549Z level=info msg="Executing migration" id="Migrate all Read Only Viewers to Viewers" 2024-09-11T07:17:34.361 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.263641116Z level=info msg="Executing migration" id="create dashboard table" 2024-09-11T07:17:34.361 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.264576269Z level=info msg="Executing migration" id="add index dashboard.account_id" 2024-09-11T07:17:34.361 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.265140551Z level=info msg="Executing migration" id="add unique index dashboard_account_id_slug" 2024-09-11T07:17:34.361 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.265943953Z level=info msg="Executing migration" id="create dashboard_tag table" 2024-09-11T07:17:34.362 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.266929604Z level=info msg="Executing migration" id="add unique index dashboard_tag.dasboard_id_term" 2024-09-11T07:17:34.362 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.267681403Z level=info msg="Executing migration" id="drop index UQE_dashboard_tag_dashboard_id_term - v1" 2024-09-11T07:17:34.362 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.268091074Z level=info msg="Executing migration" id="Rename table dashboard to dashboard_v1 - v1" 2024-09-11T07:17:34.362 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.268957197Z level=info msg="Executing migration" id="create dashboard v2" 2024-09-11T07:17:34.362 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.26978953Z level=info msg="Executing migration" id="create index IDX_dashboard_org_id - v2" 2024-09-11T07:17:34.362 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.270724902Z level=info msg="Executing migration" id="create index UQE_dashboard_org_id_slug - v2" 2024-09-11T07:17:34.362 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.271171945Z level=info msg="Executing migration" id="copy dashboard v1 to v2" 2024-09-11T07:17:34.362 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.272176754Z level=info msg="Executing migration" id="drop table dashboard_v1" 2024-09-11T07:17:34.362 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.27281115Z level=info msg="Executing migration" id="alter dashboard.data to mediumtext v1" 2024-09-11T07:17:34.362 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.273711107Z level=info msg="Executing migration" id="Add column updated_by in dashboard - v2" 2024-09-11T07:17:34.362 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.2743148Z level=info msg="Executing migration" id="Add column created_by in dashboard - v2" 2024-09-11T07:17:34.362 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.274792598Z level=info msg="Executing migration" id="Add column gnetId in dashboard" 2024-09-11T07:17:34.362 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.27764926Z level=info msg="Executing migration" id="Add index for gnetId in dashboard" 2024-09-11T07:17:34.362 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.278083269Z level=info msg="Executing migration" id="Add column plugin_id in dashboard" 2024-09-11T07:17:34.363 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.280512899Z level=info msg="Executing migration" id="Add index for plugin_id in dashboard" 2024-09-11T07:17:34.363 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.28325858Z level=info msg="Executing migration" id="Add index for dashboard_id in dashboard_tag" 2024-09-11T07:17:34.363 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.283677014Z level=info msg="Executing migration" id="Update dashboard table charset" 2024-09-11T07:17:34.363 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.286500488Z level=info msg="Executing migration" id="Update dashboard_tag table charset" 2024-09-11T07:17:34.363 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.287023991Z level=info msg="Executing migration" id="Add column folder_id in dashboard" 2024-09-11T07:17:34.363 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.289151762Z level=info msg="Executing migration" id="Add column isFolder in dashboard" 2024-09-11T07:17:34.363 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.292096022Z level=info msg="Executing migration" id="Add column has_acl in dashboard" 2024-09-11T07:17:34.363 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.292741362Z level=info msg="Executing migration" id="Add column uid in dashboard" 2024-09-11T07:17:34.363 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.29542378Z level=info msg="Executing migration" id="Update uid column values in dashboard" 2024-09-11T07:17:34.363 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.295759295Z level=info msg="Executing migration" id="Add unique index dashboard_org_id_uid" 2024-09-11T07:17:34.363 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.298437088Z level=info msg="Executing migration" id="Remove unique index org_id_slug" 2024-09-11T07:17:34.364 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.301015784Z level=info msg="Executing migration" id="Update dashboard title length" 2024-09-11T07:17:34.364 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.301339628Z level=info msg="Executing migration" id="Add unique index for dashboard_org_id_title_folder_id" 2024-09-11T07:17:34.364 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.303872861Z level=info msg="Executing migration" id="create dashboard_provisioning" 2024-09-11T07:17:34.364 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.306681082Z level=info msg="Executing migration" id="Rename table dashboard_provisioning to dashboard_provisioning_tmp_qwerty - v1" 2024-09-11T07:17:34.364 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.309467543Z level=info msg="Executing migration" id="create dashboard_provisioning v2" 2024-09-11T07:17:34.364 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.309879783Z level=info msg="Executing migration" id="create index IDX_dashboard_provisioning_dashboard_id - v2" 2024-09-11T07:17:34.364 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.312510727Z level=info msg="Executing migration" id="create index IDX_dashboard_provisioning_dashboard_id_name - v2" 2024-09-11T07:17:34.364 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.315249019Z level=info msg="Executing migration" id="copy dashboard_provisioning v1 to v2" 2024-09-11T07:17:34.364 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.317903092Z level=info msg="Executing migration" id="drop dashboard_provisioning_tmp_qwerty" 2024-09-11T07:17:34.365 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.318278213Z level=info msg="Executing migration" id="Add check_sum column" 2024-09-11T07:17:34.365 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.321107219Z level=info msg="Executing migration" id="Add index for dashboard_title" 2024-09-11T07:17:34.365 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.323673887Z level=info msg="Executing migration" id="delete tags for deleted dashboards" 2024-09-11T07:17:34.365 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.326502426Z level=info msg="Executing migration" id="delete stars for deleted dashboards" 2024-09-11T07:17:34.365 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.326989088Z level=info msg="Executing migration" id="Add index for dashboard_is_folder" 2024-09-11T07:17:34.365 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.329810207Z level=info msg="Executing migration" id="Add isPublic for dashboard" 2024-09-11T07:17:34.365 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.332192899Z level=info msg="Executing migration" id="create data_source table" 2024-09-11T07:17:34.365 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.334796467Z level=info msg="Executing migration" id="add index data_source.account_id" 2024-09-11T07:17:34.365 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.33534529Z level=info msg="Executing migration" id="add unique index data_source.account_id_name" 2024-09-11T07:17:34.365 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.337982766Z level=info msg="Executing migration" id="drop index IDX_data_source_account_id - v1" 2024-09-11T07:17:34.365 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.340583293Z level=info msg="Executing migration" id="drop index UQE_data_source_account_id_name - v1" 2024-09-11T07:17:34.365 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.343248512Z level=info msg="Executing migration" id="Rename table data_source to data_source_v1 - v1" 2024-09-11T07:17:34.365 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.34434076Z level=info msg="Executing migration" id="create data_source table v2" 2024-09-11T07:17:34.366 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.345134613Z level=info msg="Executing migration" id="create index IDX_data_source_org_id - v2" 2024-09-11T07:17:34.366 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.346057934Z level=info msg="Executing migration" id="create index UQE_data_source_org_id_name - v2" 2024-09-11T07:17:34.366 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.346845931Z level=info msg="Executing migration" id="Drop old table data_source_v1 #2" 2024-09-11T07:17:34.366 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.347223847Z level=info msg="Executing migration" id="Add column with_credentials" 2024-09-11T07:17:34.366 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.347799424Z level=info msg="Executing migration" id="Add secure json data column" 2024-09-11T07:17:34.366 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.348338923Z level=info msg="Executing migration" id="Update data_source table charset" 2024-09-11T07:17:34.366 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.349198024Z level=info msg="Executing migration" id="Update initial version to 1" 2024-09-11T07:17:34.366 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.350138219Z level=info msg="Executing migration" id="Add read_only data column" 2024-09-11T07:17:34.366 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.350763382Z level=info msg="Executing migration" id="Migrate logging ds to loki ds" 2024-09-11T07:17:34.366 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.351556831Z level=info msg="Executing migration" id="Update json_data with nulls" 2024-09-11T07:17:34.366 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.351905649Z level=info msg="Executing migration" id="Add uid column" 2024-09-11T07:17:34.366 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.352448401Z level=info msg="Executing migration" id="Update uid value" 2024-09-11T07:17:34.366 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.353349882Z level=info msg="Executing migration" id="Add unique index datasource_org_id_uid" 2024-09-11T07:17:34.366 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.353913644Z level=info msg="Executing migration" id="add unique index datasource_org_id_is_default" 2024-09-11T07:17:34.367 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.354743092Z level=info msg="Executing migration" id="create api_key table" 2024-09-11T07:17:34.367 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.355588386Z level=info msg="Executing migration" id="add index api_key.account_id" 2024-09-11T07:17:34.367 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.356429383Z level=info msg="Executing migration" id="add index api_key.key" 2024-09-11T07:17:34.367 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.356987311Z level=info msg="Executing migration" id="add index api_key.account_id_name" 2024-09-11T07:17:34.367 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.357831453Z level=info msg="Executing migration" id="drop index IDX_api_key_account_id - v1" 2024-09-11T07:17:34.554 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/mon.a/config 2024-09-11T07:17:34.606 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-iscsi-iscsi-a[58197]: debug there is no tcmu-runner data available 2024-09-11T07:17:34.607 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.358658733Z level=info msg="Executing migration" id="drop index UQE_api_key_key - v1" 2024-09-11T07:17:34.607 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.359536744Z level=info msg="Executing migration" id="drop index UQE_api_key_account_id_name - v1" 2024-09-11T07:17:34.607 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.360396376Z level=info msg="Executing migration" id="Rename table api_key to api_key_v1 - v1" 2024-09-11T07:17:34.607 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.361423871Z level=info msg="Executing migration" id="create api_key table v2" 2024-09-11T07:17:34.607 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.364981522Z level=info msg="Executing migration" id="create index IDX_api_key_org_id - v2" 2024-09-11T07:17:34.607 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.36552956Z level=info msg="Executing migration" id="create index UQE_api_key_key - v2" 2024-09-11T07:17:34.607 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.365940772Z level=info msg="Executing migration" id="create index UQE_api_key_org_id_name - v2" 2024-09-11T07:17:34.607 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.366355443Z level=info msg="Executing migration" id="copy api_key v1 to v2" 2024-09-11T07:17:34.607 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.366674496Z level=info msg="Executing migration" id="Drop old table api_key_v1" 2024-09-11T07:17:34.607 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.367017119Z level=info msg="Executing migration" id="Update api_key table charset" 2024-09-11T07:17:34.607 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.367319823Z level=info msg="Executing migration" id="Add expires to api_key table" 2024-09-11T07:17:34.608 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.367872249Z level=info msg="Executing migration" id="Add service account foreign key" 2024-09-11T07:17:34.608 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.368431834Z level=info msg="Executing migration" id="set service account foreign key to nil if 0" 2024-09-11T07:17:34.608 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.36873543Z level=info msg="Executing migration" id="Add last_used_at to api_key table" 2024-09-11T07:17:34.608 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.369254898Z level=info msg="Executing migration" id="Add is_revoked column to api_key table" 2024-09-11T07:17:34.608 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.369815138Z level=info msg="Executing migration" id="create dashboard_snapshot table v4" 2024-09-11T07:17:34.608 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.370170884Z level=info msg="Executing migration" id="drop table dashboard_snapshot_v4 #1" 2024-09-11T07:17:34.608 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.370581275Z level=info msg="Executing migration" id="create dashboard_snapshot table v5 #2" 2024-09-11T07:17:34.608 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.370933643Z level=info msg="Executing migration" id="create index UQE_dashboard_snapshot_key - v5" 2024-09-11T07:17:34.608 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.371384953Z level=info msg="Executing migration" id="create index UQE_dashboard_snapshot_delete_key - v5" 2024-09-11T07:17:34.608 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.371848735Z level=info msg="Executing migration" id="create index IDX_dashboard_snapshot_user_id - v5" 2024-09-11T07:17:34.608 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.372261129Z level=info msg="Executing migration" id="alter dashboard_snapshot to mediumtext v2" 2024-09-11T07:17:34.608 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.372551634Z level=info msg="Executing migration" id="Update dashboard_snapshot table charset" 2024-09-11T07:17:34.609 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.37280831Z level=info msg="Executing migration" id="Add column external_delete_url to dashboard_snapshots table" 2024-09-11T07:17:34.609 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.373400427Z level=info msg="Executing migration" id="Add encrypted dashboard json column" 2024-09-11T07:17:34.609 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.373960761Z level=info msg="Executing migration" id="Change dashboard_encrypted column to MEDIUMBLOB" 2024-09-11T07:17:34.609 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.374232093Z level=info msg="Executing migration" id="create quota table v1" 2024-09-11T07:17:34.609 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.374668608Z level=info msg="Executing migration" id="create index UQE_quota_org_id_user_id_target - v1" 2024-09-11T07:17:34.609 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.375103711Z level=info msg="Executing migration" id="Update quota table charset" 2024-09-11T07:17:34.609 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.375399282Z level=info msg="Executing migration" id="create plugin_setting table" 2024-09-11T07:17:34.609 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.375811827Z level=info msg="Executing migration" id="create index UQE_plugin_setting_org_id_plugin_id - v1" 2024-09-11T07:17:34.609 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.376215066Z level=info msg="Executing migration" id="Add column plugin_version to plugin_settings" 2024-09-11T07:17:34.609 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.376830902Z level=info msg="Executing migration" id="Update plugin_setting table charset" 2024-09-11T07:17:34.609 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.377094866Z level=info msg="Executing migration" id="create session table" 2024-09-11T07:17:34.609 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.377530552Z level=info msg="Executing migration" id="Drop old table playlist table" 2024-09-11T07:17:34.609 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.377794212Z level=info msg="Executing migration" id="Drop old table playlist_item table" 2024-09-11T07:17:34.610 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.378056131Z level=info msg="Executing migration" id="create playlist table v2" 2024-09-11T07:17:34.610 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.37843963Z level=info msg="Executing migration" id="create playlist item table v2" 2024-09-11T07:17:34.611 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.378852279Z level=info msg="Executing migration" id="Update playlist table charset" 2024-09-11T07:17:34.611 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.379104127Z level=info msg="Executing migration" id="Update playlist_item table charset" 2024-09-11T07:17:34.611 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.379402377Z level=info msg="Executing migration" id="drop preferences table v2" 2024-09-11T07:17:34.611 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.379657503Z level=info msg="Executing migration" id="drop preferences table v3" 2024-09-11T07:17:34.611 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.37993569Z level=info msg="Executing migration" id="create preferences table v3" 2024-09-11T07:17:34.611 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.380345617Z level=info msg="Executing migration" id="Update preferences table charset" 2024-09-11T07:17:34.611 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.380653745Z level=info msg="Executing migration" id="Add column team_id in preferences" 2024-09-11T07:17:34.611 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.381237501Z level=info msg="Executing migration" id="Update team_id column values in preferences" 2024-09-11T07:17:34.611 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.381548493Z level=info msg="Executing migration" id="Add column week_start in preferences" 2024-09-11T07:17:34.612 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.382095525Z level=info msg="Executing migration" id="Add column preferences.json_data" 2024-09-11T07:17:34.612 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.382694449Z level=info msg="Executing migration" id="alter preferences.json_data to mediumtext v1" 2024-09-11T07:17:34.612 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.382963048Z level=info msg="Executing migration" id="Add preferences index org_id" 2024-09-11T07:17:34.612 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.383395893Z level=info msg="Executing migration" id="Add preferences index user_id" 2024-09-11T07:17:34.612 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.383875829Z level=info msg="Executing migration" id="create alert table v1" 2024-09-11T07:17:34.612 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.386670046Z level=info msg="Executing migration" id="add index alert org_id & id " 2024-09-11T07:17:34.612 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.389401904Z level=info msg="Executing migration" id="add index alert state" 2024-09-11T07:17:34.612 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.39206674Z level=info msg="Executing migration" id="add index alert dashboard_id" 2024-09-11T07:17:34.612 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.392505466Z level=info msg="Executing migration" id="Create alert_rule_tag table v1" 2024-09-11T07:17:34.612 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.392872824Z level=info msg="Executing migration" id="Add unique index alert_rule_tag.alert_id_tag_id" 2024-09-11T07:17:34.612 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.393309678Z level=info msg="Executing migration" id="drop index UQE_alert_rule_tag_alert_id_tag_id - v1" 2024-09-11T07:17:34.612 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.3937165Z level=info msg="Executing migration" id="Rename table alert_rule_tag to alert_rule_tag_v1 - v1" 2024-09-11T07:17:34.612 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.395242654Z level=info msg="Executing migration" id="Create alert_rule_tag table v2" 2024-09-11T07:17:34.612 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.395651982Z level=info msg="Executing migration" id="create index UQE_alert_rule_tag_alert_id_tag_id - Add unique index alert_rule_tag.alert_id_tag_id V2" 2024-09-11T07:17:34.613 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.396064903Z level=info msg="Executing migration" id="copy alert_rule_tag v1 to v2" 2024-09-11T07:17:34.613 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.39638122Z level=info msg="Executing migration" id="drop table alert_rule_tag_v1" 2024-09-11T07:17:34.613 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.39677298Z level=info msg="Executing migration" id="create alert_notification table v1" 2024-09-11T07:17:34.613 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.397131794Z level=info msg="Executing migration" id="Add column is_default" 2024-09-11T07:17:34.613 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.397797157Z level=info msg="Executing migration" id="Add column frequency" 2024-09-11T07:17:34.613 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.398417855Z level=info msg="Executing migration" id="Add column send_reminder" 2024-09-11T07:17:34.613 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.399093643Z level=info msg="Executing migration" id="Add column disable_resolve_message" 2024-09-11T07:17:34.613 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.399789229Z level=info msg="Executing migration" id="add index alert_notification org_id & name" 2024-09-11T07:17:34.613 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.40021936Z level=info msg="Executing migration" id="Update alert table charset" 2024-09-11T07:17:34.613 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.400508143Z level=info msg="Executing migration" id="Update alert_notification table charset" 2024-09-11T07:17:34.613 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.400766013Z level=info msg="Executing migration" id="create notification_journal table v1" 2024-09-11T07:17:34.613 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.401152972Z level=info msg="Executing migration" id="add index notification_journal org_id & alert_id & notifier_id" 2024-09-11T07:17:34.613 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.401612789Z level=info msg="Executing migration" id="drop alert_notification_journal" 2024-09-11T07:17:34.613 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.401956854Z level=info msg="Executing migration" id="create alert_notification_state table v1" 2024-09-11T07:17:34.614 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.40239516Z level=info msg="Executing migration" id="add index alert_notification_state org_id & alert_id & notifier_id" 2024-09-11T07:17:34.614 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.405655484Z level=info msg="Executing migration" id="Add for to alert table" 2024-09-11T07:17:34.614 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.408546526Z level=info msg="Executing migration" id="Add column uid in alert_notification" 2024-09-11T07:17:34.614 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.411362017Z level=info msg="Executing migration" id="Update uid column values in alert_notification" 2024-09-11T07:17:34.614 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.414254514Z level=info msg="Executing migration" id="Add unique index alert_notification_org_id_uid" 2024-09-11T07:17:34.614 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.417249419Z level=info msg="Executing migration" id="Remove unique index org_id_name" 2024-09-11T07:17:34.614 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.420028671Z level=info msg="Executing migration" id="Add column secure_settings in alert_notification" 2024-09-11T07:17:34.614 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.422747119Z level=info msg="Executing migration" id="alter alert.settings to mediumtext" 2024-09-11T07:17:34.614 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.425609169Z level=info msg="Executing migration" id="Add non-unique index alert_notification_state_alert_id" 2024-09-11T07:17:34.614 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.428203211Z level=info msg="Executing migration" id="Add non-unique index alert_rule_tag_alert_id" 2024-09-11T07:17:34.614 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.431055788Z level=info msg="Executing migration" id="Drop old annotation table v4" 2024-09-11T07:17:34.614 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.433865184Z level=info msg="Executing migration" id="create annotation table v5" 2024-09-11T07:17:34.614 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.43646654Z level=info msg="Executing migration" id="add index annotation 0 v3" 2024-09-11T07:17:34.614 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.439340771Z level=info msg="Executing migration" id="add index annotation 1 v3" 2024-09-11T07:17:34.614 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.441902578Z level=info msg="Executing migration" id="add index annotation 2 v3" 2024-09-11T07:17:34.615 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.444812814Z level=info msg="Executing migration" id="add index annotation 3 v3" 2024-09-11T07:17:34.615 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.447446239Z level=info msg="Executing migration" id="add index annotation 4 v3" 2024-09-11T07:17:34.615 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.449993926Z level=info msg="Executing migration" id="Update annotation table charset" 2024-09-11T07:17:34.615 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.452671432Z level=info msg="Executing migration" id="Add column region_id to annotation table" 2024-09-11T07:17:34.615 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.455415624Z level=info msg="Executing migration" id="Drop category_id index" 2024-09-11T07:17:34.615 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.458402173Z level=info msg="Executing migration" id="Add column tags to annotation table" 2024-09-11T07:17:34.615 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.461204796Z level=info msg="Executing migration" id="Create annotation_tag table v2" 2024-09-11T07:17:34.615 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.463921942Z level=info msg="Executing migration" id="Add unique index annotation_tag.annotation_id_tag_id" 2024-09-11T07:17:34.615 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.466516713Z level=info msg="Executing migration" id="drop index UQE_annotation_tag_annotation_id_tag_id - v2" 2024-09-11T07:17:34.615 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.466966717Z level=info msg="Executing migration" id="Rename table annotation_tag to annotation_tag_v2 - v2" 2024-09-11T07:17:34.615 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.469703818Z level=info msg="Executing migration" id="Create annotation_tag table v3" 2024-09-11T07:17:34.615 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.47252397Z level=info msg="Executing migration" id="create index UQE_annotation_tag_annotation_id_tag_id - Add unique index annotation_tag.annotation_id_tag_id V3" 2024-09-11T07:17:34.615 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.475204951Z level=info msg="Executing migration" id="copy annotation_tag v2 to v3" 2024-09-11T07:17:34.615 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.475665815Z level=info msg="Executing migration" id="drop table annotation_tag_v2" 2024-09-11T07:17:34.615 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.478259815Z level=info msg="Executing migration" id="Update alert annotations and set TEXT to empty" 2024-09-11T07:17:34.616 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.478858999Z level=info msg="Executing migration" id="Add created time to annotation table" 2024-09-11T07:17:34.616 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.480944183Z level=info msg="Executing migration" id="Add updated time to annotation table" 2024-09-11T07:17:34.616 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.481759041Z level=info msg="Executing migration" id="Add index for created in annotation table" 2024-09-11T07:17:34.616 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.482212856Z level=info msg="Executing migration" id="Add index for updated in annotation table" 2024-09-11T07:17:34.616 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.48306559Z level=info msg="Executing migration" id="Convert existing annotations from seconds to milliseconds" 2024-09-11T07:17:34.616 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.483871068Z level=info msg="Executing migration" id="Add epoch_end column" 2024-09-11T07:17:34.616 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.484692553Z level=info msg="Executing migration" id="Add index for epoch_end" 2024-09-11T07:17:34.616 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.485164424Z level=info msg="Executing migration" id="Make epoch_end the same as epoch" 2024-09-11T07:17:34.616 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.485980766Z level=info msg="Executing migration" id="Move region to single row" 2024-09-11T07:17:34.616 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.486924222Z level=info msg="Executing migration" id="Remove index org_id_epoch from annotation table" 2024-09-11T07:17:34.616 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.487630064Z level=info msg="Executing migration" id="Remove index org_id_dashboard_id_panel_id_epoch from annotation table" 2024-09-11T07:17:34.616 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.488051117Z level=info msg="Executing migration" id="Add index for org_id_dashboard_id_epoch_end_epoch on annotation table" 2024-09-11T07:17:34.616 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.488862625Z level=info msg="Executing migration" id="Add index for org_id_epoch_end_epoch on annotation table" 2024-09-11T07:17:34.616 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.489658837Z level=info msg="Executing migration" id="Remove index org_id_epoch_epoch_end from annotation table" 2024-09-11T07:17:34.617 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.490414241Z level=info msg="Executing migration" id="Add index for alert_id on annotation table" 2024-09-11T07:17:34.617 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.490856219Z level=info msg="Executing migration" id="Increase tags column to length 4096" 2024-09-11T07:17:34.617 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.491630471Z level=info msg="Executing migration" id="create test_data table" 2024-09-11T07:17:34.617 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.492065325Z level=info msg="Executing migration" id="create dashboard_version table v1" 2024-09-11T07:17:34.617 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.492489167Z level=info msg="Executing migration" id="add index dashboard_version.dashboard_id" 2024-09-11T07:17:34.617 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.493339849Z level=info msg="Executing migration" id="add unique index dashboard_version.dashboard_id and dashboard_version.version" 2024-09-11T07:17:34.617 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.493787287Z level=info msg="Executing migration" id="Set dashboard version to 1 where 0" 2024-09-11T07:17:34.617 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.494591946Z level=info msg="Executing migration" id="save existing dashboard data in dashboard_version table v1" 2024-09-11T07:17:34.617 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.495017426Z level=info msg="Executing migration" id="alter dashboard_version.data to mediumtext v1" 2024-09-11T07:17:34.617 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.495489017Z level=info msg="Executing migration" id="create team table" 2024-09-11T07:17:34.617 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.496347615Z level=info msg="Executing migration" id="add index team.org_id" 2024-09-11T07:17:34.617 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.496832344Z level=info msg="Executing migration" id="add unique index team_org_id_name" 2024-09-11T07:17:34.617 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.497627781Z level=info msg="Executing migration" id="create team member table" 2024-09-11T07:17:34.617 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.498407986Z level=info msg="Executing migration" id="add index team_member.org_id" 2024-09-11T07:17:34.618 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.499279857Z level=info msg="Executing migration" id="add unique index team_member_org_id_team_id_user_id" 2024-09-11T07:17:34.618 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.499753353Z level=info msg="Executing migration" id="add index team_member.team_id" 2024-09-11T07:17:34.618 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.50068207Z level=info msg="Executing migration" id="Add column email to team table" 2024-09-11T07:17:34.618 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.501517272Z level=info msg="Executing migration" id="Add column external to team_member table" 2024-09-11T07:17:34.618 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.502309889Z level=info msg="Executing migration" id="Add column permission to team_member table" 2024-09-11T07:17:34.618 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.503056654Z level=info msg="Executing migration" id="create dashboard acl table" 2024-09-11T07:17:34.618 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.503938252Z level=info msg="Executing migration" id="add index dashboard_acl_dashboard_id" 2024-09-11T07:17:34.618 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.504813064Z level=info msg="Executing migration" id="add unique index dashboard_acl_dashboard_id_user_id" 2024-09-11T07:17:34.618 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.505663095Z level=info msg="Executing migration" id="add unique index dashboard_acl_dashboard_id_team_id" 2024-09-11T07:17:34.618 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.506098776Z level=info msg="Executing migration" id="add index dashboard_acl_user_id" 2024-09-11T07:17:34.618 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.507095007Z level=info msg="Executing migration" id="add index dashboard_acl_team_id" 2024-09-11T07:17:34.618 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.507792692Z level=info msg="Executing migration" id="add index dashboard_acl_org_id_role" 2024-09-11T07:17:34.618 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.508632642Z level=info msg="Executing migration" id="add index dashboard_permission" 2024-09-11T07:17:34.618 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.509610925Z level=info msg="Executing migration" id="save default acl rules in dashboard_acl table" 2024-09-11T07:17:34.618 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.510318486Z level=info msg="Executing migration" id="delete acl rules for deleted dashboards and folders" 2024-09-11T07:17:34.618 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.511214658Z level=info msg="Executing migration" id="create tag table" 2024-09-11T07:17:34.619 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.512148912Z level=info msg="Executing migration" id="add index tag.key_value" 2024-09-11T07:17:34.619 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.512921372Z level=info msg="Executing migration" id="create login attempt table" 2024-09-11T07:17:34.619 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.513747757Z level=info msg="Executing migration" id="add index login_attempt.username" 2024-09-11T07:17:34.619 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.514737915Z level=info msg="Executing migration" id="drop index IDX_login_attempt_username - v1" 2024-09-11T07:17:34.619 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.51544872Z level=info msg="Executing migration" id="Rename table login_attempt to login_attempt_tmp_qwerty - v1" 2024-09-11T07:17:34.619 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.517259503Z level=info msg="Executing migration" id="create login_attempt v2" 2024-09-11T07:17:34.619 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.518107514Z level=info msg="Executing migration" id="create index IDX_login_attempt_username - v2" 2024-09-11T07:17:34.619 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.521185352Z level=info msg="Executing migration" id="copy login_attempt v1 to v2" 2024-09-11T07:17:34.619 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.523789429Z level=info msg="Executing migration" id="drop login_attempt_tmp_qwerty" 2024-09-11T07:17:34.619 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.526632178Z level=info msg="Executing migration" id="create user auth table" 2024-09-11T07:17:34.619 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.529480838Z level=info msg="Executing migration" id="create index IDX_user_auth_auth_module_auth_id - v1" 2024-09-11T07:17:34.619 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.532107979Z level=info msg="Executing migration" id="alter user_auth.auth_id to length 190" 2024-09-11T07:17:34.619 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.534898035Z level=info msg="Executing migration" id="Add OAuth access token to user_auth" 2024-09-11T07:17:34.620 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.53769511Z level=info msg="Executing migration" id="Add OAuth refresh token to user_auth" 2024-09-11T07:17:34.620 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.540611659Z level=info msg="Executing migration" id="Add OAuth token type to user_auth" 2024-09-11T07:17:34.620 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.543119095Z level=info msg="Executing migration" id="Add OAuth expiry to user_auth" 2024-09-11T07:17:34.620 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.545940831Z level=info msg="Executing migration" id="Add index to user_id column in user_auth" 2024-09-11T07:17:34.620 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.548539911Z level=info msg="Executing migration" id="Add OAuth ID token to user_auth" 2024-09-11T07:17:34.620 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.551428597Z level=info msg="Executing migration" id="create server_lock table" 2024-09-11T07:17:34.620 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.554394325Z level=info msg="Executing migration" id="add index server_lock.operation_uid" 2024-09-11T07:17:34.620 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.556863918Z level=info msg="Executing migration" id="create user auth token table" 2024-09-11T07:17:34.620 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.559658986Z level=info msg="Executing migration" id="add unique index user_auth_token.auth_token" 2024-09-11T07:17:34.620 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.56243694Z level=info msg="Executing migration" id="add unique index user_auth_token.prev_auth_token" 2024-09-11T07:17:34.620 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.565397466Z level=info msg="Executing migration" id="add index user_auth_token.user_id" 2024-09-11T07:17:34.620 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.568118349Z level=info msg="Executing migration" id="Add revoked_at to the user auth token" 2024-09-11T07:17:34.620 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.570740485Z level=info msg="Executing migration" id="create cache_data table" 2024-09-11T07:17:34.620 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.573522227Z level=info msg="Executing migration" id="add unique index cache_data.cache_key" 2024-09-11T07:17:34.621 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.576035433Z level=info msg="Executing migration" id="create short_url table v1" 2024-09-11T07:17:34.621 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.578819017Z level=info msg="Executing migration" id="add index short_url.org_id-uid" 2024-09-11T07:17:34.621 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.581309399Z level=info msg="Executing migration" id="alter table short_url alter column created_by type to bigint" 2024-09-11T07:17:34.621 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.583975988Z level=info msg="Executing migration" id="delete alert_definition table" 2024-09-11T07:17:34.621 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.58674477Z level=info msg="Executing migration" id="recreate alert_definition table" 2024-09-11T07:17:34.621 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.589369225Z level=info msg="Executing migration" id="add index in alert_definition on org_id and title columns" 2024-09-11T07:17:34.621 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.592134478Z level=info msg="Executing migration" id="add index in alert_definition on org_id and uid columns" 2024-09-11T07:17:34.621 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.594796643Z level=info msg="Executing migration" id="alter alert_definition table data column to mediumtext in mysql" 2024-09-11T07:17:34.621 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.597248153Z level=info msg="Executing migration" id="drop index in alert_definition on org_id and title columns" 2024-09-11T07:17:34.621 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.600126656Z level=info msg="Executing migration" id="drop index in alert_definition on org_id and uid columns" 2024-09-11T07:17:34.621 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.602583765Z level=info msg="Executing migration" id="add unique index in alert_definition on org_id and title columns" 2024-09-11T07:17:34.621 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.605264887Z level=info msg="Executing migration" id="add unique index in alert_definition on org_id and uid columns" 2024-09-11T07:17:34.621 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.607767574Z level=info msg="Executing migration" id="Add column paused in alert_definition" 2024-09-11T07:17:34.858 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.610638436Z level=info msg="Executing migration" id="drop alert_definition table" 2024-09-11T07:17:34.858 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.613222118Z level=info msg="Executing migration" id="delete alert_definition_version table" 2024-09-11T07:17:34.858 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.613936422Z level=info msg="Executing migration" id="recreate alert_definition_version table" 2024-09-11T07:17:34.858 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.614741973Z level=info msg="Executing migration" id="add index in alert_definition_version table on alert_definition_id and version columns" 2024-09-11T07:17:34.859 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.615665347Z level=info msg="Executing migration" id="add index in alert_definition_version table on alert_definition_uid and version columns" 2024-09-11T07:17:34.859 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.616383271Z level=info msg="Executing migration" id="alter alert_definition_version table data column to mediumtext in mysql" 2024-09-11T07:17:34.859 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.617369945Z level=info msg="Executing migration" id="drop alert_definition_version table" 2024-09-11T07:17:34.859 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.618073924Z level=info msg="Executing migration" id="create alert_instance table" 2024-09-11T07:17:34.859 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.618983258Z level=info msg="Executing migration" id="add index in alert_instance table on def_org_id, def_uid and current_state columns" 2024-09-11T07:17:34.859 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.619694238Z level=info msg="Executing migration" id="add index in alert_instance table on def_org_id, current_state columns" 2024-09-11T07:17:34.859 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.620647801Z level=info msg="Executing migration" id="add column current_state_end to alert_instance" 2024-09-11T07:17:34.859 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.621567993Z level=info msg="Executing migration" id="remove index def_org_id, def_uid, current_state on alert_instance" 2024-09-11T07:17:34.859 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.622353572Z level=info msg="Executing migration" id="remove index def_org_id, current_state on alert_instance" 2024-09-11T07:17:34.859 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.623334832Z level=info msg="Executing migration" id="rename def_org_id to rule_org_id in alert_instance" 2024-09-11T07:17:34.859 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.626969189Z level=info msg="Executing migration" id="rename def_uid to rule_uid in alert_instance" 2024-09-11T07:17:34.859 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.630278349Z level=info msg="Executing migration" id="add index rule_org_id, rule_uid, current_state on alert_instance" 2024-09-11T07:17:34.859 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.631054502Z level=info msg="Executing migration" id="add index rule_org_id, current_state on alert_instance" 2024-09-11T07:17:34.859 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.632058051Z level=info msg="Executing migration" id="add current_reason column related to current_state" 2024-09-11T07:17:34.859 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.632967982Z level=info msg="Executing migration" id="create alert_rule table" 2024-09-11T07:17:34.860 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.633884537Z level=info msg="Executing migration" id="add index in alert_rule on org_id and title columns" 2024-09-11T07:17:34.860 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.63483295Z level=info msg="Executing migration" id="add index in alert_rule on org_id and uid columns" 2024-09-11T07:17:34.860 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.635542498Z level=info msg="Executing migration" id="add index in alert_rule on org_id, namespace_uid, group_uid columns" 2024-09-11T07:17:34.860 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.636390156Z level=info msg="Executing migration" id="alter alert_rule table data column to mediumtext in mysql" 2024-09-11T07:17:34.860 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.637367647Z level=info msg="Executing migration" id="add column for to alert_rule" 2024-09-11T07:17:34.860 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.638325291Z level=info msg="Executing migration" id="add column annotations to alert_rule" 2024-09-11T07:17:34.860 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.639169667Z level=info msg="Executing migration" id="add column labels to alert_rule" 2024-09-11T07:17:34.860 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.640011481Z level=info msg="Executing migration" id="remove unique index from alert_rule on org_id, title columns" 2024-09-11T07:17:34.860 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.641036615Z level=info msg="Executing migration" id="add index in alert_rule on org_id, namespase_uid and title columns" 2024-09-11T07:17:34.860 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.641737222Z level=info msg="Executing migration" id="add dashboard_uid column to alert_rule" 2024-09-11T07:17:34.860 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.642590187Z level=info msg="Executing migration" id="add panel_id column to alert_rule" 2024-09-11T07:17:34.860 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.643427087Z level=info msg="Executing migration" id="add index in alert_rule on org_id, dashboard_uid and panel_id columns" 2024-09-11T07:17:34.860 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.644395709Z level=info msg="Executing migration" id="add rule_group_idx column to alert_rule" 2024-09-11T07:17:34.861 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.645417465Z level=info msg="Executing migration" id="add is_paused column to alert_rule table" 2024-09-11T07:17:34.861 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.646274661Z level=info msg="Executing migration" id="fix is_paused column for alert_rule table" 2024-09-11T07:17:34.861 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.647319631Z level=info msg="Executing migration" id="create alert_rule_version table" 2024-09-11T07:17:34.862 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.647999754Z level=info msg="Executing migration" id="add index in alert_rule_version table on rule_org_id, rule_uid and version columns" 2024-09-11T07:17:34.862 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.648856983Z level=info msg="Executing migration" id="add index in alert_rule_version table on rule_org_id, rule_namespace_uid and rule_group columns" 2024-09-11T07:17:34.862 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.651788942Z level=info msg="Executing migration" id="alter alert_rule_version table data column to mediumtext in mysql" 2024-09-11T07:17:34.862 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.654408472Z level=info msg="Executing migration" id="add column for to alert_rule_version" 2024-09-11T07:17:34.862 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.657328538Z level=info msg="Executing migration" id="add column annotations to alert_rule_version" 2024-09-11T07:17:34.862 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.660151912Z level=info msg="Executing migration" id="add column labels to alert_rule_version" 2024-09-11T07:17:34.862 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.66283099Z level=info msg="Executing migration" id="add rule_group_idx column to alert_rule_version" 2024-09-11T07:17:34.862 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.665524688Z level=info msg="Executing migration" id="add is_paused column to alert_rule_versions table" 2024-09-11T07:17:34.862 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.668276197Z level=info msg="Executing migration" id="fix is_paused column for alert_rule_version table" 2024-09-11T07:17:34.862 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.671112585Z level=info msg="Executing migration" id=create_alert_configuration_table 2024-09-11T07:17:34.862 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.673763412Z level=info msg="Executing migration" id="Add column default in alert_configuration" 2024-09-11T07:17:34.862 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.676457252Z level=info msg="Executing migration" id="alert alert_configuration alertmanager_configuration column from TEXT to MEDIUMTEXT if mysql" 2024-09-11T07:17:34.862 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.679128056Z level=info msg="Executing migration" id="add column org_id in alert_configuration" 2024-09-11T07:17:34.863 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.682173119Z level=info msg="Executing migration" id="add index in alert_configuration table on org_id column" 2024-09-11T07:17:34.863 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.68489642Z level=info msg="Executing migration" id="add configuration_hash column to alert_configuration" 2024-09-11T07:17:34.863 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.687535677Z level=info msg="Executing migration" id=create_ngalert_configuration_table 2024-09-11T07:17:34.863 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.690271485Z level=info msg="Executing migration" id="add index in ngalert_configuration on org_id column" 2024-09-11T07:17:34.863 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.692871605Z level=info msg="Executing migration" id="add column send_alerts_to in ngalert_configuration" 2024-09-11T07:17:34.863 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.695782764Z level=info msg="Executing migration" id="create provenance_type table" 2024-09-11T07:17:34.863 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.698444841Z level=info msg="Executing migration" id="add index to uniquify (record_key, record_type, org_id) columns" 2024-09-11T07:17:34.863 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.700938236Z level=info msg="Executing migration" id="create alert_image table" 2024-09-11T07:17:34.863 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.703642007Z level=info msg="Executing migration" id="add unique index on token to alert_image table" 2024-09-11T07:17:34.863 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.70655121Z level=info msg="Executing migration" id="support longer URLs in alert_image table" 2024-09-11T07:17:34.863 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.709112243Z level=info msg="Executing migration" id=create_alert_configuration_history_table 2024-09-11T07:17:34.863 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.711715634Z level=info msg="Executing migration" id="drop non-unique orgID index on alert_configuration" 2024-09-11T07:17:34.864 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.714475788Z level=info msg="Executing migration" id="drop unique orgID index on alert_configuration if exists" 2024-09-11T07:17:34.864 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.714574077Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop unique orgID index on alert_configuration if exists" 2024-09-11T07:17:34.864 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.71719432Z level=info msg="Executing migration" id="extract alertmanager configuration history to separate table" 2024-09-11T07:17:34.864 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.719716094Z level=info msg="Executing migration" id="add unique index on orgID to alert_configuration" 2024-09-11T07:17:34.864 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.722169894Z level=info msg="Executing migration" id="clear migration entry \"remove unified alerting data\"" 2024-09-11T07:17:34.864 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.722329173Z level=info msg="Executing migration" id="move dashboard alerts to unified alerting" 2024-09-11T07:17:34.864 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.722472917Z level=info msg="alerts found to migrate" alerts=0 2024-09-11T07:17:34.864 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.724802365Z level=info msg="Executing migration" id="create library_element table v1" 2024-09-11T07:17:34.864 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.727370966Z level=info msg="Executing migration" id="add index library_element org_id-folder_id-name-kind" 2024-09-11T07:17:34.864 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.729940256Z level=info msg="Executing migration" id="create library_element_connection table v1" 2024-09-11T07:17:34.864 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.732554321Z level=info msg="Executing migration" id="add index library_element_connection element_id-kind-connection_id" 2024-09-11T07:17:34.864 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.735171569Z level=info msg="Executing migration" id="add unique index library_element org_id_uid" 2024-09-11T07:17:34.864 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.735630266Z level=info msg="Executing migration" id="increase max description length to 2048" 2024-09-11T07:17:34.865 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.736524039Z level=info msg="Executing migration" id="clone move dashboard alerts to unified alerting" 2024-09-11T07:17:34.865 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.737050992Z level=info msg="Executing migration" id="create data_keys table" 2024-09-11T07:17:34.865 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.737485027Z level=info msg="Executing migration" id="create secrets table" 2024-09-11T07:17:34.865 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.738330588Z level=info msg="Executing migration" id="rename data_keys name column to id" 2024-09-11T07:17:34.865 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.742762492Z level=info msg="Executing migration" id="add name column into data_keys" 2024-09-11T07:17:34.865 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.743718223Z level=info msg="Executing migration" id="copy data_keys id column values into name" 2024-09-11T07:17:34.865 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.744569216Z level=info msg="Executing migration" id="rename data_keys name column to label" 2024-09-11T07:17:34.865 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.748797122Z level=info msg="Executing migration" id="rename data_keys id column back to name" 2024-09-11T07:17:34.865 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.753147174Z level=info msg="Executing migration" id="create kv_store table v1" 2024-09-11T07:17:34.865 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.75399992Z level=info msg="Executing migration" id="add index kv_store.org_id-namespace-key" 2024-09-11T07:17:34.865 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.754840829Z level=info msg="Executing migration" id="update dashboard_uid and panel_id from existing annotations" 2024-09-11T07:17:34.865 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.755236311Z level=info msg="Executing migration" id="create permission table" 2024-09-11T07:17:34.865 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.755694564Z level=info msg="Executing migration" id="add unique index permission.role_id" 2024-09-11T07:17:34.866 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.756580397Z level=info msg="Executing migration" id="add unique index role_id_action_scope" 2024-09-11T07:17:34.866 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.757441356Z level=info msg="Executing migration" id="create role table" 2024-09-11T07:17:34.866 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.758295829Z level=info msg="Executing migration" id="add column display_name" 2024-09-11T07:17:34.866 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.761458563Z level=info msg="Executing migration" id="add column group_name" 2024-09-11T07:17:34.866 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.764174315Z level=info msg="Executing migration" id="add index role.org_id" 2024-09-11T07:17:34.866 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.767068633Z level=info msg="Executing migration" id="add unique index role_org_id_name" 2024-09-11T07:17:34.866 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.769763701Z level=info msg="Executing migration" id="add index role_org_id_uid" 2024-09-11T07:17:34.866 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.772264348Z level=info msg="Executing migration" id="create team role table" 2024-09-11T07:17:34.866 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.774900454Z level=info msg="Executing migration" id="add index team_role.org_id" 2024-09-11T07:17:34.866 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.777791993Z level=info msg="Executing migration" id="add unique index team_role_org_id_team_id_role_id" 2024-09-11T07:17:34.866 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.780707481Z level=info msg="Executing migration" id="add index team_role.team_id" 2024-09-11T07:17:34.866 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.783545324Z level=info msg="Executing migration" id="create user role table" 2024-09-11T07:17:34.866 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.786319666Z level=info msg="Executing migration" id="add index user_role.org_id" 2024-09-11T07:17:34.866 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.789131381Z level=info msg="Executing migration" id="add unique index user_role_org_id_user_id_role_id" 2024-09-11T07:17:34.866 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.791962421Z level=info msg="Executing migration" id="add index user_role.user_id" 2024-09-11T07:17:34.867 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.794753714Z level=info msg="Executing migration" id="create builtin role table" 2024-09-11T07:17:34.867 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.797422586Z level=info msg="Executing migration" id="add index builtin_role.role_id" 2024-09-11T07:17:34.867 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.800263327Z level=info msg="Executing migration" id="add index builtin_role.name" 2024-09-11T07:17:34.867 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.803131829Z level=info msg="Executing migration" id="Add column org_id to builtin_role table" 2024-09-11T07:17:34.867 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.805687595Z level=info msg="Executing migration" id="add index builtin_role.org_id" 2024-09-11T07:17:34.867 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.808438207Z level=info msg="Executing migration" id="add unique index builtin_role_org_id_role_id_role" 2024-09-11T07:17:34.867 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.811094082Z level=info msg="Executing migration" id="Remove unique index role_org_id_uid" 2024-09-11T07:17:34.867 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.811595665Z level=info msg="Executing migration" id="add unique index role.uid" 2024-09-11T07:17:34.867 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.811996098Z level=info msg="Executing migration" id="create seed assignment table" 2024-09-11T07:17:34.867 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.81238861Z level=info msg="Executing migration" id="add unique index builtin_role_role_name" 2024-09-11T07:17:34.867 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.812812918Z level=info msg="Executing migration" id="add column hidden to role table" 2024-09-11T07:17:34.867 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.81382185Z level=info msg="Executing migration" id="create query_history table v1" 2024-09-11T07:17:34.867 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.816509648Z level=info msg="Executing migration" id="add index query_history.org_id-created_by-datasource_uid" 2024-09-11T07:17:34.867 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.816947723Z level=info msg="Executing migration" id="alter table query_history alter column created_by type to bigint" 2024-09-11T07:17:34.868 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.817212294Z level=info msg="Executing migration" id="teams permissions migration" 2024-09-11T07:17:34.868 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.817780799Z level=info msg="Executing migration" id="dashboard permissions" 2024-09-11T07:17:34.868 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.818425874Z level=info msg="Executing migration" id="dashboard permissions uid scopes" 2024-09-11T07:17:34.868 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.818784856Z level=info msg="Executing migration" id="drop managed folder create actions" 2024-09-11T07:17:34.868 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.819058638Z level=info msg="Executing migration" id="alerting notification permissions" 2024-09-11T07:17:34.868 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.819409244Z level=info msg="Executing migration" id="create query_history_star table v1" 2024-09-11T07:17:34.868 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.819788912Z level=info msg="Executing migration" id="add index query_history.user_id-query_uid" 2024-09-11T07:17:34.868 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.820232996Z level=info msg="Executing migration" id="add column org_id in query_history_star" 2024-09-11T07:17:34.868 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.821174688Z level=info msg="Executing migration" id="alter table query_history_star_mig column user_id type to bigint" 2024-09-11T07:17:34.868 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.821474668Z level=info msg="Executing migration" id="create correlation table v1" 2024-09-11T07:17:34.868 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.821874218Z level=info msg="Executing migration" id="add index correlations.uid" 2024-09-11T07:17:34.868 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.822291371Z level=info msg="Executing migration" id="add index correlations.source_uid" 2024-09-11T07:17:34.868 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.822705186Z level=info msg="Executing migration" id="add correlation config column" 2024-09-11T07:17:34.868 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.82377679Z level=info msg="Executing migration" id="create entity_events table" 2024-09-11T07:17:34.869 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.824197463Z level=info msg="Executing migration" id="create dashboard public config v1" 2024-09-11T07:17:34.869 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.826689534Z level=info msg="Executing migration" id="drop index UQE_dashboard_public_config_uid - v1" 2024-09-11T07:17:34.869 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.826774016Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop index UQE_dashboard_public_config_uid - v1" 2024-09-11T07:17:34.869 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.827047736Z level=info msg="Executing migration" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-09-11T07:17:34.869 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.827122882Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-09-11T07:17:34.869 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.827405898Z level=info msg="Executing migration" id="Drop old dashboard public config table" 2024-09-11T07:17:34.869 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.829909157Z level=info msg="Executing migration" id="recreate dashboard public config v1" 2024-09-11T07:17:34.869 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.832583571Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_uid - v1" 2024-09-11T07:17:34.869 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.835246388Z level=info msg="Executing migration" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-09-11T07:17:34.869 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.838214942Z level=info msg="Executing migration" id="drop index UQE_dashboard_public_config_uid - v2" 2024-09-11T07:17:34.869 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.840703631Z level=info msg="Executing migration" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v2" 2024-09-11T07:17:34.869 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.843301409Z level=info msg="Executing migration" id="Drop public config table" 2024-09-11T07:17:34.869 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.846208265Z level=info msg="Executing migration" id="Recreate dashboard public config v2" 2024-09-11T07:17:34.869 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.848841639Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_uid - v2" 2024-09-11T07:17:34.870 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.851413252Z level=info msg="Executing migration" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v2" 2024-09-11T07:17:34.870 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.85417713Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_access_token - v2" 2024-09-11T07:17:34.870 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.856909345Z level=info msg="Executing migration" id="Rename table dashboard_public_config to dashboard_public - v2" 2024-09-11T07:17:35.185 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:17:35 smithi196 ceph-mon[30773]: pgmap v28: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:17:35.185 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:17:35 smithi196 ceph-mon[30773]: from='client.14730 v1:172.21.15.71:0/1433920105' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T07:17:35.186 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:17:35 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:17:35.186 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:17:35 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:17:35.186 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:17:35 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:17:35.186 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:17:35 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:17:35.186 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.86089576Z level=info msg="Executing migration" id="add annotations_enabled column" 2024-09-11T07:17:35.187 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.863568759Z level=info msg="Executing migration" id="add time_selection_enabled column" 2024-09-11T07:17:35.187 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.866258272Z level=info msg="Executing migration" id="delete orphaned public dashboards" 2024-09-11T07:17:35.187 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.867103272Z level=info msg="Executing migration" id="add share column" 2024-09-11T07:17:35.187 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.868094994Z level=info msg="Executing migration" id="create default alerting folders" 2024-09-11T07:17:35.187 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.869054293Z level=info msg="Executing migration" id="create file table" 2024-09-11T07:17:35.187 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.869734321Z level=info msg="Executing migration" id="file table idx: path natural pk" 2024-09-11T07:17:35.187 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.870619184Z level=info msg="Executing migration" id="file table idx: parent_folder_path_hash fast folder retrieval" 2024-09-11T07:17:35.187 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.871295268Z level=info msg="Executing migration" id="create file_meta table" 2024-09-11T07:17:35.187 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.871703787Z level=info msg="Executing migration" id="file table idx: path key" 2024-09-11T07:17:35.187 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.872657205Z level=info msg="Executing migration" id="set path collation in file table" 2024-09-11T07:17:35.187 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.873353973Z level=info msg="Executing migration" id="managed permissions migration" 2024-09-11T07:17:35.187 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.87419128Z level=info msg="Executing migration" id="managed folder permissions alert actions migration" 2024-09-11T07:17:35.187 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.875070959Z level=info msg="Executing migration" id="RBAC action name migrator" 2024-09-11T07:17:35.188 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.875843591Z level=info msg="Executing migration" id="Add UID column to playlist" 2024-09-11T07:17:35.188 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.876866668Z level=info msg="Executing migration" id="Update uid column values in playlist" 2024-09-11T07:17:35.188 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.877739914Z level=info msg="Executing migration" id="Add index for uid in playlist" 2024-09-11T07:17:35.188 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.878539753Z level=info msg="Executing migration" id="update group index for alert rules" 2024-09-11T07:17:35.188 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.879281529Z level=info msg="Executing migration" id="managed folder permissions alert actions repeated migration" 2024-09-11T07:17:35.188 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.880211709Z level=info msg="Executing migration" id="admin only folder/dashboard permission" 2024-09-11T07:17:35.188 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.880995216Z level=info msg="Executing migration" id="add action column to seed_assignment" 2024-09-11T07:17:35.188 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.882206831Z level=info msg="Executing migration" id="add scope column to seed_assignment" 2024-09-11T07:17:35.188 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.883297227Z level=info msg="Executing migration" id="remove unique index builtin_role_role_name before nullable update" 2024-09-11T07:17:35.188 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.884151127Z level=info msg="Executing migration" id="update seed_assignment role_name column to nullable" 2024-09-11T07:17:35.188 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.89517857Z level=info msg="Executing migration" id="add unique index builtin_role_name back" 2024-09-11T07:17:35.188 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.896044602Z level=info msg="Executing migration" id="add unique index builtin_role_action_scope" 2024-09-11T07:17:35.188 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.896903053Z level=info msg="Executing migration" id="add primary key to seed_assigment" 2024-09-11T07:17:35.188 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.900268763Z level=info msg="Executing migration" id="managed folder permissions alert actions repeated fixed migration" 2024-09-11T07:17:35.189 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.901164256Z level=info msg="Executing migration" id="migrate external alertmanagers to datsourcse" 2024-09-11T07:17:35.189 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.902108415Z level=info msg="Executing migration" id="create folder table" 2024-09-11T07:17:35.189 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.902830602Z level=info msg="Executing migration" id="Add index for parent_uid" 2024-09-11T07:17:35.189 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.903668695Z level=info msg="Executing migration" id="Add unique index for folder.uid and folder.org_id" 2024-09-11T07:17:35.189 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.904568294Z level=info msg="Executing migration" id="Update folder title length" 2024-09-11T07:17:35.189 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.905365329Z level=info msg="Executing migration" id="Add unique index for folder.title and folder.parent_uid" 2024-09-11T07:17:35.189 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=migrator t=2024-09-11T07:17:34.906212052Z level=info msg="migrations completed" performed=485 skipped=0 duration=718.642341ms 2024-09-11T07:17:35.189 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=sqlstore t=2024-09-11T07:17:34.906589424Z level=info msg="Created default organization" 2024-09-11T07:17:35.189 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=plugin.loader t=2024-09-11T07:17:34.964507002Z level=info msg="Plugin registered" pluginID=input 2024-09-11T07:17:35.189 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=plugin.loader t=2024-09-11T07:17:34.993698036Z level=info msg="Plugin registered" pluginID=grafana-piechart-panel 2024-09-11T07:17:35.189 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=plugin.loader t=2024-09-11T07:17:34.993717626Z level=info msg="Plugin registered" pluginID=vonage-status-panel 2024-09-11T07:17:35.189 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=secrets t=2024-09-11T07:17:34.993751919Z level=info msg="Envelope encryption state" enabled=true currentprovider=secretKey.v1 2024-09-11T07:17:35.189 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=query_data t=2024-09-11T07:17:34.996959172Z level=info msg="Query Service initialization" 2024-09-11T07:17:35.190 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=live.push_http t=2024-09-11T07:17:34.999686225Z level=info msg="Live Push Gateway initialization" 2024-09-11T07:17:35.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:17:35 smithi071 ceph-mon[27013]: pgmap v28: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:17:35.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:17:35 smithi071 ceph-mon[27013]: from='client.14730 v1:172.21.15.71:0/1433920105' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T07:17:35.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:17:35 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:17:35.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:17:35 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:17:35.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:17:35 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:17:35.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:17:35 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:17:35.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:17:35 smithi071 ceph-mon[37727]: pgmap v28: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:17:35.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:17:35 smithi071 ceph-mon[37727]: from='client.14730 v1:172.21.15.71:0/1433920105' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-11T07:17:35.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:17:35 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:17:35.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:17:35 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:17:35.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:17:35 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:17:35.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:17:35 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:17:36.074 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:35 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=infra.usagestats.collector t=2024-09-11T07:17:35.793982952Z level=info msg="registering usage stat providers" usageStatsProvidersLen=2 2024-09-11T07:17:36.074 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:35 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=provisioning.datasources t=2024-09-11T07:17:35.794309191Z level=warn msg="[Deprecated] the datasource provisioning config is outdated. please upgrade" filename=/etc/grafana/provisioning/datasources/ceph-dashboard.yml 2024-09-11T07:17:36.074 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:35 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=provisioning.datasources t=2024-09-11T07:17:35.827749179Z level=info msg="inserting datasource from configuration " name=Dashboard1 uid=P43CA22E17D0F9596 2024-09-11T07:17:36.074 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:35 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=provisioning.datasources t=2024-09-11T07:17:35.869149011Z level=info msg="inserting datasource from configuration " name=Loki uid=P8E80F9AEF21F6940 2024-09-11T07:17:36.075 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:35 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=provisioning.alerting t=2024-09-11T07:17:35.900461496Z level=info msg="starting to provision alerting" 2024-09-11T07:17:36.075 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:35 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=provisioning.alerting t=2024-09-11T07:17:35.900513222Z level=info msg="finished to provision alerting" 2024-09-11T07:17:36.075 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:35 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=grafanaStorageLogger t=2024-09-11T07:17:35.900699853Z level=info msg="storage starting" 2024-09-11T07:17:36.075 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:35 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=ngalert.state.manager t=2024-09-11T07:17:35.900833004Z level=info msg="Warming state cache for startup" 2024-09-11T07:17:36.075 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:35 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=http.server t=2024-09-11T07:17:35.902677685Z level=info msg="HTTP Server Listen" address=[::]:3000 protocol=https subUrl= socket= 2024-09-11T07:17:36.075 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:35 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=sqlstore.transactions t=2024-09-11T07:17:35.911603567Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-09-11T07:17:36.075 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:35 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=ngalert.state.manager t=2024-09-11T07:17:35.911657938Z level=info msg="State cache has been initialized" states=0 duration=10.823941ms 2024-09-11T07:17:36.075 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:35 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=ticker t=2024-09-11T07:17:35.911705544Z level=info msg=starting first_tick=2024-09-11T07:17:40Z 2024-09-11T07:17:36.075 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:35 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=ngalert.multiorg.alertmanager t=2024-09-11T07:17:35.911724896Z level=info msg="starting MultiOrg Alertmanager" 2024-09-11T07:17:36.075 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:17:35 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a[61634]: logger=sqlstore.transactions t=2024-09-11T07:17:35.945904555Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-09-11T07:17:36.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:17:36 smithi071 ceph-mon[27013]: Deploying daemon node-exporter.a on smithi071 2024-09-11T07:17:36.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:17:36 smithi071 ceph-mon[27013]: from='client.34442 v1:172.21.15.196:0/1952921461' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:17:36.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:17:36 smithi071 ceph-mon[37727]: Deploying daemon node-exporter.a on smithi071 2024-09-11T07:17:36.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:17:36 smithi071 ceph-mon[37727]: from='client.34442 v1:172.21.15.196:0/1952921461' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:17:36.449 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-11T07:17:36.449 INFO:teuthology.orchestra.run.smithi071.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-11T07:17:36.513 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:17:36 smithi196 ceph-mon[30773]: Deploying daemon node-exporter.a on smithi071 2024-09-11T07:17:36.513 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:17:36 smithi196 ceph-mon[30773]: from='client.34442 v1:172.21.15.196:0/1952921461' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:17:36.785 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 07:17:36 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-x[32609]: ::ffff:172.21.15.196 - - [11/Sep/2024:07:17:36] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:17:37.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:17:37 smithi071 ceph-mon[27013]: pgmap v29: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:17:37.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:17:37 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/2949123682' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-11T07:17:37.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:17:37 smithi071 ceph-mon[37727]: pgmap v29: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:17:37.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:17:37 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/2949123682' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-11T07:17:37.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:17:37 smithi196 ceph-mon[30773]: pgmap v29: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:17:37.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:17:37 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/2949123682' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-11T07:17:37.795 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-11T07:17:37.796 INFO:tasks.cephadm:Setup complete, yielding 2024-09-11T07:17:37.796 INFO:teuthology.run_tasks:Running task workunit... 2024-09-11T07:17:37.815 INFO:tasks.workunit:Pulling workunits from ref d0e6828a2016d48cf25ad84064e50742bb1c39b9 2024-09-11T07:17:37.816 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-09-11T07:17:37.816 DEBUG:teuthology.orchestra.run.smithi071:> stat -- /home/ubuntu/cephtest/mnt.0 2024-09-11T07:17:37.833 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:17:37.834 INFO:teuthology.orchestra.run.smithi071.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-11T07:17:37.834 DEBUG:teuthology.orchestra.run.smithi071:> mkdir -- /home/ubuntu/cephtest/mnt.0 2024-09-11T07:17:37.890 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2024-09-11T07:17:37.890 DEBUG:teuthology.orchestra.run.smithi071:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2024-09-11T07:17:37.946 INFO:tasks.workunit:timeout=3h 2024-09-11T07:17:37.946 INFO:tasks.workunit:cleanup=True 2024-09-11T07:17:37.946 DEBUG:teuthology.orchestra.run.smithi071:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout d0e6828a2016d48cf25ad84064e50742bb1c39b9 2024-09-11T07:17:38.002 INFO:tasks.workunit.client.0.smithi071.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-09-11T07:17:38.339 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 07:17:38 smithi071 bash[70961]: Writing manifest to image destination 2024-09-11T07:17:38.731 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 07:17:38 smithi071 podman[70961]: 2024-09-11 07:17:38.454196679 +0000 UTC m=+2.592367640 image pull 0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 quay.io/prometheus/node-exporter:v1.5.0 2024-09-11T07:17:38.732 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 07:17:38 smithi071 podman[70961]: 2024-09-11 07:17:38.545454316 +0000 UTC m=+2.683625282 container create 78244d3c3309f3567517357428fdc1887942981001be75d2032ddad6d56ab7ea (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-11T07:17:39.078 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 07:17:38 smithi071 podman[70961]: 2024-09-11 07:17:38.730651303 +0000 UTC m=+2.868822265 container init 78244d3c3309f3567517357428fdc1887942981001be75d2032ddad6d56ab7ea (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-11T07:17:39.078 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 07:17:38 smithi071 podman[70961]: 2024-09-11 07:17:38.733453529 +0000 UTC m=+2.871624491 container start 78244d3c3309f3567517357428fdc1887942981001be75d2032ddad6d56ab7ea (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-11T07:17:39.078 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 07:17:38 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-node-exporter-a[71149]: ts=2024-09-11T07:17:38.737Z caller=node_exporter.go:180 level=info msg="Starting node_exporter" version="(version=1.5.0, branch=HEAD, revision=1b48970ffcf5630534fb00bb0687d73c66d1c959)" 2024-09-11T07:17:39.078 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 07:17:38 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-node-exporter-a[71149]: ts=2024-09-11T07:17:38.737Z caller=node_exporter.go:181 level=info msg="Build context" build_context="(go=go1.19.3, user=root@6e7732a7b81b, date=20221129-18:59:09)" 2024-09-11T07:17:39.079 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 07:17:38 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-node-exporter-a[71149]: ts=2024-09-11T07:17:38.738Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2024-09-11T07:17:39.079 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 07:17:38 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-node-exporter-a[71149]: ts=2024-09-11T07:17:38.738Z caller=diskstats_linux.go:264 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2024-09-11T07:17:39.079 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 07:17:38 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-node-exporter-a[71149]: ts=2024-09-11T07:17:38.738Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2024-09-11T07:17:39.079 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 07:17:38 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-node-exporter-a[71149]: ts=2024-09-11T07:17:38.738Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-09-11T07:17:39.079 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 07:17:38 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-node-exporter-a[71149]: ts=2024-09-11T07:17:38.738Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2024-09-11T07:17:39.079 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 07:17:38 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-node-exporter-a[71149]: ts=2024-09-11T07:17:38.738Z caller=node_exporter.go:117 level=info collector=arp 2024-09-11T07:17:39.079 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 07:17:38 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-node-exporter-a[71149]: ts=2024-09-11T07:17:38.738Z caller=node_exporter.go:117 level=info collector=bcache 2024-09-11T07:17:39.079 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 07:17:38 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-node-exporter-a[71149]: ts=2024-09-11T07:17:38.738Z caller=node_exporter.go:117 level=info collector=bonding 2024-09-11T07:17:39.079 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 07:17:38 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-node-exporter-a[71149]: ts=2024-09-11T07:17:38.738Z caller=node_exporter.go:117 level=info collector=btrfs 2024-09-11T07:17:39.079 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 07:17:38 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-node-exporter-a[71149]: ts=2024-09-11T07:17:38.738Z caller=node_exporter.go:117 level=info collector=conntrack 2024-09-11T07:17:39.079 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 07:17:38 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-node-exporter-a[71149]: ts=2024-09-11T07:17:38.738Z caller=node_exporter.go:117 level=info collector=cpu 2024-09-11T07:17:39.079 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 07:17:38 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-node-exporter-a[71149]: ts=2024-09-11T07:17:38.738Z caller=node_exporter.go:117 level=info collector=cpufreq 2024-09-11T07:17:39.080 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 07:17:38 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-node-exporter-a[71149]: ts=2024-09-11T07:17:38.738Z caller=node_exporter.go:117 level=info collector=diskstats 2024-09-11T07:17:39.080 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 07:17:38 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-node-exporter-a[71149]: ts=2024-09-11T07:17:38.738Z caller=node_exporter.go:117 level=info collector=dmi 2024-09-11T07:17:39.080 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 07:17:38 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-node-exporter-a[71149]: ts=2024-09-11T07:17:38.738Z caller=node_exporter.go:117 level=info collector=edac 2024-09-11T07:17:39.080 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 07:17:38 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-node-exporter-a[71149]: ts=2024-09-11T07:17:38.738Z caller=node_exporter.go:117 level=info collector=entropy 2024-09-11T07:17:39.080 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 07:17:38 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-node-exporter-a[71149]: ts=2024-09-11T07:17:38.738Z caller=node_exporter.go:117 level=info collector=fibrechannel 2024-09-11T07:17:39.080 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 07:17:38 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-node-exporter-a[71149]: ts=2024-09-11T07:17:38.738Z caller=node_exporter.go:117 level=info collector=filefd 2024-09-11T07:17:39.080 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 07:17:38 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-node-exporter-a[71149]: ts=2024-09-11T07:17:38.738Z caller=node_exporter.go:117 level=info collector=filesystem 2024-09-11T07:17:39.080 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 07:17:38 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-node-exporter-a[71149]: ts=2024-09-11T07:17:38.738Z caller=node_exporter.go:117 level=info collector=hwmon 2024-09-11T07:17:39.081 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 07:17:38 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-node-exporter-a[71149]: ts=2024-09-11T07:17:38.738Z caller=node_exporter.go:117 level=info collector=infiniband 2024-09-11T07:17:39.081 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 07:17:38 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-node-exporter-a[71149]: ts=2024-09-11T07:17:38.738Z caller=node_exporter.go:117 level=info collector=ipvs 2024-09-11T07:17:39.081 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 07:17:38 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-node-exporter-a[71149]: ts=2024-09-11T07:17:38.738Z caller=node_exporter.go:117 level=info collector=loadavg 2024-09-11T07:17:39.081 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 07:17:38 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-node-exporter-a[71149]: ts=2024-09-11T07:17:38.738Z caller=node_exporter.go:117 level=info collector=mdadm 2024-09-11T07:17:39.081 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 07:17:38 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-node-exporter-a[71149]: ts=2024-09-11T07:17:38.738Z caller=node_exporter.go:117 level=info collector=meminfo 2024-09-11T07:17:39.081 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 07:17:38 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-node-exporter-a[71149]: ts=2024-09-11T07:17:38.738Z caller=node_exporter.go:117 level=info collector=netclass 2024-09-11T07:17:39.081 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 07:17:38 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-node-exporter-a[71149]: ts=2024-09-11T07:17:38.738Z caller=node_exporter.go:117 level=info collector=netdev 2024-09-11T07:17:39.081 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 07:17:38 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-node-exporter-a[71149]: ts=2024-09-11T07:17:38.738Z caller=node_exporter.go:117 level=info collector=netstat 2024-09-11T07:17:39.081 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 07:17:38 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-node-exporter-a[71149]: ts=2024-09-11T07:17:38.738Z caller=node_exporter.go:117 level=info collector=nfs 2024-09-11T07:17:39.081 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 07:17:38 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-node-exporter-a[71149]: ts=2024-09-11T07:17:38.738Z caller=node_exporter.go:117 level=info collector=nfsd 2024-09-11T07:17:39.081 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 07:17:38 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-node-exporter-a[71149]: ts=2024-09-11T07:17:38.738Z caller=node_exporter.go:117 level=info collector=nvme 2024-09-11T07:17:39.081 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 07:17:38 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-node-exporter-a[71149]: ts=2024-09-11T07:17:38.738Z caller=node_exporter.go:117 level=info collector=os 2024-09-11T07:17:39.081 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 07:17:38 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-node-exporter-a[71149]: ts=2024-09-11T07:17:38.738Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2024-09-11T07:17:39.082 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 07:17:38 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-node-exporter-a[71149]: ts=2024-09-11T07:17:38.738Z caller=node_exporter.go:117 level=info collector=pressure 2024-09-11T07:17:39.082 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 07:17:38 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-node-exporter-a[71149]: ts=2024-09-11T07:17:38.738Z caller=node_exporter.go:117 level=info collector=rapl 2024-09-11T07:17:39.082 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 07:17:38 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-node-exporter-a[71149]: ts=2024-09-11T07:17:38.738Z caller=node_exporter.go:117 level=info collector=schedstat 2024-09-11T07:17:39.082 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 07:17:38 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-node-exporter-a[71149]: ts=2024-09-11T07:17:38.738Z caller=node_exporter.go:117 level=info collector=selinux 2024-09-11T07:17:39.082 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 07:17:38 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-node-exporter-a[71149]: ts=2024-09-11T07:17:38.738Z caller=node_exporter.go:117 level=info collector=sockstat 2024-09-11T07:17:39.082 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 07:17:38 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-node-exporter-a[71149]: ts=2024-09-11T07:17:38.738Z caller=node_exporter.go:117 level=info collector=softnet 2024-09-11T07:17:39.082 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 07:17:38 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-node-exporter-a[71149]: ts=2024-09-11T07:17:38.738Z caller=node_exporter.go:117 level=info collector=stat 2024-09-11T07:17:39.082 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 07:17:38 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-node-exporter-a[71149]: ts=2024-09-11T07:17:38.738Z caller=node_exporter.go:117 level=info collector=tapestats 2024-09-11T07:17:39.082 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 07:17:38 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-node-exporter-a[71149]: ts=2024-09-11T07:17:38.738Z caller=node_exporter.go:117 level=info collector=textfile 2024-09-11T07:17:39.082 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 07:17:38 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-node-exporter-a[71149]: ts=2024-09-11T07:17:38.738Z caller=node_exporter.go:117 level=info collector=thermal_zone 2024-09-11T07:17:39.082 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 07:17:38 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-node-exporter-a[71149]: ts=2024-09-11T07:17:38.738Z caller=node_exporter.go:117 level=info collector=time 2024-09-11T07:17:39.082 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 07:17:38 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-node-exporter-a[71149]: ts=2024-09-11T07:17:38.738Z caller=node_exporter.go:117 level=info collector=udp_queues 2024-09-11T07:17:39.083 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 07:17:38 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-node-exporter-a[71149]: ts=2024-09-11T07:17:38.738Z caller=node_exporter.go:117 level=info collector=uname 2024-09-11T07:17:39.083 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 07:17:38 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-node-exporter-a[71149]: ts=2024-09-11T07:17:38.738Z caller=node_exporter.go:117 level=info collector=vmstat 2024-09-11T07:17:39.083 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 07:17:38 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-node-exporter-a[71149]: ts=2024-09-11T07:17:38.739Z caller=node_exporter.go:117 level=info collector=xfs 2024-09-11T07:17:39.083 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 07:17:38 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-node-exporter-a[71149]: ts=2024-09-11T07:17:38.739Z caller=node_exporter.go:117 level=info collector=zfs 2024-09-11T07:17:39.083 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 07:17:38 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-node-exporter-a[71149]: ts=2024-09-11T07:17:38.739Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9100 2024-09-11T07:17:39.083 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 07:17:38 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-node-exporter-a[71149]: ts=2024-09-11T07:17:38.739Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9100 2024-09-11T07:17:39.083 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 07:17:38 smithi071 bash[70961]: 78244d3c3309f3567517357428fdc1887942981001be75d2032ddad6d56ab7ea 2024-09-11T07:17:39.083 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 07:17:38 smithi071 systemd[1]: Started Ceph node-exporter.a for a81cffe6-700c-11ef-bcea-c7b262605968. 2024-09-11T07:17:39.339 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:17:39 smithi071 ceph-mon[37727]: pgmap v30: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:17:39.339 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:17:39 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:17:39.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:17:39 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:17:39.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:17:39 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:17:39.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:17:39 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:17:39.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:17:39 smithi071 ceph-mon[27013]: pgmap v30: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:17:39.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:17:39 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:17:39.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:17:39 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:17:39.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:17:39 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:17:39.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:17:39 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:17:39.406 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:17:39 smithi196 ceph-mon[30773]: pgmap v30: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:17:39.407 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:17:39 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:17:39.407 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:17:39 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:17:39.407 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:17:39 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:17:39.407 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:17:39 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:17:40.086 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 07:17:39 smithi196 bash[61898]: Trying to pull quay.io/prometheus/node-exporter:v1.5.0... 2024-09-11T07:17:40.339 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:17:40 smithi071 ceph-mon[37727]: Deploying daemon node-exporter.b on smithi196 2024-09-11T07:17:40.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:17:40 smithi071 ceph-mon[27013]: Deploying daemon node-exporter.b on smithi196 2024-09-11T07:17:40.534 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:17:40 smithi196 ceph-mon[30773]: Deploying daemon node-exporter.b on smithi196 2024-09-11T07:17:40.535 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 07:17:40 smithi196 bash[61898]: Getting image source signatures 2024-09-11T07:17:40.535 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 07:17:40 smithi196 bash[61898]: Copying blob sha256:c0e87333d3805a7b7e1c25783dd25f27da0fe70a8e262eee7ed190c8dc843fa7 2024-09-11T07:17:40.535 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 07:17:40 smithi196 bash[61898]: Copying blob sha256:22b70bddd3acadc892fca4c2af4260629bfda5dfd11ebc106a93ce24e752b5ed 2024-09-11T07:17:40.535 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 07:17:40 smithi196 bash[61898]: Copying blob sha256:5c12815fee558b157dee7f7509dedbaba0a8379098858a65ec869e1f1526ea0c 2024-09-11T07:17:41.339 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:17:41 smithi071 ceph-mon[37727]: pgmap v31: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:17:41.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:17:41 smithi071 ceph-mon[27013]: pgmap v31: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:17:41.534 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:17:41 smithi196 ceph-mon[30773]: pgmap v31: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:17:42.020 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 07:17:41 smithi196 bash[61898]: Copying config sha256:0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 2024-09-11T07:17:42.020 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 07:17:41 smithi196 bash[61898]: Writing manifest to image destination 2024-09-11T07:17:42.285 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 07:17:42 smithi196 podman[61898]: 2024-09-11 07:17:41.986828919 +0000 UTC m=+2.293459713 image pull 0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 quay.io/prometheus/node-exporter:v1.5.0 2024-09-11T07:17:42.285 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 07:17:42 smithi196 podman[61898]: 2024-09-11 07:17:42.093241744 +0000 UTC m=+2.399872538 container create 2d67aba24099ae86cabd97bffecf07402c64547541a91f89c29d8cff6c9181b3 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-11T07:17:42.785 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 07:17:42 smithi196 podman[61898]: 2024-09-11 07:17:42.320309511 +0000 UTC m=+2.626940306 container init 2d67aba24099ae86cabd97bffecf07402c64547541a91f89c29d8cff6c9181b3 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-11T07:17:42.785 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 07:17:42 smithi196 podman[61898]: 2024-09-11 07:17:42.323190689 +0000 UTC m=+2.629821488 container start 2d67aba24099ae86cabd97bffecf07402c64547541a91f89c29d8cff6c9181b3 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-11T07:17:42.785 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 07:17:42 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-node-exporter-b[61973]: ts=2024-09-11T07:17:42.327Z caller=node_exporter.go:180 level=info msg="Starting node_exporter" version="(version=1.5.0, branch=HEAD, revision=1b48970ffcf5630534fb00bb0687d73c66d1c959)" 2024-09-11T07:17:42.785 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 07:17:42 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-node-exporter-b[61973]: ts=2024-09-11T07:17:42.327Z caller=node_exporter.go:181 level=info msg="Build context" build_context="(go=go1.19.3, user=root@6e7732a7b81b, date=20221129-18:59:09)" 2024-09-11T07:17:42.786 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 07:17:42 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-node-exporter-b[61973]: ts=2024-09-11T07:17:42.327Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2024-09-11T07:17:42.786 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 07:17:42 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-node-exporter-b[61973]: ts=2024-09-11T07:17:42.327Z caller=diskstats_linux.go:264 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2024-09-11T07:17:42.786 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 07:17:42 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-node-exporter-b[61973]: ts=2024-09-11T07:17:42.327Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2024-09-11T07:17:42.786 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 07:17:42 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-node-exporter-b[61973]: ts=2024-09-11T07:17:42.327Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-09-11T07:17:42.786 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 07:17:42 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-node-exporter-b[61973]: ts=2024-09-11T07:17:42.328Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2024-09-11T07:17:42.786 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 07:17:42 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-node-exporter-b[61973]: ts=2024-09-11T07:17:42.328Z caller=node_exporter.go:117 level=info collector=arp 2024-09-11T07:17:42.786 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 07:17:42 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-node-exporter-b[61973]: ts=2024-09-11T07:17:42.328Z caller=node_exporter.go:117 level=info collector=bcache 2024-09-11T07:17:42.786 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 07:17:42 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-node-exporter-b[61973]: ts=2024-09-11T07:17:42.328Z caller=node_exporter.go:117 level=info collector=bonding 2024-09-11T07:17:42.786 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 07:17:42 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-node-exporter-b[61973]: ts=2024-09-11T07:17:42.328Z caller=node_exporter.go:117 level=info collector=btrfs 2024-09-11T07:17:42.786 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 07:17:42 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-node-exporter-b[61973]: ts=2024-09-11T07:17:42.328Z caller=node_exporter.go:117 level=info collector=conntrack 2024-09-11T07:17:42.786 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 07:17:42 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-node-exporter-b[61973]: ts=2024-09-11T07:17:42.328Z caller=node_exporter.go:117 level=info collector=cpu 2024-09-11T07:17:42.787 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 07:17:42 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-node-exporter-b[61973]: ts=2024-09-11T07:17:42.328Z caller=node_exporter.go:117 level=info collector=cpufreq 2024-09-11T07:17:42.787 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 07:17:42 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-node-exporter-b[61973]: ts=2024-09-11T07:17:42.328Z caller=node_exporter.go:117 level=info collector=diskstats 2024-09-11T07:17:42.787 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 07:17:42 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-node-exporter-b[61973]: ts=2024-09-11T07:17:42.328Z caller=node_exporter.go:117 level=info collector=dmi 2024-09-11T07:17:42.787 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 07:17:42 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-node-exporter-b[61973]: ts=2024-09-11T07:17:42.328Z caller=node_exporter.go:117 level=info collector=edac 2024-09-11T07:17:42.787 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 07:17:42 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-node-exporter-b[61973]: ts=2024-09-11T07:17:42.328Z caller=node_exporter.go:117 level=info collector=entropy 2024-09-11T07:17:42.787 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 07:17:42 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-node-exporter-b[61973]: ts=2024-09-11T07:17:42.328Z caller=node_exporter.go:117 level=info collector=fibrechannel 2024-09-11T07:17:42.787 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 07:17:42 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-node-exporter-b[61973]: ts=2024-09-11T07:17:42.328Z caller=node_exporter.go:117 level=info collector=filefd 2024-09-11T07:17:42.787 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 07:17:42 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-node-exporter-b[61973]: ts=2024-09-11T07:17:42.328Z caller=node_exporter.go:117 level=info collector=filesystem 2024-09-11T07:17:42.788 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 07:17:42 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-node-exporter-b[61973]: ts=2024-09-11T07:17:42.328Z caller=node_exporter.go:117 level=info collector=hwmon 2024-09-11T07:17:42.788 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 07:17:42 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-node-exporter-b[61973]: ts=2024-09-11T07:17:42.328Z caller=node_exporter.go:117 level=info collector=infiniband 2024-09-11T07:17:42.788 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 07:17:42 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-node-exporter-b[61973]: ts=2024-09-11T07:17:42.328Z caller=node_exporter.go:117 level=info collector=ipvs 2024-09-11T07:17:42.788 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 07:17:42 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-node-exporter-b[61973]: ts=2024-09-11T07:17:42.328Z caller=node_exporter.go:117 level=info collector=loadavg 2024-09-11T07:17:42.788 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 07:17:42 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-node-exporter-b[61973]: ts=2024-09-11T07:17:42.328Z caller=node_exporter.go:117 level=info collector=mdadm 2024-09-11T07:17:42.788 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 07:17:42 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-node-exporter-b[61973]: ts=2024-09-11T07:17:42.328Z caller=node_exporter.go:117 level=info collector=meminfo 2024-09-11T07:17:42.788 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 07:17:42 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-node-exporter-b[61973]: ts=2024-09-11T07:17:42.328Z caller=node_exporter.go:117 level=info collector=netclass 2024-09-11T07:17:42.788 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 07:17:42 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-node-exporter-b[61973]: ts=2024-09-11T07:17:42.328Z caller=node_exporter.go:117 level=info collector=netdev 2024-09-11T07:17:42.788 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 07:17:42 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-node-exporter-b[61973]: ts=2024-09-11T07:17:42.328Z caller=node_exporter.go:117 level=info collector=netstat 2024-09-11T07:17:42.788 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 07:17:42 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-node-exporter-b[61973]: ts=2024-09-11T07:17:42.328Z caller=node_exporter.go:117 level=info collector=nfs 2024-09-11T07:17:42.788 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 07:17:42 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-node-exporter-b[61973]: ts=2024-09-11T07:17:42.328Z caller=node_exporter.go:117 level=info collector=nfsd 2024-09-11T07:17:42.789 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 07:17:42 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-node-exporter-b[61973]: ts=2024-09-11T07:17:42.328Z caller=node_exporter.go:117 level=info collector=nvme 2024-09-11T07:17:42.789 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 07:17:42 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-node-exporter-b[61973]: ts=2024-09-11T07:17:42.328Z caller=node_exporter.go:117 level=info collector=os 2024-09-11T07:17:42.789 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 07:17:42 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-node-exporter-b[61973]: ts=2024-09-11T07:17:42.328Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2024-09-11T07:17:42.789 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 07:17:42 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-node-exporter-b[61973]: ts=2024-09-11T07:17:42.328Z caller=node_exporter.go:117 level=info collector=pressure 2024-09-11T07:17:42.789 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 07:17:42 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-node-exporter-b[61973]: ts=2024-09-11T07:17:42.328Z caller=node_exporter.go:117 level=info collector=rapl 2024-09-11T07:17:42.789 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 07:17:42 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-node-exporter-b[61973]: ts=2024-09-11T07:17:42.328Z caller=node_exporter.go:117 level=info collector=schedstat 2024-09-11T07:17:42.789 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 07:17:42 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-node-exporter-b[61973]: ts=2024-09-11T07:17:42.328Z caller=node_exporter.go:117 level=info collector=selinux 2024-09-11T07:17:42.789 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 07:17:42 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-node-exporter-b[61973]: ts=2024-09-11T07:17:42.328Z caller=node_exporter.go:117 level=info collector=sockstat 2024-09-11T07:17:42.789 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 07:17:42 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-node-exporter-b[61973]: ts=2024-09-11T07:17:42.328Z caller=node_exporter.go:117 level=info collector=softnet 2024-09-11T07:17:42.789 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 07:17:42 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-node-exporter-b[61973]: ts=2024-09-11T07:17:42.328Z caller=node_exporter.go:117 level=info collector=stat 2024-09-11T07:17:42.789 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 07:17:42 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-node-exporter-b[61973]: ts=2024-09-11T07:17:42.328Z caller=node_exporter.go:117 level=info collector=tapestats 2024-09-11T07:17:42.789 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 07:17:42 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-node-exporter-b[61973]: ts=2024-09-11T07:17:42.328Z caller=node_exporter.go:117 level=info collector=textfile 2024-09-11T07:17:42.789 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 07:17:42 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-node-exporter-b[61973]: ts=2024-09-11T07:17:42.328Z caller=node_exporter.go:117 level=info collector=thermal_zone 2024-09-11T07:17:42.790 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 07:17:42 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-node-exporter-b[61973]: ts=2024-09-11T07:17:42.328Z caller=node_exporter.go:117 level=info collector=time 2024-09-11T07:17:42.790 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 07:17:42 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-node-exporter-b[61973]: ts=2024-09-11T07:17:42.328Z caller=node_exporter.go:117 level=info collector=udp_queues 2024-09-11T07:17:42.790 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 07:17:42 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-node-exporter-b[61973]: ts=2024-09-11T07:17:42.328Z caller=node_exporter.go:117 level=info collector=uname 2024-09-11T07:17:42.790 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 07:17:42 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-node-exporter-b[61973]: ts=2024-09-11T07:17:42.328Z caller=node_exporter.go:117 level=info collector=vmstat 2024-09-11T07:17:42.790 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 07:17:42 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-node-exporter-b[61973]: ts=2024-09-11T07:17:42.328Z caller=node_exporter.go:117 level=info collector=xfs 2024-09-11T07:17:42.790 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 07:17:42 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-node-exporter-b[61973]: ts=2024-09-11T07:17:42.328Z caller=node_exporter.go:117 level=info collector=zfs 2024-09-11T07:17:42.790 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 07:17:42 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-node-exporter-b[61973]: ts=2024-09-11T07:17:42.329Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9100 2024-09-11T07:17:42.790 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 07:17:42 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-node-exporter-b[61973]: ts=2024-09-11T07:17:42.329Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9100 2024-09-11T07:17:42.790 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 07:17:42 smithi196 bash[61898]: 2d67aba24099ae86cabd97bffecf07402c64547541a91f89c29d8cff6c9181b3 2024-09-11T07:17:42.790 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 07:17:42 smithi196 systemd[1]: Started Ceph node-exporter.b for a81cffe6-700c-11ef-bcea-c7b262605968. 2024-09-11T07:17:43.270 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:17:43 smithi071 ceph-mon[27013]: pgmap v32: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:17:43.271 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:17:43 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:17:43.271 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:17:43 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:17:43.271 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:17:43 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:17:43.271 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:17:43 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:17:43.271 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:17:43 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:17:43.271 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:17:43 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:17:43.271 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:17:43 smithi071 ceph-mon[37727]: pgmap v32: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:17:43.271 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:17:43 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:17:43.272 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:17:43 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:17:43.272 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:17:43 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:17:43.272 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:17:43 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:17:43.272 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:17:43 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:17:43.272 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:17:43 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:17:43.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:17:43 smithi196 ceph-mon[30773]: pgmap v32: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:17:43.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:17:43 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:17:43.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:17:43 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:17:43.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:17:43 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:17:43.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:17:43 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:17:43.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:17:43 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:17:43.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:17:43 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:17:44.284 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:17:44 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [7, 2]}]: dispatch 2024-09-11T07:17:44.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:17:44 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2024-09-11T07:17:44.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:17:44 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [1, 2]}]: dispatch 2024-09-11T07:17:44.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:17:44 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 2]}]: dispatch 2024-09-11T07:17:44.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:17:44 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:17:44.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:17:44 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:17:44.339 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:17:44 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [7, 2]}]: dispatch 2024-09-11T07:17:44.339 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:17:44 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2024-09-11T07:17:44.339 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:17:44 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [1, 2]}]: dispatch 2024-09-11T07:17:44.339 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:17:44 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 2]}]: dispatch 2024-09-11T07:17:44.339 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:17:44 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:17:44.339 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:17:44 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:17:44.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:17:44 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [7, 2]}]: dispatch 2024-09-11T07:17:44.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:17:44 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2024-09-11T07:17:44.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:17:44 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [1, 2]}]: dispatch 2024-09-11T07:17:44.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:17:44 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 2]}]: dispatch 2024-09-11T07:17:44.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:17:44 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:17:44.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:17:44 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:17:45.034 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 11 07:17:44 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-iscsi-iscsi-a[58197]: debug there is no tcmu-runner data available 2024-09-11T07:17:45.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:17:45 smithi196 ceph-mon[30773]: pgmap v33: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:17:45.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:17:45 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [7, 2]}]': finished 2024-09-11T07:17:45.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:17:45 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]': finished 2024-09-11T07:17:45.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:17:45 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [1, 2]}]': finished 2024-09-11T07:17:45.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:17:45 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 2]}]': finished 2024-09-11T07:17:45.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:17:45 smithi196 ceph-mon[30773]: osdmap e62: 8 total, 8 up, 8 in 2024-09-11T07:17:45.589 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:17:45 smithi071 ceph-mon[37727]: pgmap v33: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:17:45.589 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:17:45 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [7, 2]}]': finished 2024-09-11T07:17:45.589 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:17:45 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]': finished 2024-09-11T07:17:45.589 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:17:45 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [1, 2]}]': finished 2024-09-11T07:17:45.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:17:45 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 2]}]': finished 2024-09-11T07:17:45.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:17:45 smithi071 ceph-mon[37727]: osdmap e62: 8 total, 8 up, 8 in 2024-09-11T07:17:45.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:17:45 smithi071 ceph-mon[27013]: pgmap v33: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:17:45.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:17:45 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [7, 2]}]': finished 2024-09-11T07:17:45.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:17:45 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]': finished 2024-09-11T07:17:45.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:17:45 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [1, 2]}]': finished 2024-09-11T07:17:45.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:17:45 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 2]}]': finished 2024-09-11T07:17:45.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:17:45 smithi071 ceph-mon[27013]: osdmap e62: 8 total, 8 up, 8 in 2024-09-11T07:17:46.284 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:17:46 smithi196 ceph-mon[30773]: from='client.34442 v1:172.21.15.196:0/1952921461' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:17:46.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:17:46 smithi196 ceph-mon[30773]: osdmap e63: 8 total, 8 up, 8 in 2024-09-11T07:17:46.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:17:46 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:17:46.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:17:46 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:17:46.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:17:46 smithi196 ceph-mon[30773]: pgmap v36: 132 pgs: 4 peering, 128 active+clean; 581 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-09-11T07:17:46.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:17:46 smithi071 ceph-mon[27013]: from='client.34442 v1:172.21.15.196:0/1952921461' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:17:46.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:17:46 smithi071 ceph-mon[27013]: osdmap e63: 8 total, 8 up, 8 in 2024-09-11T07:17:46.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:17:46 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:17:46.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:17:46 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:17:46.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:17:46 smithi071 ceph-mon[27013]: pgmap v36: 132 pgs: 4 peering, 128 active+clean; 581 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-09-11T07:17:46.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:17:46 smithi071 ceph-mon[37727]: from='client.34442 v1:172.21.15.196:0/1952921461' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:17:46.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:17:46 smithi071 ceph-mon[37727]: osdmap e63: 8 total, 8 up, 8 in 2024-09-11T07:17:46.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:17:46 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:17:46.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:17:46 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:17:46.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:17:46 smithi071 ceph-mon[37727]: pgmap v36: 132 pgs: 4 peering, 128 active+clean; 581 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-09-11T07:17:47.034 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 07:17:46 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-x[32609]: ::ffff:172.21.15.196 - - [11/Sep/2024:07:17:46] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:17:47.518 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:17:47 smithi071 ceph-mon[27013]: Health check failed: Reduced data availability: 3 pgs peering (PG_AVAILABILITY) 2024-09-11T07:17:47.518 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:17:47 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:17:47.518 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:17:47 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:17:47.518 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:17:47 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:17:47.519 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:17:47 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:17:47.519 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:17:47 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:17:47.519 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:17:47 smithi071 ceph-mon[27013]: Reconfiguring alertmanager.a (dependencies changed)... 2024-09-11T07:17:47.519 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:17:47 smithi071 ceph-mon[27013]: Reconfiguring daemon alertmanager.a on smithi071 2024-09-11T07:17:47.519 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:17:47 smithi071 ceph-mon[37727]: Health check failed: Reduced data availability: 3 pgs peering (PG_AVAILABILITY) 2024-09-11T07:17:47.519 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:17:47 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:17:47.519 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:17:47 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:17:47.519 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:17:47 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:17:47.519 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:17:47 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:17:47.519 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:17:47 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:17:47.519 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:17:47 smithi071 ceph-mon[37727]: Reconfiguring alertmanager.a (dependencies changed)... 2024-09-11T07:17:47.519 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:17:47 smithi071 ceph-mon[37727]: Reconfiguring daemon alertmanager.a on smithi071 2024-09-11T07:17:47.534 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:17:47 smithi196 ceph-mon[30773]: Health check failed: Reduced data availability: 3 pgs peering (PG_AVAILABILITY) 2024-09-11T07:17:47.534 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:17:47 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:17:47.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:17:47 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:17:47.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:17:47 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:17:47.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:17:47 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:17:47.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:17:47 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:17:47.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:17:47 smithi196 ceph-mon[30773]: Reconfiguring alertmanager.a (dependencies changed)... 2024-09-11T07:17:47.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:17:47 smithi196 ceph-mon[30773]: Reconfiguring daemon alertmanager.a on smithi071 2024-09-11T07:17:48.502 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:17:48 smithi071 ceph-mon[27013]: pgmap v37: 132 pgs: 4 peering, 128 active+clean; 581 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:17:48.502 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:17:48 smithi071 ceph-mon[37727]: pgmap v37: 132 pgs: 4 peering, 128 active+clean; 581 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:17:48.534 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:17:48 smithi196 ceph-mon[30773]: pgmap v37: 132 pgs: 4 peering, 128 active+clean; 581 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:17:49.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:17:49 smithi071 ceph-mon[27013]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 3 pgs peering) 2024-09-11T07:17:49.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:17:49 smithi071 ceph-mon[27013]: Cluster is now healthy 2024-09-11T07:17:49.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:17:49 smithi071 ceph-mon[37727]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 3 pgs peering) 2024-09-11T07:17:49.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:17:49 smithi071 ceph-mon[37727]: Cluster is now healthy 2024-09-11T07:17:50.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:17:49 smithi196 ceph-mon[30773]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 3 pgs peering) 2024-09-11T07:17:50.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:17:49 smithi196 ceph-mon[30773]: Cluster is now healthy 2024-09-11T07:17:50.339 INFO:journalctl@ceph.alertmanager.a.smithi071.stdout:Sep 11 07:17:50 smithi071 systemd[1]: Stopping Ceph alertmanager.a for a81cffe6-700c-11ef-bcea-c7b262605968... 2024-09-11T07:17:50.641 INFO:journalctl@ceph.alertmanager.a.smithi071.stdout:Sep 11 07:17:50 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-alertmanager-a[70009]: ts=2024-09-11T07:17:50.379Z caller=main.go:583 level=info msg="Received SIGTERM, exiting gracefully..." 2024-09-11T07:17:50.642 INFO:journalctl@ceph.alertmanager.a.smithi071.stdout:Sep 11 07:17:50 smithi071 podman[71865]: 2024-09-11 07:17:50.400147862 +0000 UTC m=+0.133860796 container died 5581897770a7032d42952e97b7967338d64267239d5d2fb227485c3fba4e36f1 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-11T07:17:50.973 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:17:50 smithi071 ceph-mon[27013]: pgmap v38: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 255 B/s rd, 0 op/s; 150 B/s, 1 objects/s recovering 2024-09-11T07:17:50.974 INFO:journalctl@ceph.alertmanager.a.smithi071.stdout:Sep 11 07:17:50 smithi071 podman[71865]: 2024-09-11 07:17:50.754177657 +0000 UTC m=+0.487890587 container cleanup 5581897770a7032d42952e97b7967338d64267239d5d2fb227485c3fba4e36f1 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-11T07:17:50.974 INFO:journalctl@ceph.alertmanager.a.smithi071.stdout:Sep 11 07:17:50 smithi071 bash[71865]: ceph-a81cffe6-700c-11ef-bcea-c7b262605968-alertmanager-a 2024-09-11T07:17:50.974 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:17:50 smithi071 ceph-mon[37727]: pgmap v38: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 255 B/s rd, 0 op/s; 150 B/s, 1 objects/s recovering 2024-09-11T07:17:51.041 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:17:50 smithi196 ceph-mon[30773]: pgmap v38: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 255 B/s rd, 0 op/s; 150 B/s, 1 objects/s recovering 2024-09-11T07:17:51.309 INFO:journalctl@ceph.alertmanager.a.smithi071.stdout:Sep 11 07:17:50 smithi071 podman[71878]: 2024-09-11 07:17:50.972101678 +0000 UTC m=+0.566410062 container remove 5581897770a7032d42952e97b7967338d64267239d5d2fb227485c3fba4e36f1 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-11T07:17:51.309 INFO:journalctl@ceph.alertmanager.a.smithi071.stdout:Sep 11 07:17:51 smithi071 podman[71878]: 2024-09-11 07:17:51.039172202 +0000 UTC m=+0.633480545 volume remove 2764c820ff9c32bdd31edf1868be147740942c45e14dc0f6a5383e507370151c 2024-09-11T07:17:51.589 INFO:journalctl@ceph.alertmanager.a.smithi071.stdout:Sep 11 07:17:51 smithi071 systemd[1]: ceph-a81cffe6-700c-11ef-bcea-c7b262605968@alertmanager.a.service: Deactivated successfully. 2024-09-11T07:17:51.589 INFO:journalctl@ceph.alertmanager.a.smithi071.stdout:Sep 11 07:17:51 smithi071 systemd[1]: Stopped Ceph alertmanager.a for a81cffe6-700c-11ef-bcea-c7b262605968. 2024-09-11T07:17:51.589 INFO:journalctl@ceph.alertmanager.a.smithi071.stdout:Sep 11 07:17:51 smithi071 systemd[1]: Starting Ceph alertmanager.a for a81cffe6-700c-11ef-bcea-c7b262605968... 2024-09-11T07:17:51.589 INFO:journalctl@ceph.alertmanager.a.smithi071.stdout:Sep 11 07:17:51 smithi071 podman[71974]: 2024-09-11 07:17:51.498413219 +0000 UTC m=+0.089755016 volume create 64389bfa7951215fe7a9e9d1ae2e9e2dbd350e799bacd571dff2101240e8147f 2024-09-11T07:17:51.589 INFO:journalctl@ceph.alertmanager.a.smithi071.stdout:Sep 11 07:17:51 smithi071 podman[71974]: 2024-09-11 07:17:51.424361287 +0000 UTC m=+0.015703094 image pull c8568f914cd25b2062c44e9f79f9c18da6e3b85fe0c47a12a2191c61426c2b19 quay.io/prometheus/alertmanager:v0.25.0 2024-09-11T07:17:51.589 INFO:journalctl@ceph.alertmanager.a.smithi071.stdout:Sep 11 07:17:51 smithi071 podman[71974]: 2024-09-11 07:17:51.565252199 +0000 UTC m=+0.156593996 container create 5a42050d984d5b52db889262f99078937223a0d065fdfae878e6d5e0c52a8944 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-11T07:17:52.089 INFO:journalctl@ceph.alertmanager.a.smithi071.stdout:Sep 11 07:17:51 smithi071 podman[71974]: 2024-09-11 07:17:51.876183646 +0000 UTC m=+0.467525443 container init 5a42050d984d5b52db889262f99078937223a0d065fdfae878e6d5e0c52a8944 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-11T07:17:52.089 INFO:journalctl@ceph.alertmanager.a.smithi071.stdout:Sep 11 07:17:51 smithi071 podman[71974]: 2024-09-11 07:17:51.879005905 +0000 UTC m=+0.470347711 container start 5a42050d984d5b52db889262f99078937223a0d065fdfae878e6d5e0c52a8944 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-11T07:17:52.090 INFO:journalctl@ceph.alertmanager.a.smithi071.stdout:Sep 11 07:17:51 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-alertmanager-a[71989]: ts=2024-09-11T07:17:51.891Z caller=main.go:240 level=info msg="Starting Alertmanager" version="(version=0.25.0, branch=HEAD, revision=258fab7cdd551f2cf251ed0348f0ad7289aee789)" 2024-09-11T07:17:52.090 INFO:journalctl@ceph.alertmanager.a.smithi071.stdout:Sep 11 07:17:51 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-alertmanager-a[71989]: ts=2024-09-11T07:17:51.891Z caller=main.go:241 level=info build_context="(go=go1.19.4, user=root@abe866dd5717, date=20221222-14:51:36)" 2024-09-11T07:17:52.090 INFO:journalctl@ceph.alertmanager.a.smithi071.stdout:Sep 11 07:17:51 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-alertmanager-a[71989]: ts=2024-09-11T07:17:51.892Z caller=cluster.go:185 level=info component=cluster msg="setting advertise address explicitly" addr=172.21.15.71 port=9094 2024-09-11T07:17:52.090 INFO:journalctl@ceph.alertmanager.a.smithi071.stdout:Sep 11 07:17:51 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-alertmanager-a[71989]: ts=2024-09-11T07:17:51.892Z caller=cluster.go:681 level=info component=cluster msg="Waiting for gossip to settle..." interval=2s 2024-09-11T07:17:52.090 INFO:journalctl@ceph.alertmanager.a.smithi071.stdout:Sep 11 07:17:51 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-alertmanager-a[71989]: ts=2024-09-11T07:17:51.915Z caller=coordinator.go:113 level=info component=configuration msg="Loading configuration file" file=/etc/alertmanager/alertmanager.yml 2024-09-11T07:17:52.090 INFO:journalctl@ceph.alertmanager.a.smithi071.stdout:Sep 11 07:17:51 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-alertmanager-a[71989]: ts=2024-09-11T07:17:51.916Z caller=coordinator.go:126 level=info component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/alertmanager.yml 2024-09-11T07:17:52.090 INFO:journalctl@ceph.alertmanager.a.smithi071.stdout:Sep 11 07:17:51 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-alertmanager-a[71989]: ts=2024-09-11T07:17:51.919Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9093 2024-09-11T07:17:52.090 INFO:journalctl@ceph.alertmanager.a.smithi071.stdout:Sep 11 07:17:51 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-alertmanager-a[71989]: ts=2024-09-11T07:17:51.919Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9093 2024-09-11T07:17:52.090 INFO:journalctl@ceph.alertmanager.a.smithi071.stdout:Sep 11 07:17:51 smithi071 bash[71974]: 5a42050d984d5b52db889262f99078937223a0d065fdfae878e6d5e0c52a8944 2024-09-11T07:17:52.090 INFO:journalctl@ceph.alertmanager.a.smithi071.stdout:Sep 11 07:17:51 smithi071 systemd[1]: Started Ceph alertmanager.a for a81cffe6-700c-11ef-bcea-c7b262605968. 2024-09-11T07:17:53.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:17:53 smithi196 ceph-mon[30773]: pgmap v39: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 255 B/s rd, 0 op/s; 150 B/s, 1 objects/s recovering 2024-09-11T07:17:53.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:17:53 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:17:53.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:17:53 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:17:53.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:17:53 smithi196 ceph-mon[30773]: Reconfiguring prometheus.a (dependencies changed)... 2024-09-11T07:17:53.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:17:53 smithi196 ceph-mon[30773]: Reconfiguring daemon prometheus.a on smithi196 2024-09-11T07:17:53.339 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:17:53 smithi071 ceph-mon[37727]: pgmap v39: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 255 B/s rd, 0 op/s; 150 B/s, 1 objects/s recovering 2024-09-11T07:17:53.339 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:17:53 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:17:53.339 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:17:53 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:17:53.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:17:53 smithi071 ceph-mon[37727]: Reconfiguring prometheus.a (dependencies changed)... 2024-09-11T07:17:53.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:17:53 smithi071 ceph-mon[37727]: Reconfiguring daemon prometheus.a on smithi196 2024-09-11T07:17:53.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:17:53 smithi071 ceph-mon[27013]: pgmap v39: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 255 B/s rd, 0 op/s; 150 B/s, 1 objects/s recovering 2024-09-11T07:17:53.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:17:53 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:17:53.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:17:53 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:17:53.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:17:53 smithi071 ceph-mon[27013]: Reconfiguring prometheus.a (dependencies changed)... 2024-09-11T07:17:53.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:17:53 smithi071 ceph-mon[27013]: Reconfiguring daemon prometheus.a on smithi196 2024-09-11T07:17:54.339 INFO:journalctl@ceph.alertmanager.a.smithi071.stdout:Sep 11 07:17:53 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-alertmanager-a[71989]: ts=2024-09-11T07:17:53.893Z caller=cluster.go:706 level=info component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.000963772s 2024-09-11T07:17:55.016 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 11 07:17:54 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-iscsi-iscsi-a[58197]: debug there is no tcmu-runner data available 2024-09-11T07:17:55.284 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:17:55 smithi196 ceph-mon[30773]: pgmap v40: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 216 B/s rd, 0 op/s; 127 B/s, 1 objects/s recovering 2024-09-11T07:17:55.285 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 07:17:55 smithi196 systemd[1]: Stopping Ceph prometheus.a for a81cffe6-700c-11ef-bcea-c7b262605968... 2024-09-11T07:17:55.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:17:55 smithi071 ceph-mon[27013]: pgmap v40: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 216 B/s rd, 0 op/s; 127 B/s, 1 objects/s recovering 2024-09-11T07:17:55.339 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:17:55 smithi071 ceph-mon[37727]: pgmap v40: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 216 B/s rd, 0 op/s; 127 B/s, 1 objects/s recovering 2024-09-11T07:17:55.681 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 07:17:55 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-prometheus-a[59726]: ts=2024-09-11T07:17:55.413Z caller=main.go:840 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-09-11T07:17:55.682 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 07:17:55 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-prometheus-a[59726]: ts=2024-09-11T07:17:55.413Z caller=main.go:864 level=info msg="Stopping scrape discovery manager..." 2024-09-11T07:17:55.682 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 07:17:55 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-prometheus-a[59726]: ts=2024-09-11T07:17:55.413Z caller=main.go:878 level=info msg="Stopping notify discovery manager..." 2024-09-11T07:17:55.682 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 07:17:55 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-prometheus-a[59726]: ts=2024-09-11T07:17:55.413Z caller=manager.go:988 level=info component="rule manager" msg="Stopping rule manager..." 2024-09-11T07:17:55.682 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 07:17:55 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-prometheus-a[59726]: ts=2024-09-11T07:17:55.413Z caller=main.go:860 level=info msg="Scrape discovery manager stopped" 2024-09-11T07:17:55.682 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 07:17:55 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-prometheus-a[59726]: ts=2024-09-11T07:17:55.413Z caller=manager.go:998 level=info component="rule manager" msg="Rule manager stopped" 2024-09-11T07:17:55.682 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 07:17:55 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-prometheus-a[59726]: ts=2024-09-11T07:17:55.413Z caller=main.go:874 level=info msg="Notify discovery manager stopped" 2024-09-11T07:17:55.682 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 07:17:55 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-prometheus-a[59726]: ts=2024-09-11T07:17:55.413Z caller=main.go:915 level=info msg="Stopping scrape manager..." 2024-09-11T07:17:55.682 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 07:17:55 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-prometheus-a[59726]: ts=2024-09-11T07:17:55.413Z caller=main.go:907 level=info msg="Scrape manager stopped" 2024-09-11T07:17:55.682 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 07:17:55 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-prometheus-a[59726]: ts=2024-09-11T07:17:55.413Z caller=notifier.go:606 level=info component=notifier msg="Stopping notification manager..." 2024-09-11T07:17:55.682 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 07:17:55 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-prometheus-a[59726]: ts=2024-09-11T07:17:55.413Z caller=main.go:1135 level=info msg="Notifier manager stopped" 2024-09-11T07:17:55.682 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 07:17:55 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-prometheus-a[59726]: ts=2024-09-11T07:17:55.413Z caller=main.go:1147 level=info msg="See you next time!" 2024-09-11T07:17:55.683 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 07:17:55 smithi196 podman[62773]: 2024-09-11 07:17:55.423387482 +0000 UTC m=+0.106818652 container died effc2eee58c985d81e3191836b25652719637c072b8a662273c7cc5a3f66627e (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-11T07:17:56.021 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 07:17:55 smithi196 podman[62773]: 2024-09-11 07:17:55.779477673 +0000 UTC m=+0.462908849 container cleanup effc2eee58c985d81e3191836b25652719637c072b8a662273c7cc5a3f66627e (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-11T07:17:56.021 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 07:17:55 smithi196 bash[62773]: ceph-a81cffe6-700c-11ef-bcea-c7b262605968-prometheus-a 2024-09-11T07:17:56.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:17:56 smithi196 ceph-mon[30773]: from='client.34442 v1:172.21.15.196:0/1952921461' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:17:56.285 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 07:17:56 smithi196 podman[62788]: 2024-09-11 07:17:56.081053075 +0000 UTC m=+0.659409754 container remove effc2eee58c985d81e3191836b25652719637c072b8a662273c7cc5a3f66627e (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-11T07:17:56.285 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 07:17:56 smithi196 systemd[1]: ceph-a81cffe6-700c-11ef-bcea-c7b262605968@prometheus.a.service: Deactivated successfully. 2024-09-11T07:17:56.285 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 07:17:56 smithi196 systemd[1]: Stopped Ceph prometheus.a for a81cffe6-700c-11ef-bcea-c7b262605968. 2024-09-11T07:17:56.285 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 07:17:56 smithi196 systemd[1]: ceph-a81cffe6-700c-11ef-bcea-c7b262605968@prometheus.a.service: Consumed 1.018s CPU time. 2024-09-11T07:17:56.285 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 07:17:56 smithi196 systemd[1]: Starting Ceph prometheus.a for a81cffe6-700c-11ef-bcea-c7b262605968... 2024-09-11T07:17:56.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:17:56 smithi071 ceph-mon[27013]: from='client.34442 v1:172.21.15.196:0/1952921461' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:17:56.339 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:17:56 smithi071 ceph-mon[37727]: from='client.34442 v1:172.21.15.196:0/1952921461' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:17:56.693 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 07:17:56 smithi196 podman[62881]: 2024-09-11 07:17:56.3058968 +0000 UTC m=+0.016753380 image pull a07b618ecd1dce142bce4c52f0e80982eaf1f14265a2415c2d35978ccaa0a464 quay.io/prometheus/prometheus:v2.43.0 2024-09-11T07:17:56.694 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 07:17:56 smithi196 podman[62881]: 2024-09-11 07:17:56.42375622 +0000 UTC m=+0.134612864 container create 026075a3cf15820e30f8392a39c1e1f8568c677ae9bb86c91063927cddab2367 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-11T07:17:57.023 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 07:17:56 smithi196 podman[62881]: 2024-09-11 07:17:56.692728326 +0000 UTC m=+0.403584900 container init 026075a3cf15820e30f8392a39c1e1f8568c677ae9bb86c91063927cddab2367 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-11T07:17:57.023 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 07:17:56 smithi196 podman[62881]: 2024-09-11 07:17:56.695491568 +0000 UTC m=+0.406348153 container start 026075a3cf15820e30f8392a39c1e1f8568c677ae9bb86c91063927cddab2367 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-11T07:17:57.023 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 07:17:56 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-prometheus-a[62896]: ts=2024-09-11T07:17:56.719Z caller=main.go:564 level=info msg="Starting Prometheus Server" mode=server version="(version=2.43.0, branch=HEAD, revision=edfc3bcd025dd6fe296c167a14a216cab1e552ee)" 2024-09-11T07:17:57.023 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 07:17:56 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-prometheus-a[62896]: ts=2024-09-11T07:17:56.719Z caller=main.go:569 level=info build_context="(go=go1.19.7, platform=linux/amd64, user=root@8a0ee342e522, date=20230321-12:56:07, tags=netgo,builtinassets)" 2024-09-11T07:17:57.023 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 07:17:56 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-prometheus-a[62896]: ts=2024-09-11T07:17:56.719Z caller=main.go:570 level=info host_details="(Linux 5.14.0-505.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Thu Sep 5 07:54:07 UTC 2024 x86_64 smithi196 (none))" 2024-09-11T07:17:57.023 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 07:17:56 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-prometheus-a[62896]: ts=2024-09-11T07:17:56.719Z caller=main.go:571 level=info fd_limits="(soft=1048576, hard=1048576)" 2024-09-11T07:17:57.024 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 07:17:56 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-prometheus-a[62896]: ts=2024-09-11T07:17:56.719Z caller=main.go:572 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2024-09-11T07:17:57.024 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 07:17:56 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-prometheus-a[62896]: ts=2024-09-11T07:17:56.720Z caller=web.go:561 level=info component=web msg="Start listening for connections" address=:9095 2024-09-11T07:17:57.024 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 07:17:56 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-prometheus-a[62896]: ts=2024-09-11T07:17:56.721Z caller=main.go:1005 level=info msg="Starting TSDB ..." 2024-09-11T07:17:57.024 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 07:17:56 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-prometheus-a[62896]: ts=2024-09-11T07:17:56.722Z caller=tls_config.go:232 level=info component=web msg="Listening on" address=[::]:9095 2024-09-11T07:17:57.024 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 07:17:56 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-prometheus-a[62896]: ts=2024-09-11T07:17:56.722Z caller=tls_config.go:235 level=info component=web msg="TLS is disabled." http2=false address=[::]:9095 2024-09-11T07:17:57.024 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 07:17:56 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-prometheus-a[62896]: ts=2024-09-11T07:17:56.724Z caller=head.go:587 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2024-09-11T07:17:57.024 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 07:17:56 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-prometheus-a[62896]: ts=2024-09-11T07:17:56.724Z caller=head.go:658 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=2.8µs 2024-09-11T07:17:57.024 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 07:17:56 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-prometheus-a[62896]: ts=2024-09-11T07:17:56.724Z caller=head.go:664 level=info component=tsdb msg="Replaying WAL, this may take a while" 2024-09-11T07:17:57.024 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 07:17:56 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-prometheus-a[62896]: ts=2024-09-11T07:17:56.725Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=1 2024-09-11T07:17:57.024 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 07:17:56 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-prometheus-a[62896]: ts=2024-09-11T07:17:56.725Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=1 maxSegment=1 2024-09-11T07:17:57.024 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 07:17:56 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-prometheus-a[62896]: ts=2024-09-11T07:17:56.725Z caller=head.go:772 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=28.353µs wal_replay_duration=505.6µs wbl_replay_duration=129ns total_replay_duration=567.459µs 2024-09-11T07:17:57.024 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 07:17:56 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-prometheus-a[62896]: ts=2024-09-11T07:17:56.725Z caller=main.go:1026 level=info fs_type=XFS_SUPER_MAGIC 2024-09-11T07:17:57.025 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 07:17:56 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-prometheus-a[62896]: ts=2024-09-11T07:17:56.725Z caller=main.go:1029 level=info msg="TSDB started" 2024-09-11T07:17:57.025 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 07:17:56 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-prometheus-a[62896]: ts=2024-09-11T07:17:56.725Z caller=main.go:1209 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2024-09-11T07:17:57.025 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 07:17:56 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-prometheus-a[62896]: ts=2024-09-11T07:17:56.739Z caller=main.go:1246 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=13.808341ms db_storage=660ns remote_storage=974ns web_handler=393ns query_engine=556ns scrape=375.005µs scrape_sd=25.358µs notify=24.749µs notify_sd=6.773µs rules=13.128049ms tracing=6.711µs 2024-09-11T07:17:57.025 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 07:17:56 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-prometheus-a[62896]: ts=2024-09-11T07:17:56.739Z caller=main.go:990 level=info msg="Server is ready to receive web requests." 2024-09-11T07:17:57.025 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 07:17:56 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-prometheus-a[62896]: ts=2024-09-11T07:17:56.739Z caller=manager.go:974 level=info component="rule manager" msg="Starting rule manager..." 2024-09-11T07:17:57.025 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 07:17:56 smithi196 bash[62881]: 026075a3cf15820e30f8392a39c1e1f8568c677ae9bb86c91063927cddab2367 2024-09-11T07:17:57.025 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 07:17:56 smithi196 systemd[1]: Started Ceph prometheus.a for a81cffe6-700c-11ef-bcea-c7b262605968. 2024-09-11T07:17:57.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:17:57 smithi071 ceph-mon[27013]: pgmap v41: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 980 B/s rd, 98 B/s wr, 1 op/s; 115 B/s, 1 objects/s recovering 2024-09-11T07:17:57.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:17:57 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:17:57.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:17:57 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:17:57.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:17:57 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-11T07:17:57.090 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:17:57 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi071.front.sepia.ceph.com:9093"}]: dispatch 2024-09-11T07:17:57.090 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:17:57 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:17:57.090 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:17:57 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-11T07:17:57.090 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:17:57 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi196.front.sepia.ceph.com:9095"}]: dispatch 2024-09-11T07:17:57.090 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:17:57 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:17:57.090 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:17:57 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-11T07:17:57.090 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:17:57 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi196.front.sepia.ceph.com:3000"}]: dispatch 2024-09-11T07:17:57.090 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:17:57 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:17:57.090 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:17:57 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:17:57.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:17:57 smithi071 ceph-mon[37727]: pgmap v41: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 980 B/s rd, 98 B/s wr, 1 op/s; 115 B/s, 1 objects/s recovering 2024-09-11T07:17:57.091 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:17:57 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:17:57.091 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:17:57 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:17:57.091 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:17:57 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-11T07:17:57.091 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:17:57 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi071.front.sepia.ceph.com:9093"}]: dispatch 2024-09-11T07:17:57.091 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:17:57 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:17:57.091 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:17:57 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-11T07:17:57.091 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:17:57 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi196.front.sepia.ceph.com:9095"}]: dispatch 2024-09-11T07:17:57.091 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:17:57 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:17:57.091 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:17:57 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-11T07:17:57.091 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:17:57 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi196.front.sepia.ceph.com:3000"}]: dispatch 2024-09-11T07:17:57.091 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:17:57 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:17:57.091 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:17:57 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:17:57.091 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:17:56 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: [11/Sep/2024:07:17:56] ENGINE Bus STOPPING 2024-09-11T07:17:57.092 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:17:56 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: [11/Sep/2024:07:17:56] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-11T07:17:57.092 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:17:56 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: [11/Sep/2024:07:17:56] ENGINE Bus STOPPED 2024-09-11T07:17:57.092 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:17:56 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: [11/Sep/2024:07:17:56] ENGINE Bus STARTING 2024-09-11T07:17:57.092 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:17:56 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: [11/Sep/2024:07:17:56] ENGINE Serving on http://:::9283 2024-09-11T07:17:57.092 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:17:56 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: [11/Sep/2024:07:17:56] ENGINE Bus STARTED 2024-09-11T07:17:57.092 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:17:56 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: [11/Sep/2024:07:17:56] ENGINE Bus STOPPING 2024-09-11T07:17:57.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:17:57 smithi196 ceph-mon[30773]: pgmap v41: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 980 B/s rd, 98 B/s wr, 1 op/s; 115 B/s, 1 objects/s recovering 2024-09-11T07:17:57.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:17:57 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:17:57.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:17:57 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:17:57.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:17:57 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-11T07:17:57.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:17:57 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi071.front.sepia.ceph.com:9093"}]: dispatch 2024-09-11T07:17:57.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:17:57 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:17:57.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:17:57 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-11T07:17:57.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:17:57 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi196.front.sepia.ceph.com:9095"}]: dispatch 2024-09-11T07:17:57.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:17:57 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:17:57.286 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:17:57 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-11T07:17:57.286 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:17:57 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi196.front.sepia.ceph.com:3000"}]: dispatch 2024-09-11T07:17:57.286 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:17:57 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:17:57.286 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:17:57 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:17:57.797 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:17:57 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: [11/Sep/2024:07:17:57] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-11T07:17:57.798 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:17:57 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: [11/Sep/2024:07:17:57] ENGINE Bus STOPPED 2024-09-11T07:17:57.798 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:17:57 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: [11/Sep/2024:07:17:57] ENGINE Bus STARTING 2024-09-11T07:17:57.798 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:17:57 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: [11/Sep/2024:07:17:57] ENGINE Serving on http://:::9283 2024-09-11T07:17:57.798 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:17:57 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: [11/Sep/2024:07:17:57] ENGINE Bus STARTED 2024-09-11T07:17:57.798 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:17:57 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: [11/Sep/2024:07:17:57] ENGINE Bus STOPPING 2024-09-11T07:17:58.089 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:17:58 smithi071 ceph-mon[37727]: from='mon.0 v1:172.21.15.71:0/2658792160' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-11T07:17:58.089 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:17:58 smithi071 ceph-mon[37727]: from='mon.0 v1:172.21.15.71:0/2658792160' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi071.front.sepia.ceph.com:9093"}]: dispatch 2024-09-11T07:17:58.089 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:17:58 smithi071 ceph-mon[37727]: from='mon.0 v1:172.21.15.71:0/2658792160' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-11T07:17:58.089 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:17:58 smithi071 ceph-mon[37727]: from='mon.0 v1:172.21.15.71:0/2658792160' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi196.front.sepia.ceph.com:9095"}]: dispatch 2024-09-11T07:17:58.089 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:17:58 smithi071 ceph-mon[37727]: from='mon.0 v1:172.21.15.71:0/2658792160' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-11T07:17:58.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:17:58 smithi071 ceph-mon[37727]: from='mon.0 v1:172.21.15.71:0/2658792160' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi196.front.sepia.ceph.com:3000"}]: dispatch 2024-09-11T07:17:58.090 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:17:58 smithi071 ceph-mon[27013]: from='mon.0 v1:172.21.15.71:0/2658792160' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-11T07:17:58.090 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:17:58 smithi071 ceph-mon[27013]: from='mon.0 v1:172.21.15.71:0/2658792160' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi071.front.sepia.ceph.com:9093"}]: dispatch 2024-09-11T07:17:58.090 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:17:58 smithi071 ceph-mon[27013]: from='mon.0 v1:172.21.15.71:0/2658792160' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-11T07:17:58.090 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:17:58 smithi071 ceph-mon[27013]: from='mon.0 v1:172.21.15.71:0/2658792160' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi196.front.sepia.ceph.com:9095"}]: dispatch 2024-09-11T07:17:58.090 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:17:58 smithi071 ceph-mon[27013]: from='mon.0 v1:172.21.15.71:0/2658792160' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-11T07:17:58.090 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:17:58 smithi071 ceph-mon[27013]: from='mon.0 v1:172.21.15.71:0/2658792160' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi196.front.sepia.ceph.com:3000"}]: dispatch 2024-09-11T07:17:58.284 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:17:58 smithi196 ceph-mon[30773]: from='mon.0 v1:172.21.15.71:0/2658792160' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-11T07:17:58.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:17:58 smithi196 ceph-mon[30773]: from='mon.0 v1:172.21.15.71:0/2658792160' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi071.front.sepia.ceph.com:9093"}]: dispatch 2024-09-11T07:17:58.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:17:58 smithi196 ceph-mon[30773]: from='mon.0 v1:172.21.15.71:0/2658792160' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-11T07:17:58.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:17:58 smithi196 ceph-mon[30773]: from='mon.0 v1:172.21.15.71:0/2658792160' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi196.front.sepia.ceph.com:9095"}]: dispatch 2024-09-11T07:17:58.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:17:58 smithi196 ceph-mon[30773]: from='mon.0 v1:172.21.15.71:0/2658792160' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-11T07:17:58.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:17:58 smithi196 ceph-mon[30773]: from='mon.0 v1:172.21.15.71:0/2658792160' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi196.front.sepia.ceph.com:3000"}]: dispatch 2024-09-11T07:17:58.589 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:17:58 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: [11/Sep/2024:07:17:58] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-11T07:17:58.589 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:17:58 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: [11/Sep/2024:07:17:58] ENGINE Bus STOPPED 2024-09-11T07:17:58.589 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:17:58 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: [11/Sep/2024:07:17:58] ENGINE Bus STARTING 2024-09-11T07:17:58.589 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:17:58 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: [11/Sep/2024:07:17:58] ENGINE Serving on http://:::9283 2024-09-11T07:17:58.590 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:17:58 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: [11/Sep/2024:07:17:58] ENGINE Bus STARTED 2024-09-11T07:17:59.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:17:59 smithi196 ceph-mon[30773]: pgmap v42: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 85 B/s wr, 0 op/s; 100 B/s, 1 objects/s recovering 2024-09-11T07:17:59.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:17:59 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:17:59.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:17:59 smithi071 ceph-mon[27013]: pgmap v42: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 85 B/s wr, 0 op/s; 100 B/s, 1 objects/s recovering 2024-09-11T07:17:59.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:17:59 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:17:59.339 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:17:59 smithi071 ceph-mon[37727]: pgmap v42: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 85 B/s wr, 0 op/s; 100 B/s, 1 objects/s recovering 2024-09-11T07:17:59.339 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:17:59 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:18:01.089 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:18:00 smithi071 ceph-mon[37727]: pgmap v43: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 85 B/s wr, 1 op/s; 100 B/s, 1 objects/s recovering 2024-09-11T07:18:01.089 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:18:00 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:18:01.089 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:18:00 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:18:01.090 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:18:00 smithi071 ceph-mon[27013]: pgmap v43: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 85 B/s wr, 1 op/s; 100 B/s, 1 objects/s recovering 2024-09-11T07:18:01.090 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:18:00 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:18:01.090 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:18:00 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:18:01.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:18:00 smithi196 ceph-mon[30773]: pgmap v43: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 85 B/s wr, 1 op/s; 100 B/s, 1 objects/s recovering 2024-09-11T07:18:01.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:18:00 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:18:01.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:18:00 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:18:02.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:18:01 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:18:02.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:18:01 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:18:02.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:18:01 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:18:02.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:18:01 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:18:02.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:18:01 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:18:02.140 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:18:01 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:18:02.160 INFO:journalctl@ceph.alertmanager.a.smithi071.stdout:Sep 11 07:18:01 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-alertmanager-a[71989]: ts=2024-09-11T07:18:01.896Z caller=cluster.go:698 level=info component=cluster msg="gossip settled; proceeding" elapsed=10.003664674s 2024-09-11T07:18:02.160 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:18:01 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:18:02.160 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:18:01 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:18:02.160 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:18:01 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:18:02.160 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:18:01 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:18:02.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:18:01 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:18:02.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:18:01 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:18:02.161 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:18:01 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:18:02.161 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:18:01 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:18:02.161 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:18:01 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:18:02.161 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:18:01 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:18:02.161 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:18:01 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:18:02.161 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:18:01 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:18:03.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:18:02 smithi196 ceph-mon[30773]: pgmap v44: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:18:03.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:18:02 smithi071 ceph-mon[27013]: pgmap v44: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:18:03.339 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:18:02 smithi071 ceph-mon[37727]: pgmap v44: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:18:04.858 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 11 07:18:04 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-iscsi-iscsi-a[58197]: debug there is no tcmu-runner data available 2024-09-11T07:18:05.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:18:04 smithi071 ceph-mon[27013]: pgmap v45: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:18:05.089 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:18:04 smithi071 ceph-mon[37727]: pgmap v45: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:18:05.155 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:18:04 smithi196 ceph-mon[30773]: pgmap v45: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:18:06.157 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:18:05 smithi071 ceph-mon[27013]: from='client.34442 v1:172.21.15.196:0/1952921461' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:18:06.158 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:18:05 smithi071 ceph-mon[37727]: from='client.34442 v1:172.21.15.196:0/1952921461' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:18:06.173 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:18:05 smithi196 ceph-mon[30773]: from='client.34442 v1:172.21.15.196:0/1952921461' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:18:06.589 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:18:06 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: ::ffff:172.21.15.196 - - [11/Sep/2024:07:18:06] "GET /metrics HTTP/1.1" 200 219812 "" "Prometheus/2.43.0" 2024-09-11T07:18:06.753 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 07:18:06 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-x[32609]: ::ffff:172.21.15.196 - - [11/Sep/2024:07:18:06] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:18:07.034 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:18:06 smithi196 ceph-mon[30773]: pgmap v46: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:18:07.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:18:06 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:18:07.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:18:06 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:18:07.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:18:06 smithi071 ceph-mon[27013]: pgmap v46: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:18:07.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:18:06 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:18:07.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:18:06 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:18:07.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:18:06 smithi071 ceph-mon[37727]: pgmap v46: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:18:07.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:18:06 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:18:07.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:18:06 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:18:08.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:18:08 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:18:08.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:18:08 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:18:08.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:18:08 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:18:08.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:18:08 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:18:08.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:18:08 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:18:08.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:18:08 smithi196 ceph-mon[30773]: pgmap v47: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:18:08.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:18:08 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:18:08.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:18:08 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:18:08.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:18:08 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:18:08.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:18:08 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:18:08.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:18:08 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:18:08.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:18:08 smithi071 ceph-mon[27013]: pgmap v47: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:18:08.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:18:08 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:18:08.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:18:08 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:18:08.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:18:08 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:18:08.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:18:08 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:18:08.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:18:08 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:18:08.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:18:08 smithi071 ceph-mon[37727]: pgmap v47: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:18:10.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:18:10 smithi071 ceph-mon[27013]: pgmap v48: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:18:10.839 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:18:10 smithi071 ceph-mon[37727]: pgmap v48: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:18:11.034 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:18:10 smithi196 ceph-mon[30773]: pgmap v48: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:18:13.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:18:12 smithi196 ceph-mon[30773]: pgmap v49: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:18:13.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:18:12 smithi071 ceph-mon[27013]: pgmap v49: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:18:13.089 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:18:12 smithi071 ceph-mon[37727]: pgmap v49: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:18:14.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:18:13 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:18:14.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:18:13 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:18:14.089 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:18:13 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:18:15.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:18:14 smithi196 ceph-mon[30773]: pgmap v50: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:18:15.035 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 11 07:18:14 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-iscsi-iscsi-a[58197]: debug there is no tcmu-runner data available 2024-09-11T07:18:15.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:18:14 smithi071 ceph-mon[27013]: pgmap v50: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:18:15.089 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:18:14 smithi071 ceph-mon[37727]: pgmap v50: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:18:16.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:18:15 smithi196 ceph-mon[30773]: from='client.34442 v1:172.21.15.196:0/1952921461' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:18:16.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:18:15 smithi071 ceph-mon[27013]: from='client.34442 v1:172.21.15.196:0/1952921461' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:18:16.089 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:18:15 smithi071 ceph-mon[37727]: from='client.34442 v1:172.21.15.196:0/1952921461' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:18:16.589 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:18:16 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: ::ffff:172.21.15.196 - - [11/Sep/2024:07:18:16] "GET /metrics HTTP/1.1" 200 219800 "" "Prometheus/2.43.0" 2024-09-11T07:18:16.928 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:18:16 smithi196 ceph-mon[30773]: pgmap v51: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:18:16.929 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 07:18:16 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-x[32609]: ::ffff:172.21.15.196 - - [11/Sep/2024:07:18:16] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:18:17.089 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:18:16 smithi071 ceph-mon[37727]: pgmap v51: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:18:17.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:18:16 smithi071 ceph-mon[27013]: pgmap v51: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:18:19.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:18:18 smithi196 ceph-mon[30773]: pgmap v52: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:18:19.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:18:18 smithi071 ceph-mon[27013]: pgmap v52: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:18:19.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:18:18 smithi071 ceph-mon[37727]: pgmap v52: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:18:21.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:18:20 smithi196 ceph-mon[30773]: pgmap v53: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:18:21.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:18:20 smithi071 ceph-mon[27013]: pgmap v53: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:18:21.089 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:18:20 smithi071 ceph-mon[37727]: pgmap v53: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:18:23.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:18:22 smithi196 ceph-mon[30773]: pgmap v54: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:18:23.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:18:22 smithi071 ceph-mon[27013]: pgmap v54: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:18:23.089 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:18:22 smithi071 ceph-mon[37727]: pgmap v54: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:18:25.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:18:24 smithi196 ceph-mon[30773]: pgmap v55: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:18:25.035 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 11 07:18:24 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-iscsi-iscsi-a[58197]: debug there is no tcmu-runner data available 2024-09-11T07:18:25.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:18:24 smithi071 ceph-mon[27013]: pgmap v55: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:18:25.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:18:24 smithi071 ceph-mon[37727]: pgmap v55: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:18:26.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:18:25 smithi196 ceph-mon[30773]: from='client.34442 v1:172.21.15.196:0/1952921461' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:18:26.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:18:25 smithi071 ceph-mon[27013]: from='client.34442 v1:172.21.15.196:0/1952921461' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:18:26.089 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:18:25 smithi071 ceph-mon[37727]: from='client.34442 v1:172.21.15.196:0/1952921461' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:18:26.589 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:18:26 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: ::ffff:172.21.15.196 - - [11/Sep/2024:07:18:26] "GET /metrics HTTP/1.1" 200 219800 "" "Prometheus/2.43.0" 2024-09-11T07:18:27.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:18:26 smithi196 ceph-mon[30773]: pgmap v56: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:18:27.035 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 07:18:26 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-x[32609]: ::ffff:172.21.15.196 - - [11/Sep/2024:07:18:26] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:18:27.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:18:26 smithi071 ceph-mon[27013]: pgmap v56: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:18:27.089 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:18:26 smithi071 ceph-mon[37727]: pgmap v56: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:18:28.891 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:18:28 smithi071 ceph-mon[27013]: pgmap v57: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:18:28.892 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:18:28 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:18:28.892 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:18:28 smithi071 ceph-mon[37727]: pgmap v57: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:18:28.892 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:18:28 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:18:29.034 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:18:28 smithi196 ceph-mon[30773]: pgmap v57: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:18:29.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:18:28 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:18:30.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:18:30 smithi071 ceph-mon[27013]: pgmap v58: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:18:30.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:18:30 smithi071 ceph-mon[37727]: pgmap v58: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:18:31.034 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:18:30 smithi196 ceph-mon[30773]: pgmap v58: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:18:33.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:18:32 smithi196 ceph-mon[30773]: pgmap v59: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:18:33.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:18:32 smithi071 ceph-mon[27013]: pgmap v59: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:18:33.089 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:18:32 smithi071 ceph-mon[37727]: pgmap v59: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:18:35.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:18:34 smithi196 ceph-mon[30773]: pgmap v60: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:18:35.035 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 11 07:18:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-iscsi-iscsi-a[58197]: debug there is no tcmu-runner data available 2024-09-11T07:18:35.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:18:34 smithi071 ceph-mon[27013]: pgmap v60: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:18:35.089 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:18:34 smithi071 ceph-mon[37727]: pgmap v60: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:18:36.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:18:35 smithi196 ceph-mon[30773]: from='client.34442 v1:172.21.15.196:0/1952921461' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:18:36.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:18:35 smithi071 ceph-mon[27013]: from='client.34442 v1:172.21.15.196:0/1952921461' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:18:36.089 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:18:35 smithi071 ceph-mon[37727]: from='client.34442 v1:172.21.15.196:0/1952921461' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:18:36.589 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:18:36 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: ::ffff:172.21.15.196 - - [11/Sep/2024:07:18:36] "GET /metrics HTTP/1.1" 200 219772 "" "Prometheus/2.43.0" 2024-09-11T07:18:36.936 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:18:36 smithi196 ceph-mon[30773]: pgmap v61: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:18:36.937 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 07:18:36 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-x[32609]: ::ffff:172.21.15.196 - - [11/Sep/2024:07:18:36] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:18:37.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:18:36 smithi071 ceph-mon[27013]: pgmap v61: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:18:37.089 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:18:36 smithi071 ceph-mon[37727]: pgmap v61: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:18:39.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:18:38 smithi196 ceph-mon[30773]: pgmap v62: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:18:39.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:18:38 smithi071 ceph-mon[27013]: pgmap v62: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:18:39.089 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:18:38 smithi071 ceph-mon[37727]: pgmap v62: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:18:39.941 INFO:tasks.workunit.client.0.smithi071.stderr:Note: switching to 'd0e6828a2016d48cf25ad84064e50742bb1c39b9'. 2024-09-11T07:18:39.942 INFO:tasks.workunit.client.0.smithi071.stderr: 2024-09-11T07:18:39.942 INFO:tasks.workunit.client.0.smithi071.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-11T07:18:39.942 INFO:tasks.workunit.client.0.smithi071.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-11T07:18:39.942 INFO:tasks.workunit.client.0.smithi071.stderr:state without impacting any branches by switching back to a branch. 2024-09-11T07:18:39.942 INFO:tasks.workunit.client.0.smithi071.stderr: 2024-09-11T07:18:39.942 INFO:tasks.workunit.client.0.smithi071.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-11T07:18:39.942 INFO:tasks.workunit.client.0.smithi071.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-11T07:18:39.942 INFO:tasks.workunit.client.0.smithi071.stderr: 2024-09-11T07:18:39.942 INFO:tasks.workunit.client.0.smithi071.stderr: git switch -c 2024-09-11T07:18:39.942 INFO:tasks.workunit.client.0.smithi071.stderr: 2024-09-11T07:18:39.942 INFO:tasks.workunit.client.0.smithi071.stderr:Or undo this operation with: 2024-09-11T07:18:39.943 INFO:tasks.workunit.client.0.smithi071.stderr: 2024-09-11T07:18:39.943 INFO:tasks.workunit.client.0.smithi071.stderr: git switch - 2024-09-11T07:18:39.943 INFO:tasks.workunit.client.0.smithi071.stderr: 2024-09-11T07:18:39.943 INFO:tasks.workunit.client.0.smithi071.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-11T07:18:39.943 INFO:tasks.workunit.client.0.smithi071.stderr: 2024-09-11T07:18:39.943 INFO:tasks.workunit.client.0.smithi071.stderr:HEAD is now at d0e6828a201 Merge branch 'wip-63919-quincy' of https://github.com/guits/ceph into wip-guits-quincy-2024-09-10-2008-quincy 2024-09-11T07:18:39.949 DEBUG:teuthology.orchestra.run.smithi071:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-09-11T07:18:39.973 INFO:tasks.workunit.client.0.smithi071.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-09-11T07:18:39.975 INFO:tasks.workunit.client.0.smithi071.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-11T07:18:39.975 INFO:tasks.workunit.client.0.smithi071.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-09-11T07:18:40.745 INFO:tasks.workunit.client.0.smithi071.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-09-11T07:18:40.817 INFO:tasks.workunit.client.0.smithi071.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-09-11T07:18:40.859 INFO:tasks.workunit.client.0.smithi071.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-11T07:18:40.861 INFO:tasks.workunit.client.0.smithi071.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-11T07:18:40.861 INFO:tasks.workunit.client.0.smithi071.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-09-11T07:18:40.901 INFO:tasks.workunit.client.0.smithi071.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-11T07:18:40.904 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-11T07:18:40.905 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-09-11T07:18:40.918 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:18:40 smithi071 ceph-mon[27013]: pgmap v63: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:18:40.919 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:18:40 smithi071 ceph-mon[37727]: pgmap v63: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:18:40.934 INFO:tasks.workunit:Running workunits matching rados/test_python.sh on client.0... 2024-09-11T07:18:40.935 INFO:tasks.workunit:Running workunit rados/test_python.sh... 2024-09-11T07:18:40.935 DEBUG:teuthology.orchestra.run.smithi071:workunit test rados/test_python.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=d0e6828a2016d48cf25ad84064e50742bb1c39b9 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test_python.sh 2024-09-11T07:18:40.997 INFO:tasks.workunit.client.0.smithi071.stderr:+ ceph osd pool create rbd 2024-09-11T07:18:41.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:18:40 smithi196 ceph-mon[30773]: pgmap v63: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:18:41.758 INFO:tasks.workunit.client.0.smithi071.stderr:pool 'rbd' already exists 2024-09-11T07:18:41.769 INFO:tasks.workunit.client.0.smithi071.stderr:++ dirname /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test_python.sh 2024-09-11T07:18:41.769 INFO:tasks.workunit.client.0.smithi071.stderr:+ python3 -m pytest -v /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/../../../src/test/pybind/test_rados.py 2024-09-11T07:18:41.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:18:41 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/2697516705' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "rbd"}]: dispatch 2024-09-11T07:18:41.839 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:18:41 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/2697516705' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "rbd"}]: dispatch 2024-09-11T07:18:41.937 INFO:tasks.workunit.client.0.smithi071.stdout:============================= test session starts ============================== 2024-09-11T07:18:41.938 INFO:tasks.workunit.client.0.smithi071.stdout:platform linux -- Python 3.9.19, pytest-6.2.2, py-1.10.0, pluggy-0.13.1 -- /usr/bin/python3 2024-09-11T07:18:41.938 INFO:tasks.workunit.client.0.smithi071.stdout:cachedir: .pytest_cache 2024-09-11T07:18:41.938 INFO:tasks.workunit.client.0.smithi071.stdout:rootdir: /home/ubuntu/cephtest/clone.client.0/src/test/pybind, configfile: pytest.ini 2024-09-11T07:18:42.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:18:41 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/2697516705' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "rbd"}]: dispatch 2024-09-11T07:18:42.170 INFO:tasks.workunit.client.0.smithi071.stdout:collecting ... collected 89 items 2024-09-11T07:18:42.170 INFO:tasks.workunit.client.0.smithi071.stdout: 2024-09-11T07:18:42.179 INFO:tasks.workunit.client.0.smithi071.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::test_rados_init_error PASSED [ 1%] 2024-09-11T07:18:42.229 INFO:tasks.workunit.client.0.smithi071.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::test_rados_init PASSED [ 2%] 2024-09-11T07:18:42.244 INFO:tasks.workunit.client.0.smithi071.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::test_ioctx_context_manager PASSED [ 3%] 2024-09-11T07:18:42.255 INFO:tasks.workunit.client.0.smithi071.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::test_parse_argv PASSED [ 4%] 2024-09-11T07:18:42.261 INFO:tasks.workunit.client.0.smithi071.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::test_parse_argv_empty_str PASSED [ 5%] 2024-09-11T07:18:42.268 INFO:tasks.workunit.client.0.smithi071.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestRadosStateError::test_configuring PASSED [ 6%] 2024-09-11T07:18:42.282 INFO:tasks.workunit.client.0.smithi071.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestRadosStateError::test_connected PASSED [ 7%] 2024-09-11T07:18:42.295 INFO:tasks.workunit.client.0.smithi071.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestRadosStateError::test_shutdown PASSED [ 8%] 2024-09-11T07:18:42.315 INFO:tasks.workunit.client.0.smithi071.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestRados::test_ping_monitor PASSED [ 10%] 2024-09-11T07:18:42.332 INFO:tasks.workunit.client.0.smithi071.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestRados::test_annotations PASSED [ 11%] 2024-09-11T07:18:43.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:18:42 smithi196 ceph-mon[30773]: pgmap v64: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:18:43.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:18:42 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/2697516705' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "rbd"}]': finished 2024-09-11T07:18:43.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:18:42 smithi196 ceph-mon[30773]: osdmap e64: 8 total, 8 up, 8 in 2024-09-11T07:18:43.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:18:42 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/2697516705' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "rbd"}]: dispatch 2024-09-11T07:18:43.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:18:42 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/1779660088' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T07:18:43.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:18:42 smithi071 ceph-mon[27013]: pgmap v64: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:18:43.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:18:42 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/2697516705' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "rbd"}]': finished 2024-09-11T07:18:43.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:18:42 smithi071 ceph-mon[27013]: osdmap e64: 8 total, 8 up, 8 in 2024-09-11T07:18:43.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:18:42 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/2697516705' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "rbd"}]: dispatch 2024-09-11T07:18:43.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:18:42 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/1779660088' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T07:18:43.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:18:42 smithi071 ceph-mon[37727]: pgmap v64: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-11T07:18:43.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:18:42 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/2697516705' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "rbd"}]': finished 2024-09-11T07:18:43.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:18:42 smithi071 ceph-mon[37727]: osdmap e64: 8 total, 8 up, 8 in 2024-09-11T07:18:43.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:18:42 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/2697516705' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "rbd"}]: dispatch 2024-09-11T07:18:43.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:18:42 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/1779660088' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T07:18:43.672 INFO:tasks.workunit.client.0.smithi071.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestRados::test_create PASSED [ 12%] 2024-09-11T07:18:44.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:18:43 smithi196 ceph-mon[30773]: osdmap e65: 8 total, 8 up, 8 in 2024-09-11T07:18:44.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:18:43 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:18:44.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:18:43 smithi071 ceph-mon[27013]: osdmap e65: 8 total, 8 up, 8 in 2024-09-11T07:18:44.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:18:43 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:18:44.089 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:18:43 smithi071 ceph-mon[37727]: osdmap e65: 8 total, 8 up, 8 in 2024-09-11T07:18:44.089 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:18:43 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:18:45.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:18:44 smithi196 ceph-mon[30773]: pgmap v67: 196 pgs: 64 unknown, 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-11T07:18:45.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:18:44 smithi196 ceph-mon[30773]: osdmap e66: 8 total, 8 up, 8 in 2024-09-11T07:18:45.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:18:44 smithi196 ceph-mon[30773]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T07:18:45.035 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 11 07:18:44 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-iscsi-iscsi-a[58197]: debug there is no tcmu-runner data available 2024-09-11T07:18:45.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:18:44 smithi071 ceph-mon[27013]: pgmap v67: 196 pgs: 64 unknown, 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-11T07:18:45.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:18:44 smithi071 ceph-mon[27013]: osdmap e66: 8 total, 8 up, 8 in 2024-09-11T07:18:45.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:18:44 smithi071 ceph-mon[27013]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T07:18:45.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:18:44 smithi071 ceph-mon[37727]: pgmap v67: 196 pgs: 64 unknown, 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-11T07:18:45.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:18:44 smithi071 ceph-mon[37727]: osdmap e66: 8 total, 8 up, 8 in 2024-09-11T07:18:45.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:18:44 smithi071 ceph-mon[37727]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T07:18:45.702 INFO:tasks.workunit.client.0.smithi071.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestRados::test_create_utf8 PASSED [ 13%] 2024-09-11T07:18:46.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:18:45 smithi196 ceph-mon[30773]: from='client.34442 v1:172.21.15.196:0/1952921461' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:18:46.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:18:45 smithi196 ceph-mon[30773]: osdmap e67: 8 total, 8 up, 8 in 2024-09-11T07:18:46.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:18:45 smithi071 ceph-mon[27013]: from='client.34442 v1:172.21.15.196:0/1952921461' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:18:46.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:18:45 smithi071 ceph-mon[27013]: osdmap e67: 8 total, 8 up, 8 in 2024-09-11T07:18:46.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:18:45 smithi071 ceph-mon[37727]: from='client.34442 v1:172.21.15.196:0/1952921461' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:18:46.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:18:45 smithi071 ceph-mon[37727]: osdmap e67: 8 total, 8 up, 8 in 2024-09-11T07:18:46.589 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:18:46 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: ::ffff:172.21.15.196 - - [11/Sep/2024:07:18:46] "GET /metrics HTTP/1.1" 200 224324 "" "Prometheus/2.43.0" 2024-09-11T07:18:47.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:18:46 smithi196 ceph-mon[30773]: pgmap v70: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:18:47.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:18:46 smithi196 ceph-mon[30773]: osdmap e68: 8 total, 8 up, 8 in 2024-09-11T07:18:47.035 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 07:18:46 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-x[32609]: ::ffff:172.21.15.196 - - [11/Sep/2024:07:18:46] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:18:47.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:18:46 smithi071 ceph-mon[27013]: pgmap v70: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:18:47.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:18:46 smithi071 ceph-mon[27013]: osdmap e68: 8 total, 8 up, 8 in 2024-09-11T07:18:47.089 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:18:46 smithi071 ceph-mon[37727]: pgmap v70: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:18:47.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:18:46 smithi071 ceph-mon[37727]: osdmap e68: 8 total, 8 up, 8 in 2024-09-11T07:18:47.712 INFO:tasks.workunit.client.0.smithi071.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestRados::test_pool_lookup_utf8 PASSED [ 14%] 2024-09-11T07:18:48.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:18:47 smithi196 ceph-mon[30773]: osdmap e69: 8 total, 8 up, 8 in 2024-09-11T07:18:48.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:18:47 smithi071 ceph-mon[27013]: osdmap e69: 8 total, 8 up, 8 in 2024-09-11T07:18:48.089 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:18:47 smithi071 ceph-mon[37727]: osdmap e69: 8 total, 8 up, 8 in 2024-09-11T07:18:49.034 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:18:48 smithi196 ceph-mon[30773]: pgmap v73: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:18:49.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:18:48 smithi196 ceph-mon[30773]: osdmap e70: 8 total, 8 up, 8 in 2024-09-11T07:18:49.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:18:48 smithi071 ceph-mon[27013]: pgmap v73: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:18:49.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:18:48 smithi071 ceph-mon[27013]: osdmap e70: 8 total, 8 up, 8 in 2024-09-11T07:18:49.089 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:18:48 smithi071 ceph-mon[37727]: pgmap v73: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:18:49.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:18:48 smithi071 ceph-mon[37727]: osdmap e70: 8 total, 8 up, 8 in 2024-09-11T07:18:49.717 INFO:tasks.workunit.client.0.smithi071.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestRados::test_eexist PASSED [ 15%] 2024-09-11T07:18:50.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:18:49 smithi196 ceph-mon[30773]: osdmap e71: 8 total, 8 up, 8 in 2024-09-11T07:18:50.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:18:49 smithi196 ceph-mon[30773]: osdmap e72: 8 total, 8 up, 8 in 2024-09-11T07:18:50.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:18:49 smithi071 ceph-mon[27013]: osdmap e71: 8 total, 8 up, 8 in 2024-09-11T07:18:50.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:18:49 smithi071 ceph-mon[27013]: osdmap e72: 8 total, 8 up, 8 in 2024-09-11T07:18:50.089 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:18:49 smithi071 ceph-mon[37727]: osdmap e71: 8 total, 8 up, 8 in 2024-09-11T07:18:50.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:18:49 smithi071 ceph-mon[37727]: osdmap e72: 8 total, 8 up, 8 in 2024-09-11T07:18:51.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:18:50 smithi196 ceph-mon[30773]: pgmap v76: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:18:51.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:18:50 smithi071 ceph-mon[27013]: pgmap v76: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:18:51.089 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:18:50 smithi071 ceph-mon[37727]: pgmap v76: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:18:52.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:18:51 smithi196 ceph-mon[30773]: osdmap e73: 8 total, 8 up, 8 in 2024-09-11T07:18:52.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:18:51 smithi071 ceph-mon[27013]: osdmap e73: 8 total, 8 up, 8 in 2024-09-11T07:18:52.089 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:18:51 smithi071 ceph-mon[37727]: osdmap e73: 8 total, 8 up, 8 in 2024-09-11T07:18:53.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:18:52 smithi196 ceph-mon[30773]: pgmap v79: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:18:53.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:18:52 smithi196 ceph-mon[30773]: osdmap e74: 8 total, 8 up, 8 in 2024-09-11T07:18:53.089 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:18:52 smithi071 ceph-mon[37727]: pgmap v79: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:18:53.089 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:18:52 smithi071 ceph-mon[37727]: osdmap e74: 8 total, 8 up, 8 in 2024-09-11T07:18:53.090 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:18:52 smithi071 ceph-mon[27013]: pgmap v79: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:18:53.090 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:18:52 smithi071 ceph-mon[27013]: osdmap e74: 8 total, 8 up, 8 in 2024-09-11T07:18:54.034 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:18:53 smithi196 ceph-mon[30773]: osdmap e75: 8 total, 8 up, 8 in 2024-09-11T07:18:54.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:18:53 smithi071 ceph-mon[27013]: osdmap e75: 8 total, 8 up, 8 in 2024-09-11T07:18:54.089 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:18:53 smithi071 ceph-mon[37727]: osdmap e75: 8 total, 8 up, 8 in 2024-09-11T07:18:55.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:18:54 smithi196 ceph-mon[30773]: pgmap v82: 260 pgs: 96 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:18:55.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:18:54 smithi196 ceph-mon[30773]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T07:18:55.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:18:54 smithi196 ceph-mon[30773]: osdmap e76: 8 total, 8 up, 8 in 2024-09-11T07:18:55.035 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 11 07:18:54 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-iscsi-iscsi-a[58197]: debug there is no tcmu-runner data available 2024-09-11T07:18:55.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:18:54 smithi071 ceph-mon[27013]: pgmap v82: 260 pgs: 96 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:18:55.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:18:54 smithi071 ceph-mon[27013]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T07:18:55.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:18:54 smithi071 ceph-mon[27013]: osdmap e76: 8 total, 8 up, 8 in 2024-09-11T07:18:55.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:18:54 smithi071 ceph-mon[37727]: pgmap v82: 260 pgs: 96 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:18:55.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:18:54 smithi071 ceph-mon[37727]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T07:18:55.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:18:54 smithi071 ceph-mon[37727]: osdmap e76: 8 total, 8 up, 8 in 2024-09-11T07:18:56.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:18:55 smithi196 ceph-mon[30773]: from='client.34442 v1:172.21.15.196:0/1952921461' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:18:56.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:18:55 smithi196 ceph-mon[30773]: osdmap e77: 8 total, 8 up, 8 in 2024-09-11T07:18:56.339 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:18:56 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: ::ffff:172.21.15.196 - - [11/Sep/2024:07:18:56] "GET /metrics HTTP/1.1" 200 224324 "" "Prometheus/2.43.0" 2024-09-11T07:18:56.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:18:55 smithi071 ceph-mon[27013]: from='client.34442 v1:172.21.15.196:0/1952921461' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:18:56.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:18:55 smithi071 ceph-mon[27013]: osdmap e77: 8 total, 8 up, 8 in 2024-09-11T07:18:56.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:18:55 smithi071 ceph-mon[37727]: from='client.34442 v1:172.21.15.196:0/1952921461' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:18:56.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:18:55 smithi071 ceph-mon[37727]: osdmap e77: 8 total, 8 up, 8 in 2024-09-11T07:18:57.001 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 07:18:56 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-x[32609]: ::ffff:172.21.15.196 - - [11/Sep/2024:07:18:56] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:18:57.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:18:57 smithi196 ceph-mon[30773]: pgmap v85: 196 pgs: 196 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:18:57.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:18:57 smithi196 ceph-mon[30773]: osdmap e78: 8 total, 8 up, 8 in 2024-09-11T07:18:57.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:18:57 smithi196 ceph-mon[30773]: osdmap e79: 8 total, 8 up, 8 in 2024-09-11T07:18:57.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:18:57 smithi071 ceph-mon[27013]: pgmap v85: 196 pgs: 196 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:18:57.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:18:57 smithi071 ceph-mon[27013]: osdmap e78: 8 total, 8 up, 8 in 2024-09-11T07:18:57.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:18:57 smithi071 ceph-mon[27013]: osdmap e79: 8 total, 8 up, 8 in 2024-09-11T07:18:57.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:18:57 smithi071 ceph-mon[37727]: pgmap v85: 196 pgs: 196 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:18:57.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:18:57 smithi071 ceph-mon[37727]: osdmap e78: 8 total, 8 up, 8 in 2024-09-11T07:18:57.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:18:57 smithi071 ceph-mon[37727]: osdmap e79: 8 total, 8 up, 8 in 2024-09-11T07:18:57.773 INFO:tasks.workunit.client.0.smithi071.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestRados::test_list_pools PASSED [ 16%] 2024-09-11T07:18:59.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:18:58 smithi196 ceph-mon[30773]: pgmap v88: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:18:59.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:18:58 smithi196 ceph-mon[30773]: osdmap e80: 8 total, 8 up, 8 in 2024-09-11T07:18:59.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:18:58 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:18:59.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:18:58 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:18:59.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:18:58 smithi071 ceph-mon[27013]: pgmap v88: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:18:59.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:18:58 smithi071 ceph-mon[27013]: osdmap e80: 8 total, 8 up, 8 in 2024-09-11T07:18:59.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:18:58 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:18:59.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:18:58 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:18:59.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:18:58 smithi071 ceph-mon[37727]: pgmap v88: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:18:59.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:18:58 smithi071 ceph-mon[37727]: osdmap e80: 8 total, 8 up, 8 in 2024-09-11T07:18:59.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:18:58 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:18:59.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:18:58 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:19:00.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:18:59 smithi196 ceph-mon[30773]: osdmap e81: 8 total, 8 up, 8 in 2024-09-11T07:19:00.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:18:59 smithi071 ceph-mon[27013]: osdmap e81: 8 total, 8 up, 8 in 2024-09-11T07:19:00.089 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:18:59 smithi071 ceph-mon[37727]: osdmap e81: 8 total, 8 up, 8 in 2024-09-11T07:19:00.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:00 smithi071 ceph-mon[27013]: pgmap v91: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:19:00.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:00 smithi071 ceph-mon[27013]: osdmap e82: 8 total, 8 up, 8 in 2024-09-11T07:19:00.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:00 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/435035031' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "foo", "tierpool": "foo-cache", "force_nonempty": ""}]: dispatch 2024-09-11T07:19:00.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:00 smithi071 ceph-mon[27013]: from='client.34745 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "foo", "tierpool": "foo-cache", "force_nonempty": ""}]: dispatch 2024-09-11T07:19:00.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:00 smithi071 ceph-mon[37727]: pgmap v91: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:19:00.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:00 smithi071 ceph-mon[37727]: osdmap e82: 8 total, 8 up, 8 in 2024-09-11T07:19:00.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:00 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/435035031' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "foo", "tierpool": "foo-cache", "force_nonempty": ""}]: dispatch 2024-09-11T07:19:00.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:00 smithi071 ceph-mon[37727]: from='client.34745 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "foo", "tierpool": "foo-cache", "force_nonempty": ""}]: dispatch 2024-09-11T07:19:01.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:00 smithi196 ceph-mon[30773]: pgmap v91: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:19:01.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:00 smithi196 ceph-mon[30773]: osdmap e82: 8 total, 8 up, 8 in 2024-09-11T07:19:01.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:00 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/435035031' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "foo", "tierpool": "foo-cache", "force_nonempty": ""}]: dispatch 2024-09-11T07:19:01.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:00 smithi196 ceph-mon[30773]: from='client.34745 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "foo", "tierpool": "foo-cache", "force_nonempty": ""}]: dispatch 2024-09-11T07:19:02.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:01 smithi071 ceph-mon[27013]: from='client.34745 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "foo", "tierpool": "foo-cache", "force_nonempty": ""}]': finished 2024-09-11T07:19:02.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:01 smithi071 ceph-mon[27013]: osdmap e83: 8 total, 8 up, 8 in 2024-09-11T07:19:02.090 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:01 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/435035031' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "foo-cache", "tierpool": "foo-cache", "mode": "readonly", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T07:19:02.090 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:01 smithi071 ceph-mon[27013]: from='client.34745 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "foo-cache", "tierpool": "foo-cache", "mode": "readonly", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T07:19:02.090 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:01 smithi071 ceph-mon[27013]: from='client.34745 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "foo-cache", "tierpool": "foo-cache", "mode": "readonly", "yes_i_really_mean_it": true}]': finished 2024-09-11T07:19:02.090 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:01 smithi071 ceph-mon[27013]: osdmap e84: 8 total, 8 up, 8 in 2024-09-11T07:19:02.090 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:01 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/435035031' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "foo", "tierpool": "foo-cache"}]: dispatch 2024-09-11T07:19:02.090 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:01 smithi071 ceph-mon[27013]: from='client.34745 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "foo", "tierpool": "foo-cache"}]: dispatch 2024-09-11T07:19:02.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:01 smithi071 ceph-mon[37727]: from='client.34745 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "foo", "tierpool": "foo-cache", "force_nonempty": ""}]': finished 2024-09-11T07:19:02.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:01 smithi071 ceph-mon[37727]: osdmap e83: 8 total, 8 up, 8 in 2024-09-11T07:19:02.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:01 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/435035031' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "foo-cache", "tierpool": "foo-cache", "mode": "readonly", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T07:19:02.091 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:01 smithi071 ceph-mon[37727]: from='client.34745 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "foo-cache", "tierpool": "foo-cache", "mode": "readonly", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T07:19:02.091 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:01 smithi071 ceph-mon[37727]: from='client.34745 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "foo-cache", "tierpool": "foo-cache", "mode": "readonly", "yes_i_really_mean_it": true}]': finished 2024-09-11T07:19:02.091 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:01 smithi071 ceph-mon[37727]: osdmap e84: 8 total, 8 up, 8 in 2024-09-11T07:19:02.091 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:01 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/435035031' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "foo", "tierpool": "foo-cache"}]: dispatch 2024-09-11T07:19:02.091 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:01 smithi071 ceph-mon[37727]: from='client.34745 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "foo", "tierpool": "foo-cache"}]: dispatch 2024-09-11T07:19:02.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:01 smithi196 ceph-mon[30773]: from='client.34745 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "foo", "tierpool": "foo-cache", "force_nonempty": ""}]': finished 2024-09-11T07:19:02.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:01 smithi196 ceph-mon[30773]: osdmap e83: 8 total, 8 up, 8 in 2024-09-11T07:19:02.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:01 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/435035031' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "foo-cache", "tierpool": "foo-cache", "mode": "readonly", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T07:19:02.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:01 smithi196 ceph-mon[30773]: from='client.34745 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "foo-cache", "tierpool": "foo-cache", "mode": "readonly", "yes_i_really_mean_it": true}]: dispatch 2024-09-11T07:19:02.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:01 smithi196 ceph-mon[30773]: from='client.34745 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "foo-cache", "tierpool": "foo-cache", "mode": "readonly", "yes_i_really_mean_it": true}]': finished 2024-09-11T07:19:02.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:01 smithi196 ceph-mon[30773]: osdmap e84: 8 total, 8 up, 8 in 2024-09-11T07:19:02.286 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:01 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/435035031' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "foo", "tierpool": "foo-cache"}]: dispatch 2024-09-11T07:19:02.286 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:01 smithi196 ceph-mon[30773]: from='client.34745 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "foo", "tierpool": "foo-cache"}]: dispatch 2024-09-11T07:19:03.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:02 smithi071 ceph-mon[27013]: pgmap v94: 228 pgs: 32 unknown, 32 creating+peering, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:19:03.090 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:02 smithi071 ceph-mon[27013]: from='client.34745 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "foo", "tierpool": "foo-cache"}]': finished 2024-09-11T07:19:03.090 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:02 smithi071 ceph-mon[27013]: osdmap e85: 8 total, 8 up, 8 in 2024-09-11T07:19:03.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:02 smithi071 ceph-mon[37727]: pgmap v94: 228 pgs: 32 unknown, 32 creating+peering, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:19:03.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:02 smithi071 ceph-mon[37727]: from='client.34745 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "foo", "tierpool": "foo-cache"}]': finished 2024-09-11T07:19:03.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:02 smithi071 ceph-mon[37727]: osdmap e85: 8 total, 8 up, 8 in 2024-09-11T07:19:03.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:02 smithi196 ceph-mon[30773]: pgmap v94: 228 pgs: 32 unknown, 32 creating+peering, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:19:03.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:02 smithi196 ceph-mon[30773]: from='client.34745 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "foo", "tierpool": "foo-cache"}]': finished 2024-09-11T07:19:03.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:02 smithi196 ceph-mon[30773]: osdmap e85: 8 total, 8 up, 8 in 2024-09-11T07:19:04.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:03 smithi071 ceph-mon[27013]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T07:19:04.090 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:03 smithi071 ceph-mon[27013]: osdmap e86: 8 total, 8 up, 8 in 2024-09-11T07:19:04.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:03 smithi071 ceph-mon[37727]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T07:19:04.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:03 smithi071 ceph-mon[37727]: osdmap e86: 8 total, 8 up, 8 in 2024-09-11T07:19:04.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:03 smithi196 ceph-mon[30773]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T07:19:04.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:03 smithi196 ceph-mon[30773]: osdmap e86: 8 total, 8 up, 8 in 2024-09-11T07:19:04.796 INFO:tasks.workunit.client.0.smithi071.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestRados::test_get_pool_base_tier PASSED [ 17%] 2024-09-11T07:19:04.818 INFO:tasks.workunit.client.0.smithi071.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestRados::test_get_fsid PASSED [ 19%] 2024-09-11T07:19:05.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:04 smithi196 ceph-mon[30773]: pgmap v97: 228 pgs: 32 unknown, 32 creating+peering, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:19:05.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:04 smithi196 ceph-mon[30773]: osdmap e87: 8 total, 8 up, 8 in 2024-09-11T07:19:05.035 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 11 07:19:04 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-iscsi-iscsi-a[58197]: debug there is no tcmu-runner data available 2024-09-11T07:19:05.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:04 smithi071 ceph-mon[27013]: pgmap v97: 228 pgs: 32 unknown, 32 creating+peering, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:19:05.090 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:04 smithi071 ceph-mon[27013]: osdmap e87: 8 total, 8 up, 8 in 2024-09-11T07:19:05.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:04 smithi071 ceph-mon[37727]: pgmap v97: 228 pgs: 32 unknown, 32 creating+peering, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:19:05.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:04 smithi071 ceph-mon[37727]: osdmap e87: 8 total, 8 up, 8 in 2024-09-11T07:19:05.821 INFO:tasks.workunit.client.0.smithi071.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestRados::test_blocklist_add PASSED [ 20%] 2024-09-11T07:19:05.843 INFO:tasks.workunit.client.0.smithi071.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestRados::test_get_cluster_stats PASSED [ 21%] 2024-09-11T07:19:05.862 INFO:tasks.workunit.client.0.smithi071.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestRados::test_monitor_log PASSED [ 22%] 2024-09-11T07:19:06.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:05 smithi071 ceph-mon[27013]: from='client.34442 v1:172.21.15.196:0/1952921461' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:19:06.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:05 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/1483997258' entity='client.admin' cmd=[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "1.2.3.4/123", "expire": 1.0}]: dispatch 2024-09-11T07:19:06.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:05 smithi071 ceph-mon[27013]: from='client.34757 ' entity='client.admin' cmd=[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "1.2.3.4/123", "expire": 1.0}]: dispatch 2024-09-11T07:19:06.089 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:05 smithi071 ceph-mon[37727]: from='client.34442 v1:172.21.15.196:0/1952921461' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:19:06.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:05 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/1483997258' entity='client.admin' cmd=[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "1.2.3.4/123", "expire": 1.0}]: dispatch 2024-09-11T07:19:06.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:05 smithi071 ceph-mon[37727]: from='client.34757 ' entity='client.admin' cmd=[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "1.2.3.4/123", "expire": 1.0}]: dispatch 2024-09-11T07:19:06.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:05 smithi196 ceph-mon[30773]: from='client.34442 v1:172.21.15.196:0/1952921461' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:19:06.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:05 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/1483997258' entity='client.admin' cmd=[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "1.2.3.4/123", "expire": 1.0}]: dispatch 2024-09-11T07:19:06.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:05 smithi196 ceph-mon[30773]: from='client.34757 ' entity='client.admin' cmd=[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "1.2.3.4/123", "expire": 1.0}]: dispatch 2024-09-11T07:19:06.589 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:19:06 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: ::ffff:172.21.15.196 - - [11/Sep/2024:07:19:06] "GET /metrics HTTP/1.1" 200 222321 "" "Prometheus/2.43.0" 2024-09-11T07:19:06.829 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 07:19:06 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-x[32609]: ::ffff:172.21.15.196 - - [11/Sep/2024:07:19:06] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:19:07.089 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:06 smithi071 ceph-mon[37727]: pgmap v100: 164 pgs: 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:19:07.089 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:06 smithi071 ceph-mon[37727]: from='client.34757 ' entity='client.admin' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "1.2.3.4/123", "expire": 1.0}]': finished 2024-09-11T07:19:07.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:06 smithi071 ceph-mon[37727]: osdmap e88: 8 total, 8 up, 8 in 2024-09-11T07:19:07.090 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:06 smithi071 ceph-mon[27013]: pgmap v100: 164 pgs: 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:19:07.090 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:06 smithi071 ceph-mon[27013]: from='client.34757 ' entity='client.admin' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "1.2.3.4/123", "expire": 1.0}]': finished 2024-09-11T07:19:07.090 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:06 smithi071 ceph-mon[27013]: osdmap e88: 8 total, 8 up, 8 in 2024-09-11T07:19:07.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:06 smithi196 ceph-mon[30773]: pgmap v100: 164 pgs: 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:19:07.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:06 smithi196 ceph-mon[30773]: from='client.34757 ' entity='client.admin' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "1.2.3.4/123", "expire": 1.0}]': finished 2024-09-11T07:19:07.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:06 smithi196 ceph-mon[30773]: osdmap e88: 8 total, 8 up, 8 in 2024-09-11T07:19:08.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:07 smithi196 ceph-mon[30773]: osdmap e89: 8 total, 8 up, 8 in 2024-09-11T07:19:08.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:07 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/1320390488' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:19:08.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:07 smithi196 ceph-mon[30773]: from='client.34775 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:19:08.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:07 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:19:08.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:07 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:19:08.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:07 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:19:08.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:07 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:19:08.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:07 smithi196 ceph-mon[30773]: from='client.34775 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:19:08.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:07 smithi196 ceph-mon[30773]: osdmap e90: 8 total, 8 up, 8 in 2024-09-11T07:19:08.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:07 smithi071 ceph-mon[27013]: osdmap e89: 8 total, 8 up, 8 in 2024-09-11T07:19:08.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:07 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/1320390488' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:19:08.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:07 smithi071 ceph-mon[27013]: from='client.34775 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:19:08.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:07 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:19:08.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:07 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:19:08.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:07 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:19:08.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:07 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:19:08.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:07 smithi071 ceph-mon[27013]: from='client.34775 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:19:08.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:07 smithi071 ceph-mon[27013]: osdmap e90: 8 total, 8 up, 8 in 2024-09-11T07:19:08.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:07 smithi071 ceph-mon[37727]: osdmap e89: 8 total, 8 up, 8 in 2024-09-11T07:19:08.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:07 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/1320390488' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:19:08.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:07 smithi071 ceph-mon[37727]: from='client.34775 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:19:08.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:07 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:19:08.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:07 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:19:08.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:07 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:19:08.341 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:07 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:19:08.341 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:07 smithi071 ceph-mon[37727]: from='client.34775 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:19:08.341 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:07 smithi071 ceph-mon[37727]: osdmap e90: 8 total, 8 up, 8 in 2024-09-11T07:19:08.580 INFO:tasks.workunit.client.0.smithi071.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_get_last_version PASSED [ 23%] 2024-09-11T07:19:09.284 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:08 smithi196 ceph-mon[30773]: pgmap v103: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:19:09.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:08 smithi196 ceph-mon[30773]: osdmap e91: 8 total, 8 up, 8 in 2024-09-11T07:19:09.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:08 smithi071 ceph-mon[27013]: pgmap v103: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:19:09.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:08 smithi071 ceph-mon[27013]: osdmap e91: 8 total, 8 up, 8 in 2024-09-11T07:19:09.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:08 smithi071 ceph-mon[37727]: pgmap v103: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:19:09.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:08 smithi071 ceph-mon[37727]: osdmap e91: 8 total, 8 up, 8 in 2024-09-11T07:19:10.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:10 smithi071 ceph-mon[27013]: pgmap v106: 164 pgs: 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:19:10.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:10 smithi071 ceph-mon[27013]: osdmap e92: 8 total, 8 up, 8 in 2024-09-11T07:19:10.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:10 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/2843761434' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:19:10.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:10 smithi071 ceph-mon[27013]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T07:19:10.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:10 smithi071 ceph-mon[37727]: pgmap v106: 164 pgs: 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:19:10.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:10 smithi071 ceph-mon[37727]: osdmap e92: 8 total, 8 up, 8 in 2024-09-11T07:19:10.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:10 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/2843761434' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:19:10.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:10 smithi071 ceph-mon[37727]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T07:19:11.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:10 smithi196 ceph-mon[30773]: pgmap v106: 164 pgs: 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:19:11.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:10 smithi196 ceph-mon[30773]: osdmap e92: 8 total, 8 up, 8 in 2024-09-11T07:19:11.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:10 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/2843761434' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:19:11.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:10 smithi196 ceph-mon[30773]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T07:19:11.584 INFO:tasks.workunit.client.0.smithi071.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_get_stats PASSED [ 24%] 2024-09-11T07:19:11.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:11 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/2843761434' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:19:11.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:11 smithi071 ceph-mon[27013]: osdmap e93: 8 total, 8 up, 8 in 2024-09-11T07:19:11.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:11 smithi071 ceph-mon[27013]: osdmap e94: 8 total, 8 up, 8 in 2024-09-11T07:19:11.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:11 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/2843761434' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:19:11.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:11 smithi071 ceph-mon[37727]: osdmap e93: 8 total, 8 up, 8 in 2024-09-11T07:19:11.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:11 smithi071 ceph-mon[37727]: osdmap e94: 8 total, 8 up, 8 in 2024-09-11T07:19:12.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:11 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/2843761434' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:19:12.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:11 smithi196 ceph-mon[30773]: osdmap e93: 8 total, 8 up, 8 in 2024-09-11T07:19:12.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:11 smithi196 ceph-mon[30773]: osdmap e94: 8 total, 8 up, 8 in 2024-09-11T07:19:13.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:12 smithi196 ceph-mon[30773]: pgmap v109: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:19:13.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:12 smithi071 ceph-mon[27013]: pgmap v109: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:19:13.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:12 smithi071 ceph-mon[37727]: pgmap v109: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:19:14.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:13 smithi196 ceph-mon[30773]: osdmap e95: 8 total, 8 up, 8 in 2024-09-11T07:19:14.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:13 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:19:14.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:13 smithi071 ceph-mon[27013]: osdmap e95: 8 total, 8 up, 8 in 2024-09-11T07:19:14.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:13 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:19:14.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:13 smithi071 ceph-mon[37727]: osdmap e95: 8 total, 8 up, 8 in 2024-09-11T07:19:14.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:13 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:19:15.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:14 smithi196 ceph-mon[30773]: pgmap v112: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:19:15.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:14 smithi196 ceph-mon[30773]: osdmap e96: 8 total, 8 up, 8 in 2024-09-11T07:19:15.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:14 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/3243240492' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:19:15.035 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 11 07:19:14 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-iscsi-iscsi-a[58197]: debug there is no tcmu-runner data available 2024-09-11T07:19:15.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:14 smithi071 ceph-mon[27013]: pgmap v112: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:19:15.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:14 smithi071 ceph-mon[27013]: osdmap e96: 8 total, 8 up, 8 in 2024-09-11T07:19:15.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:14 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/3243240492' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:19:15.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:14 smithi071 ceph-mon[37727]: pgmap v112: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:19:15.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:14 smithi071 ceph-mon[37727]: osdmap e96: 8 total, 8 up, 8 in 2024-09-11T07:19:15.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:14 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/3243240492' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:19:15.631 INFO:tasks.workunit.client.0.smithi071.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_write PASSED [ 25%] 2024-09-11T07:19:16.034 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:15 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/3243240492' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:19:16.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:15 smithi196 ceph-mon[30773]: osdmap e97: 8 total, 8 up, 8 in 2024-09-11T07:19:16.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:15 smithi196 ceph-mon[30773]: from='client.34442 v1:172.21.15.196:0/1952921461' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:19:16.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:15 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/3243240492' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:19:16.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:15 smithi071 ceph-mon[27013]: osdmap e97: 8 total, 8 up, 8 in 2024-09-11T07:19:16.090 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:15 smithi071 ceph-mon[27013]: from='client.34442 v1:172.21.15.196:0/1952921461' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:19:16.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:15 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/3243240492' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:19:16.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:15 smithi071 ceph-mon[37727]: osdmap e97: 8 total, 8 up, 8 in 2024-09-11T07:19:16.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:15 smithi071 ceph-mon[37727]: from='client.34442 v1:172.21.15.196:0/1952921461' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:19:16.589 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:19:16 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: ::ffff:172.21.15.196 - - [11/Sep/2024:07:19:16] "GET /metrics HTTP/1.1" 200 224776 "" "Prometheus/2.43.0" 2024-09-11T07:19:17.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:16 smithi196 ceph-mon[30773]: pgmap v115: 196 pgs: 196 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T07:19:17.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:16 smithi196 ceph-mon[30773]: osdmap e98: 8 total, 8 up, 8 in 2024-09-11T07:19:17.035 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 07:19:16 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-x[32609]: ::ffff:172.21.15.196 - - [11/Sep/2024:07:19:16] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:19:17.089 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:16 smithi071 ceph-mon[37727]: pgmap v115: 196 pgs: 196 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T07:19:17.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:16 smithi071 ceph-mon[37727]: osdmap e98: 8 total, 8 up, 8 in 2024-09-11T07:19:17.090 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:16 smithi071 ceph-mon[27013]: pgmap v115: 196 pgs: 196 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T07:19:17.090 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:16 smithi071 ceph-mon[27013]: osdmap e98: 8 total, 8 up, 8 in 2024-09-11T07:19:18.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:17 smithi196 ceph-mon[30773]: osdmap e99: 8 total, 8 up, 8 in 2024-09-11T07:19:18.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:17 smithi071 ceph-mon[27013]: osdmap e99: 8 total, 8 up, 8 in 2024-09-11T07:19:18.089 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:17 smithi071 ceph-mon[37727]: osdmap e99: 8 total, 8 up, 8 in 2024-09-11T07:19:19.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:18 smithi196 ceph-mon[30773]: pgmap v118: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:19:19.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:18 smithi196 ceph-mon[30773]: osdmap e100: 8 total, 8 up, 8 in 2024-09-11T07:19:19.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:18 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/2714975872' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:19:19.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:18 smithi196 ceph-mon[30773]: from='client.34793 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:19:19.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:18 smithi196 ceph-mon[30773]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T07:19:19.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:18 smithi071 ceph-mon[27013]: pgmap v118: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:19:19.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:18 smithi071 ceph-mon[27013]: osdmap e100: 8 total, 8 up, 8 in 2024-09-11T07:19:19.090 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:18 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/2714975872' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:19:19.090 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:18 smithi071 ceph-mon[27013]: from='client.34793 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:19:19.090 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:18 smithi071 ceph-mon[27013]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T07:19:19.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:18 smithi071 ceph-mon[37727]: pgmap v118: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:19:19.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:18 smithi071 ceph-mon[37727]: osdmap e100: 8 total, 8 up, 8 in 2024-09-11T07:19:19.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:18 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/2714975872' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:19:19.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:18 smithi071 ceph-mon[37727]: from='client.34793 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:19:19.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:18 smithi071 ceph-mon[37727]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T07:19:19.649 INFO:tasks.workunit.client.0.smithi071.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_write_full PASSED [ 26%] 2024-09-11T07:19:20.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:19 smithi196 ceph-mon[30773]: from='client.34793 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:19:20.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:19 smithi196 ceph-mon[30773]: osdmap e101: 8 total, 8 up, 8 in 2024-09-11T07:19:20.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:19 smithi071 ceph-mon[27013]: from='client.34793 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:19:20.090 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:19 smithi071 ceph-mon[27013]: osdmap e101: 8 total, 8 up, 8 in 2024-09-11T07:19:20.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:19 smithi071 ceph-mon[37727]: from='client.34793 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:19:20.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:19 smithi071 ceph-mon[37727]: osdmap e101: 8 total, 8 up, 8 in 2024-09-11T07:19:21.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:20 smithi196 ceph-mon[30773]: pgmap v121: 196 pgs: 196 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T07:19:21.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:20 smithi196 ceph-mon[30773]: osdmap e102: 8 total, 8 up, 8 in 2024-09-11T07:19:21.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:20 smithi071 ceph-mon[27013]: pgmap v121: 196 pgs: 196 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T07:19:21.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:20 smithi071 ceph-mon[27013]: osdmap e102: 8 total, 8 up, 8 in 2024-09-11T07:19:21.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:20 smithi071 ceph-mon[37727]: pgmap v121: 196 pgs: 196 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T07:19:21.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:20 smithi071 ceph-mon[37727]: osdmap e102: 8 total, 8 up, 8 in 2024-09-11T07:19:22.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:21 smithi196 ceph-mon[30773]: osdmap e103: 8 total, 8 up, 8 in 2024-09-11T07:19:22.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:21 smithi071 ceph-mon[27013]: osdmap e103: 8 total, 8 up, 8 in 2024-09-11T07:19:22.089 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:21 smithi071 ceph-mon[37727]: osdmap e103: 8 total, 8 up, 8 in 2024-09-11T07:19:23.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:22 smithi196 ceph-mon[30773]: pgmap v124: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:19:23.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:22 smithi196 ceph-mon[30773]: osdmap e104: 8 total, 8 up, 8 in 2024-09-11T07:19:23.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:22 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/1179247227' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:19:23.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:22 smithi196 ceph-mon[30773]: from='client.34799 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:19:23.090 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:22 smithi071 ceph-mon[27013]: pgmap v124: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:19:23.090 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:22 smithi071 ceph-mon[27013]: osdmap e104: 8 total, 8 up, 8 in 2024-09-11T07:19:23.090 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:22 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/1179247227' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:19:23.090 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:22 smithi071 ceph-mon[27013]: from='client.34799 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:19:23.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:22 smithi071 ceph-mon[37727]: pgmap v124: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:19:23.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:22 smithi071 ceph-mon[37727]: osdmap e104: 8 total, 8 up, 8 in 2024-09-11T07:19:23.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:22 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/1179247227' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:19:23.091 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:22 smithi071 ceph-mon[37727]: from='client.34799 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:19:23.702 INFO:tasks.workunit.client.0.smithi071.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_writesame PASSED [ 28%] 2024-09-11T07:19:24.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:23 smithi196 ceph-mon[30773]: from='client.34799 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:19:24.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:23 smithi196 ceph-mon[30773]: osdmap e105: 8 total, 8 up, 8 in 2024-09-11T07:19:24.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:23 smithi071 ceph-mon[27013]: from='client.34799 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:19:24.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:23 smithi071 ceph-mon[27013]: osdmap e105: 8 total, 8 up, 8 in 2024-09-11T07:19:24.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:23 smithi071 ceph-mon[37727]: from='client.34799 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:19:24.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:23 smithi071 ceph-mon[37727]: osdmap e105: 8 total, 8 up, 8 in 2024-09-11T07:19:25.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:24 smithi196 ceph-mon[30773]: pgmap v127: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:19:25.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:24 smithi196 ceph-mon[30773]: osdmap e106: 8 total, 8 up, 8 in 2024-09-11T07:19:25.035 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 11 07:19:24 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-iscsi-iscsi-a[58197]: debug there is no tcmu-runner data available 2024-09-11T07:19:25.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:24 smithi071 ceph-mon[27013]: pgmap v127: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:19:25.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:24 smithi071 ceph-mon[27013]: osdmap e106: 8 total, 8 up, 8 in 2024-09-11T07:19:25.089 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:24 smithi071 ceph-mon[37727]: pgmap v127: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:19:25.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:24 smithi071 ceph-mon[37727]: osdmap e106: 8 total, 8 up, 8 in 2024-09-11T07:19:26.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:25 smithi196 ceph-mon[30773]: from='client.34442 v1:172.21.15.196:0/1952921461' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:19:26.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:25 smithi196 ceph-mon[30773]: osdmap e107: 8 total, 8 up, 8 in 2024-09-11T07:19:26.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:25 smithi071 ceph-mon[27013]: from='client.34442 v1:172.21.15.196:0/1952921461' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:19:26.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:25 smithi071 ceph-mon[27013]: osdmap e107: 8 total, 8 up, 8 in 2024-09-11T07:19:26.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:25 smithi071 ceph-mon[37727]: from='client.34442 v1:172.21.15.196:0/1952921461' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:19:26.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:25 smithi071 ceph-mon[37727]: osdmap e107: 8 total, 8 up, 8 in 2024-09-11T07:19:26.589 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:19:26 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: ::ffff:172.21.15.196 - - [11/Sep/2024:07:19:26] "GET /metrics HTTP/1.1" 200 224776 "" "Prometheus/2.43.0" 2024-09-11T07:19:27.035 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 07:19:26 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-x[32609]: ::ffff:172.21.15.196 - - [11/Sep/2024:07:19:26] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:19:27.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:26 smithi196 ceph-mon[30773]: pgmap v130: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:19:27.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:26 smithi196 ceph-mon[30773]: osdmap e108: 8 total, 8 up, 8 in 2024-09-11T07:19:27.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:26 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/2905972032' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:19:27.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:26 smithi071 ceph-mon[27013]: pgmap v130: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:19:27.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:26 smithi071 ceph-mon[27013]: osdmap e108: 8 total, 8 up, 8 in 2024-09-11T07:19:27.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:26 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/2905972032' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:19:27.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:26 smithi071 ceph-mon[37727]: pgmap v130: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:19:27.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:26 smithi071 ceph-mon[37727]: osdmap e108: 8 total, 8 up, 8 in 2024-09-11T07:19:27.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:26 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/2905972032' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:19:27.719 INFO:tasks.workunit.client.0.smithi071.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_append PASSED [ 29%] 2024-09-11T07:19:28.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:27 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/2905972032' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:19:28.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:27 smithi196 ceph-mon[30773]: osdmap e109: 8 total, 8 up, 8 in 2024-09-11T07:19:28.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:27 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/2905972032' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:19:28.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:27 smithi071 ceph-mon[27013]: osdmap e109: 8 total, 8 up, 8 in 2024-09-11T07:19:28.089 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:27 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/2905972032' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:19:28.089 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:27 smithi071 ceph-mon[37727]: osdmap e109: 8 total, 8 up, 8 in 2024-09-11T07:19:29.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:28 smithi196 ceph-mon[30773]: pgmap v133: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:19:29.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:28 smithi196 ceph-mon[30773]: osdmap e110: 8 total, 8 up, 8 in 2024-09-11T07:19:29.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:28 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:19:29.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:28 smithi071 ceph-mon[27013]: pgmap v133: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:19:29.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:28 smithi071 ceph-mon[27013]: osdmap e110: 8 total, 8 up, 8 in 2024-09-11T07:19:29.090 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:28 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:19:29.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:28 smithi071 ceph-mon[37727]: pgmap v133: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:19:29.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:28 smithi071 ceph-mon[37727]: osdmap e110: 8 total, 8 up, 8 in 2024-09-11T07:19:29.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:28 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:19:30.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:29 smithi196 ceph-mon[30773]: osdmap e111: 8 total, 8 up, 8 in 2024-09-11T07:19:30.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:29 smithi071 ceph-mon[27013]: osdmap e111: 8 total, 8 up, 8 in 2024-09-11T07:19:30.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:29 smithi071 ceph-mon[37727]: osdmap e111: 8 total, 8 up, 8 in 2024-09-11T07:19:31.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:30 smithi196 ceph-mon[30773]: pgmap v136: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:19:31.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:30 smithi196 ceph-mon[30773]: osdmap e112: 8 total, 8 up, 8 in 2024-09-11T07:19:31.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:30 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/2802793040' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:19:31.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:30 smithi071 ceph-mon[27013]: pgmap v136: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:19:31.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:30 smithi071 ceph-mon[27013]: osdmap e112: 8 total, 8 up, 8 in 2024-09-11T07:19:31.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:30 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/2802793040' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:19:31.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:30 smithi071 ceph-mon[37727]: pgmap v136: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:19:31.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:30 smithi071 ceph-mon[37727]: osdmap e112: 8 total, 8 up, 8 in 2024-09-11T07:19:31.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:30 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/2802793040' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:19:31.755 INFO:tasks.workunit.client.0.smithi071.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_write_zeros PASSED [ 30%] 2024-09-11T07:19:32.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:31 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/2802793040' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:19:32.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:31 smithi196 ceph-mon[30773]: osdmap e113: 8 total, 8 up, 8 in 2024-09-11T07:19:32.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:31 smithi196 ceph-mon[30773]: osdmap e114: 8 total, 8 up, 8 in 2024-09-11T07:19:32.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:31 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/2802793040' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:19:32.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:31 smithi071 ceph-mon[27013]: osdmap e113: 8 total, 8 up, 8 in 2024-09-11T07:19:32.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:31 smithi071 ceph-mon[27013]: osdmap e114: 8 total, 8 up, 8 in 2024-09-11T07:19:32.089 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:31 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/2802793040' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:19:32.089 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:31 smithi071 ceph-mon[37727]: osdmap e113: 8 total, 8 up, 8 in 2024-09-11T07:19:32.089 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:31 smithi071 ceph-mon[37727]: osdmap e114: 8 total, 8 up, 8 in 2024-09-11T07:19:33.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:32 smithi196 ceph-mon[30773]: pgmap v139: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:19:33.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:32 smithi071 ceph-mon[27013]: pgmap v139: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:19:33.089 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:32 smithi071 ceph-mon[37727]: pgmap v139: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:19:34.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:33 smithi196 ceph-mon[30773]: osdmap e115: 8 total, 8 up, 8 in 2024-09-11T07:19:34.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:33 smithi071 ceph-mon[27013]: osdmap e115: 8 total, 8 up, 8 in 2024-09-11T07:19:34.089 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:33 smithi071 ceph-mon[37727]: osdmap e115: 8 total, 8 up, 8 in 2024-09-11T07:19:35.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:34 smithi196 ceph-mon[30773]: pgmap v142: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:19:35.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:34 smithi196 ceph-mon[30773]: osdmap e116: 8 total, 8 up, 8 in 2024-09-11T07:19:35.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:34 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/155870959' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:19:35.035 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 11 07:19:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-iscsi-iscsi-a[58197]: debug there is no tcmu-runner data available 2024-09-11T07:19:35.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:34 smithi071 ceph-mon[27013]: pgmap v142: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:19:35.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:34 smithi071 ceph-mon[27013]: osdmap e116: 8 total, 8 up, 8 in 2024-09-11T07:19:35.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:34 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/155870959' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:19:35.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:34 smithi071 ceph-mon[37727]: pgmap v142: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:19:35.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:34 smithi071 ceph-mon[37727]: osdmap e116: 8 total, 8 up, 8 in 2024-09-11T07:19:35.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:34 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/155870959' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:19:35.803 INFO:tasks.workunit.client.0.smithi071.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_trunc PASSED [ 31%] 2024-09-11T07:19:36.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:35 smithi071 ceph-mon[27013]: from='client.34442 v1:172.21.15.196:0/1952921461' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:19:36.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:35 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/155870959' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:19:36.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:35 smithi071 ceph-mon[27013]: osdmap e117: 8 total, 8 up, 8 in 2024-09-11T07:19:36.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:35 smithi071 ceph-mon[37727]: from='client.34442 v1:172.21.15.196:0/1952921461' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:19:36.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:35 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/155870959' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:19:36.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:35 smithi071 ceph-mon[37727]: osdmap e117: 8 total, 8 up, 8 in 2024-09-11T07:19:36.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:35 smithi196 ceph-mon[30773]: from='client.34442 v1:172.21.15.196:0/1952921461' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:19:36.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:35 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/155870959' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:19:36.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:35 smithi196 ceph-mon[30773]: osdmap e117: 8 total, 8 up, 8 in 2024-09-11T07:19:36.589 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:19:36 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: ::ffff:172.21.15.196 - - [11/Sep/2024:07:19:36] "GET /metrics HTTP/1.1" 200 222482 "" "Prometheus/2.43.0" 2024-09-11T07:19:37.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:36 smithi196 ceph-mon[30773]: pgmap v145: 196 pgs: 196 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-11T07:19:37.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:36 smithi196 ceph-mon[30773]: osdmap e118: 8 total, 8 up, 8 in 2024-09-11T07:19:37.035 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 07:19:36 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-x[32609]: ::ffff:172.21.15.196 - - [11/Sep/2024:07:19:36] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:19:37.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:36 smithi071 ceph-mon[27013]: pgmap v145: 196 pgs: 196 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-11T07:19:37.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:36 smithi071 ceph-mon[27013]: osdmap e118: 8 total, 8 up, 8 in 2024-09-11T07:19:37.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:36 smithi071 ceph-mon[37727]: pgmap v145: 196 pgs: 196 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-11T07:19:37.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:36 smithi071 ceph-mon[37727]: osdmap e118: 8 total, 8 up, 8 in 2024-09-11T07:19:38.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:37 smithi071 ceph-mon[27013]: osdmap e119: 8 total, 8 up, 8 in 2024-09-11T07:19:38.089 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:37 smithi071 ceph-mon[37727]: osdmap e119: 8 total, 8 up, 8 in 2024-09-11T07:19:38.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:37 smithi196 ceph-mon[30773]: osdmap e119: 8 total, 8 up, 8 in 2024-09-11T07:19:39.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:38 smithi071 ceph-mon[27013]: pgmap v148: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:19:39.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:38 smithi071 ceph-mon[27013]: osdmap e120: 8 total, 8 up, 8 in 2024-09-11T07:19:39.090 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:38 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/2972517778' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:19:39.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:38 smithi071 ceph-mon[37727]: pgmap v148: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:19:39.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:38 smithi071 ceph-mon[37727]: osdmap e120: 8 total, 8 up, 8 in 2024-09-11T07:19:39.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:38 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/2972517778' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:19:39.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:38 smithi196 ceph-mon[30773]: pgmap v148: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:19:39.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:38 smithi196 ceph-mon[30773]: osdmap e120: 8 total, 8 up, 8 in 2024-09-11T07:19:39.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:38 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/2972517778' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:19:39.832 INFO:tasks.workunit.client.0.smithi071.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_cmpext PASSED [ 32%] 2024-09-11T07:19:40.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:39 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/2972517778' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:19:40.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:39 smithi071 ceph-mon[27013]: osdmap e121: 8 total, 8 up, 8 in 2024-09-11T07:19:40.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:39 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/2972517778' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:19:40.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:39 smithi071 ceph-mon[37727]: osdmap e121: 8 total, 8 up, 8 in 2024-09-11T07:19:40.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:39 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/2972517778' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:19:40.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:39 smithi196 ceph-mon[30773]: osdmap e121: 8 total, 8 up, 8 in 2024-09-11T07:19:41.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:40 smithi196 ceph-mon[30773]: pgmap v151: 196 pgs: 196 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-09-11T07:19:41.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:40 smithi196 ceph-mon[30773]: osdmap e122: 8 total, 8 up, 8 in 2024-09-11T07:19:41.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:40 smithi071 ceph-mon[27013]: pgmap v151: 196 pgs: 196 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-09-11T07:19:41.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:40 smithi071 ceph-mon[27013]: osdmap e122: 8 total, 8 up, 8 in 2024-09-11T07:19:41.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:40 smithi071 ceph-mon[37727]: pgmap v151: 196 pgs: 196 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-09-11T07:19:41.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:40 smithi071 ceph-mon[37727]: osdmap e122: 8 total, 8 up, 8 in 2024-09-11T07:19:42.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:41 smithi196 ceph-mon[30773]: osdmap e123: 8 total, 8 up, 8 in 2024-09-11T07:19:42.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:41 smithi196 ceph-mon[30773]: osdmap e124: 8 total, 8 up, 8 in 2024-09-11T07:19:42.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:41 smithi071 ceph-mon[27013]: osdmap e123: 8 total, 8 up, 8 in 2024-09-11T07:19:42.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:41 smithi071 ceph-mon[27013]: osdmap e124: 8 total, 8 up, 8 in 2024-09-11T07:19:42.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:41 smithi071 ceph-mon[37727]: osdmap e123: 8 total, 8 up, 8 in 2024-09-11T07:19:42.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:41 smithi071 ceph-mon[37727]: osdmap e124: 8 total, 8 up, 8 in 2024-09-11T07:19:43.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:42 smithi196 ceph-mon[30773]: pgmap v154: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:19:43.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:42 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/772633583' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:19:43.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:42 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/772633583' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:19:43.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:42 smithi196 ceph-mon[30773]: osdmap e125: 8 total, 8 up, 8 in 2024-09-11T07:19:43.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:42 smithi071 ceph-mon[27013]: pgmap v154: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:19:43.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:42 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/772633583' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:19:43.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:42 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/772633583' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:19:43.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:42 smithi071 ceph-mon[27013]: osdmap e125: 8 total, 8 up, 8 in 2024-09-11T07:19:43.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:42 smithi071 ceph-mon[37727]: pgmap v154: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:19:43.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:42 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/772633583' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:19:43.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:42 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/772633583' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:19:43.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:42 smithi071 ceph-mon[37727]: osdmap e125: 8 total, 8 up, 8 in 2024-09-11T07:19:43.859 INFO:tasks.workunit.client.0.smithi071.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_list_objects_empty PASSED [ 33%] 2024-09-11T07:19:44.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:43 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:19:44.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:43 smithi196 ceph-mon[30773]: osdmap e126: 8 total, 8 up, 8 in 2024-09-11T07:19:44.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:43 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:19:44.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:43 smithi071 ceph-mon[27013]: osdmap e126: 8 total, 8 up, 8 in 2024-09-11T07:19:44.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:43 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:19:44.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:43 smithi071 ceph-mon[37727]: osdmap e126: 8 total, 8 up, 8 in 2024-09-11T07:19:45.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:44 smithi196 ceph-mon[30773]: pgmap v157: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:19:45.035 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 11 07:19:44 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-iscsi-iscsi-a[58197]: debug there is no tcmu-runner data available 2024-09-11T07:19:45.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:44 smithi071 ceph-mon[27013]: pgmap v157: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:19:45.339 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:44 smithi071 ceph-mon[37727]: pgmap v157: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:19:46.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:45 smithi071 ceph-mon[27013]: from='client.34442 v1:172.21.15.196:0/1952921461' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:19:46.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:45 smithi071 ceph-mon[27013]: osdmap e127: 8 total, 8 up, 8 in 2024-09-11T07:19:46.161 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:45 smithi071 ceph-mon[37727]: from='client.34442 v1:172.21.15.196:0/1952921461' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:19:46.161 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:45 smithi071 ceph-mon[37727]: osdmap e127: 8 total, 8 up, 8 in 2024-09-11T07:19:46.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:45 smithi196 ceph-mon[30773]: from='client.34442 v1:172.21.15.196:0/1952921461' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:19:46.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:45 smithi196 ceph-mon[30773]: osdmap e127: 8 total, 8 up, 8 in 2024-09-11T07:19:46.589 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:19:46 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: ::ffff:172.21.15.196 - - [11/Sep/2024:07:19:46] "GET /metrics HTTP/1.1" 200 224861 "" "Prometheus/2.43.0" 2024-09-11T07:19:46.879 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 07:19:46 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-x[32609]: ::ffff:172.21.15.196 - - [11/Sep/2024:07:19:46] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:19:47.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:46 smithi196 ceph-mon[30773]: pgmap v160: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:19:47.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:46 smithi196 ceph-mon[30773]: osdmap e128: 8 total, 8 up, 8 in 2024-09-11T07:19:47.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:46 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/3431697024' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:19:47.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:46 smithi196 ceph-mon[30773]: from='client.34835 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:19:47.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:46 smithi071 ceph-mon[27013]: pgmap v160: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:19:47.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:46 smithi071 ceph-mon[27013]: osdmap e128: 8 total, 8 up, 8 in 2024-09-11T07:19:47.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:46 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/3431697024' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:19:47.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:46 smithi071 ceph-mon[27013]: from='client.34835 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:19:47.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:46 smithi071 ceph-mon[37727]: pgmap v160: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:19:47.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:46 smithi071 ceph-mon[37727]: osdmap e128: 8 total, 8 up, 8 in 2024-09-11T07:19:47.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:46 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/3431697024' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:19:47.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:46 smithi071 ceph-mon[37727]: from='client.34835 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:19:47.901 INFO:tasks.workunit.client.0.smithi071.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_list_objects PASSED [ 34%] 2024-09-11T07:19:48.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:47 smithi196 ceph-mon[30773]: from='client.34835 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:19:48.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:47 smithi196 ceph-mon[30773]: osdmap e129: 8 total, 8 up, 8 in 2024-09-11T07:19:48.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:47 smithi071 ceph-mon[27013]: from='client.34835 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:19:48.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:47 smithi071 ceph-mon[27013]: osdmap e129: 8 total, 8 up, 8 in 2024-09-11T07:19:48.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:47 smithi071 ceph-mon[37727]: from='client.34835 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:19:48.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:47 smithi071 ceph-mon[37727]: osdmap e129: 8 total, 8 up, 8 in 2024-09-11T07:19:49.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:48 smithi196 ceph-mon[30773]: pgmap v163: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:19:49.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:48 smithi196 ceph-mon[30773]: osdmap e130: 8 total, 8 up, 8 in 2024-09-11T07:19:49.339 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:48 smithi071 ceph-mon[37727]: pgmap v163: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:19:49.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:48 smithi071 ceph-mon[37727]: osdmap e130: 8 total, 8 up, 8 in 2024-09-11T07:19:49.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:48 smithi071 ceph-mon[27013]: pgmap v163: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:19:49.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:48 smithi071 ceph-mon[27013]: osdmap e130: 8 total, 8 up, 8 in 2024-09-11T07:19:50.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:49 smithi196 ceph-mon[30773]: osdmap e131: 8 total, 8 up, 8 in 2024-09-11T07:19:50.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:49 smithi196 ceph-mon[30773]: osdmap e132: 8 total, 8 up, 8 in 2024-09-11T07:19:50.339 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:49 smithi071 ceph-mon[37727]: osdmap e131: 8 total, 8 up, 8 in 2024-09-11T07:19:50.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:49 smithi071 ceph-mon[37727]: osdmap e132: 8 total, 8 up, 8 in 2024-09-11T07:19:50.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:49 smithi071 ceph-mon[27013]: osdmap e131: 8 total, 8 up, 8 in 2024-09-11T07:19:50.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:49 smithi071 ceph-mon[27013]: osdmap e132: 8 total, 8 up, 8 in 2024-09-11T07:19:51.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:50 smithi196 ceph-mon[30773]: pgmap v166: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:19:51.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:50 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/747100215' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:19:51.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:50 smithi196 ceph-mon[30773]: from='client.34841 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:19:51.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:50 smithi196 ceph-mon[30773]: from='client.34841 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:19:51.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:50 smithi196 ceph-mon[30773]: osdmap e133: 8 total, 8 up, 8 in 2024-09-11T07:19:51.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:50 smithi071 ceph-mon[27013]: pgmap v166: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:19:51.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:50 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/747100215' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:19:51.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:50 smithi071 ceph-mon[27013]: from='client.34841 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:19:51.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:50 smithi071 ceph-mon[27013]: from='client.34841 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:19:51.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:50 smithi071 ceph-mon[27013]: osdmap e133: 8 total, 8 up, 8 in 2024-09-11T07:19:51.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:50 smithi071 ceph-mon[37727]: pgmap v166: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:19:51.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:50 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/747100215' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:19:51.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:50 smithi071 ceph-mon[37727]: from='client.34841 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:19:51.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:50 smithi071 ceph-mon[37727]: from='client.34841 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:19:51.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:50 smithi071 ceph-mon[37727]: osdmap e133: 8 total, 8 up, 8 in 2024-09-11T07:19:51.905 INFO:tasks.workunit.client.0.smithi071.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_list_ns_objects PASSED [ 35%] 2024-09-11T07:19:53.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:52 smithi196 ceph-mon[30773]: pgmap v169: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:19:53.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:52 smithi196 ceph-mon[30773]: osdmap e134: 8 total, 8 up, 8 in 2024-09-11T07:19:53.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:52 smithi071 ceph-mon[27013]: pgmap v169: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:19:53.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:52 smithi071 ceph-mon[27013]: osdmap e134: 8 total, 8 up, 8 in 2024-09-11T07:19:53.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:52 smithi071 ceph-mon[37727]: pgmap v169: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:19:53.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:52 smithi071 ceph-mon[37727]: osdmap e134: 8 total, 8 up, 8 in 2024-09-11T07:19:54.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:53 smithi196 ceph-mon[30773]: osdmap e135: 8 total, 8 up, 8 in 2024-09-11T07:19:54.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:53 smithi071 ceph-mon[27013]: osdmap e135: 8 total, 8 up, 8 in 2024-09-11T07:19:54.339 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:53 smithi071 ceph-mon[37727]: osdmap e135: 8 total, 8 up, 8 in 2024-09-11T07:19:55.036 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:54 smithi196 ceph-mon[30773]: pgmap v172: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:19:55.036 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:54 smithi196 ceph-mon[30773]: osdmap e136: 8 total, 8 up, 8 in 2024-09-11T07:19:55.037 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:54 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/2334627040' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:19:55.037 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 11 07:19:54 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-iscsi-iscsi-a[58197]: debug there is no tcmu-runner data available 2024-09-11T07:19:55.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:54 smithi071 ceph-mon[27013]: pgmap v172: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:19:55.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:54 smithi071 ceph-mon[27013]: osdmap e136: 8 total, 8 up, 8 in 2024-09-11T07:19:55.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:54 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/2334627040' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:19:55.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:54 smithi071 ceph-mon[37727]: pgmap v172: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:19:55.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:54 smithi071 ceph-mon[37727]: osdmap e136: 8 total, 8 up, 8 in 2024-09-11T07:19:55.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:54 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/2334627040' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:19:55.941 INFO:tasks.workunit.client.0.smithi071.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_xattrs PASSED [ 37%] 2024-09-11T07:19:56.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:55 smithi196 ceph-mon[30773]: from='client.34442 v1:172.21.15.196:0/1952921461' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:19:56.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:55 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/2334627040' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:19:56.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:55 smithi196 ceph-mon[30773]: osdmap e137: 8 total, 8 up, 8 in 2024-09-11T07:19:56.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:55 smithi196 ceph-mon[30773]: osdmap e138: 8 total, 8 up, 8 in 2024-09-11T07:19:56.339 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:19:56 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: ::ffff:172.21.15.196 - - [11/Sep/2024:07:19:56] "GET /metrics HTTP/1.1" 200 224861 "" "Prometheus/2.43.0" 2024-09-11T07:19:56.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:55 smithi071 ceph-mon[27013]: from='client.34442 v1:172.21.15.196:0/1952921461' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:19:56.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:55 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/2334627040' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:19:56.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:55 smithi071 ceph-mon[27013]: osdmap e137: 8 total, 8 up, 8 in 2024-09-11T07:19:56.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:55 smithi071 ceph-mon[27013]: osdmap e138: 8 total, 8 up, 8 in 2024-09-11T07:19:56.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:55 smithi071 ceph-mon[37727]: from='client.34442 v1:172.21.15.196:0/1952921461' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:19:56.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:55 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/2334627040' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:19:56.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:55 smithi071 ceph-mon[37727]: osdmap e137: 8 total, 8 up, 8 in 2024-09-11T07:19:56.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:55 smithi071 ceph-mon[37727]: osdmap e138: 8 total, 8 up, 8 in 2024-09-11T07:19:56.948 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 07:19:56 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-x[32609]: ::ffff:172.21.15.196 - - [11/Sep/2024:07:19:56] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:19:57.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:56 smithi196 ceph-mon[30773]: pgmap v175: 196 pgs: 196 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 2.7 KiB/s rd, 0 B/s wr, 4 op/s 2024-09-11T07:19:57.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:56 smithi071 ceph-mon[27013]: pgmap v175: 196 pgs: 196 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 2.7 KiB/s rd, 0 B/s wr, 4 op/s 2024-09-11T07:19:57.339 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:56 smithi071 ceph-mon[37727]: pgmap v175: 196 pgs: 196 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 2.7 KiB/s rd, 0 B/s wr, 4 op/s 2024-09-11T07:19:58.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:57 smithi196 ceph-mon[30773]: osdmap e139: 8 total, 8 up, 8 in 2024-09-11T07:19:58.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:57 smithi071 ceph-mon[27013]: osdmap e139: 8 total, 8 up, 8 in 2024-09-11T07:19:58.339 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:57 smithi071 ceph-mon[37727]: osdmap e139: 8 total, 8 up, 8 in 2024-09-11T07:19:59.241 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:58 smithi071 ceph-mon[27013]: pgmap v178: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:19:59.241 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:58 smithi071 ceph-mon[27013]: osdmap e140: 8 total, 8 up, 8 in 2024-09-11T07:19:59.241 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:58 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/75612168' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:19:59.241 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:58 smithi071 ceph-mon[27013]: from='client.34853 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:19:59.241 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:58 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:19:59.241 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:58 smithi071 ceph-mon[37727]: pgmap v178: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:19:59.241 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:58 smithi071 ceph-mon[37727]: osdmap e140: 8 total, 8 up, 8 in 2024-09-11T07:19:59.242 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:58 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/75612168' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:19:59.242 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:58 smithi071 ceph-mon[37727]: from='client.34853 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:19:59.242 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:58 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:19:59.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:58 smithi196 ceph-mon[30773]: pgmap v178: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:19:59.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:58 smithi196 ceph-mon[30773]: osdmap e140: 8 total, 8 up, 8 in 2024-09-11T07:19:59.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:58 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/75612168' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:19:59.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:58 smithi196 ceph-mon[30773]: from='client.34853 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:19:59.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:58 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:19:59.984 INFO:tasks.workunit.client.0.smithi071.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_obj_xattrs PASSED [ 38%] 2024-09-11T07:20:00.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:59 smithi196 ceph-mon[30773]: from='client.34853 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:20:00.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:19:59 smithi196 ceph-mon[30773]: osdmap e141: 8 total, 8 up, 8 in 2024-09-11T07:20:00.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:59 smithi071 ceph-mon[27013]: from='client.34853 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:20:00.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:19:59 smithi071 ceph-mon[27013]: osdmap e141: 8 total, 8 up, 8 in 2024-09-11T07:20:00.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:59 smithi071 ceph-mon[37727]: from='client.34853 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:20:00.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:19:59 smithi071 ceph-mon[37727]: osdmap e141: 8 total, 8 up, 8 in 2024-09-11T07:20:01.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:00 smithi196 ceph-mon[30773]: pgmap v181: 196 pgs: 196 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:20:01.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:00 smithi196 ceph-mon[30773]: osdmap e142: 8 total, 8 up, 8 in 2024-09-11T07:20:01.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:00 smithi196 ceph-mon[30773]: Health detail: HEALTH_WARN 2 pool(s) do not have an application enabled 2024-09-11T07:20:01.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:00 smithi196 ceph-mon[30773]: [WRN] POOL_APP_NOT_ENABLED: 2 pool(s) do not have an application enabled 2024-09-11T07:20:01.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:00 smithi196 ceph-mon[30773]: application not enabled on pool 'rbd' 2024-09-11T07:20:01.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:00 smithi196 ceph-mon[30773]: application not enabled on pool 'test_pool' 2024-09-11T07:20:01.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:00 smithi196 ceph-mon[30773]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-09-11T07:20:01.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:00 smithi071 ceph-mon[27013]: pgmap v181: 196 pgs: 196 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:20:01.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:00 smithi071 ceph-mon[27013]: osdmap e142: 8 total, 8 up, 8 in 2024-09-11T07:20:01.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:00 smithi071 ceph-mon[27013]: Health detail: HEALTH_WARN 2 pool(s) do not have an application enabled 2024-09-11T07:20:01.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:00 smithi071 ceph-mon[27013]: [WRN] POOL_APP_NOT_ENABLED: 2 pool(s) do not have an application enabled 2024-09-11T07:20:01.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:00 smithi071 ceph-mon[27013]: application not enabled on pool 'rbd' 2024-09-11T07:20:01.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:00 smithi071 ceph-mon[27013]: application not enabled on pool 'test_pool' 2024-09-11T07:20:01.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:00 smithi071 ceph-mon[27013]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-09-11T07:20:01.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:00 smithi071 ceph-mon[37727]: pgmap v181: 196 pgs: 196 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:20:01.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:00 smithi071 ceph-mon[37727]: osdmap e142: 8 total, 8 up, 8 in 2024-09-11T07:20:01.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:00 smithi071 ceph-mon[37727]: Health detail: HEALTH_WARN 2 pool(s) do not have an application enabled 2024-09-11T07:20:01.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:00 smithi071 ceph-mon[37727]: [WRN] POOL_APP_NOT_ENABLED: 2 pool(s) do not have an application enabled 2024-09-11T07:20:01.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:00 smithi071 ceph-mon[37727]: application not enabled on pool 'rbd' 2024-09-11T07:20:01.341 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:00 smithi071 ceph-mon[37727]: application not enabled on pool 'test_pool' 2024-09-11T07:20:01.341 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:00 smithi071 ceph-mon[37727]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-09-11T07:20:02.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:01 smithi196 ceph-mon[30773]: osdmap e143: 8 total, 8 up, 8 in 2024-09-11T07:20:02.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:01 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/1475293231' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:20:02.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:01 smithi196 ceph-mon[30773]: from='client.34859 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:20:02.341 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:01 smithi071 ceph-mon[37727]: osdmap e143: 8 total, 8 up, 8 in 2024-09-11T07:20:02.341 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:01 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/1475293231' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:20:02.341 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:01 smithi071 ceph-mon[37727]: from='client.34859 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:20:02.341 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:01 smithi071 ceph-mon[27013]: osdmap e143: 8 total, 8 up, 8 in 2024-09-11T07:20:02.341 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:01 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/1475293231' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:20:02.341 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:01 smithi071 ceph-mon[27013]: from='client.34859 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:20:03.005 INFO:tasks.workunit.client.0.smithi071.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_get_pool_id PASSED [ 39%] 2024-09-11T07:20:03.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:03 smithi196 ceph-mon[30773]: pgmap v184: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:20:03.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:03 smithi196 ceph-mon[30773]: from='client.34859 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:20:03.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:03 smithi196 ceph-mon[30773]: osdmap e144: 8 total, 8 up, 8 in 2024-09-11T07:20:03.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:03 smithi071 ceph-mon[27013]: pgmap v184: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:20:03.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:03 smithi071 ceph-mon[27013]: from='client.34859 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:20:03.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:03 smithi071 ceph-mon[27013]: osdmap e144: 8 total, 8 up, 8 in 2024-09-11T07:20:03.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:03 smithi071 ceph-mon[37727]: pgmap v184: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:20:03.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:03 smithi071 ceph-mon[37727]: from='client.34859 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:20:03.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:03 smithi071 ceph-mon[37727]: osdmap e144: 8 total, 8 up, 8 in 2024-09-11T07:20:04.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:04 smithi196 ceph-mon[30773]: osdmap e145: 8 total, 8 up, 8 in 2024-09-11T07:20:04.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:04 smithi071 ceph-mon[27013]: osdmap e145: 8 total, 8 up, 8 in 2024-09-11T07:20:04.339 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:04 smithi071 ceph-mon[37727]: osdmap e145: 8 total, 8 up, 8 in 2024-09-11T07:20:05.013 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 11 07:20:04 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-iscsi-iscsi-a[58197]: debug there is no tcmu-runner data available 2024-09-11T07:20:05.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:05 smithi196 ceph-mon[30773]: pgmap v187: 164 pgs: 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:20:05.286 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:05 smithi196 ceph-mon[30773]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T07:20:05.286 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:05 smithi196 ceph-mon[30773]: osdmap e146: 8 total, 8 up, 8 in 2024-09-11T07:20:05.286 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:05 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/836811736' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:20:05.286 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:05 smithi196 ceph-mon[30773]: from='client.34865 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:20:05.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:05 smithi071 ceph-mon[27013]: pgmap v187: 164 pgs: 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:20:05.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:05 smithi071 ceph-mon[27013]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T07:20:05.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:05 smithi071 ceph-mon[27013]: osdmap e146: 8 total, 8 up, 8 in 2024-09-11T07:20:05.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:05 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/836811736' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:20:05.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:05 smithi071 ceph-mon[27013]: from='client.34865 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:20:05.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:05 smithi071 ceph-mon[37727]: pgmap v187: 164 pgs: 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:20:05.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:05 smithi071 ceph-mon[37727]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T07:20:05.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:05 smithi071 ceph-mon[37727]: osdmap e146: 8 total, 8 up, 8 in 2024-09-11T07:20:05.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:05 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/836811736' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:20:05.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:05 smithi071 ceph-mon[37727]: from='client.34865 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:20:06.033 INFO:tasks.workunit.client.0.smithi071.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_get_pool_name PASSED [ 40%] 2024-09-11T07:20:06.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:06 smithi196 ceph-mon[30773]: from='client.34442 v1:172.21.15.196:0/1952921461' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:20:06.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:06 smithi196 ceph-mon[30773]: from='client.34865 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:20:06.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:06 smithi196 ceph-mon[30773]: osdmap e147: 8 total, 8 up, 8 in 2024-09-11T07:20:06.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:06 smithi071 ceph-mon[27013]: from='client.34442 v1:172.21.15.196:0/1952921461' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:20:06.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:06 smithi071 ceph-mon[27013]: from='client.34865 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:20:06.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:06 smithi071 ceph-mon[27013]: osdmap e147: 8 total, 8 up, 8 in 2024-09-11T07:20:06.340 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:20:06 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: ::ffff:172.21.15.196 - - [11/Sep/2024:07:20:06] "GET /metrics HTTP/1.1" 200 224871 "" "Prometheus/2.43.0" 2024-09-11T07:20:06.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:06 smithi071 ceph-mon[37727]: from='client.34442 v1:172.21.15.196:0/1952921461' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:20:06.341 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:06 smithi071 ceph-mon[37727]: from='client.34865 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:20:06.341 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:06 smithi071 ceph-mon[37727]: osdmap e147: 8 total, 8 up, 8 in 2024-09-11T07:20:07.026 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 07:20:06 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-x[32609]: ::ffff:172.21.15.196 - - [11/Sep/2024:07:20:06] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:20:07.284 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:07 smithi196 ceph-mon[30773]: pgmap v190: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:20:07.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:07 smithi196 ceph-mon[30773]: osdmap e148: 8 total, 8 up, 8 in 2024-09-11T07:20:07.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:07 smithi071 ceph-mon[27013]: pgmap v190: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:20:07.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:07 smithi071 ceph-mon[27013]: osdmap e148: 8 total, 8 up, 8 in 2024-09-11T07:20:07.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:07 smithi071 ceph-mon[37727]: pgmap v190: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:20:07.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:07 smithi071 ceph-mon[37727]: osdmap e148: 8 total, 8 up, 8 in 2024-09-11T07:20:08.301 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:08 smithi196 ceph-mon[30773]: osdmap e149: 8 total, 8 up, 8 in 2024-09-11T07:20:08.301 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:08 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:20:08.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:08 smithi071 ceph-mon[27013]: osdmap e149: 8 total, 8 up, 8 in 2024-09-11T07:20:08.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:08 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:20:08.339 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:08 smithi071 ceph-mon[37727]: osdmap e149: 8 total, 8 up, 8 in 2024-09-11T07:20:08.339 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:08 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:20:09.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:09 smithi071 ceph-mon[27013]: pgmap v193: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:20:09.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:09 smithi071 ceph-mon[27013]: osdmap e150: 8 total, 8 up, 8 in 2024-09-11T07:20:09.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:09 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:20:09.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:09 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:20:09.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:09 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:20:09.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:09 smithi071 ceph-mon[37727]: pgmap v193: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:20:09.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:09 smithi071 ceph-mon[37727]: osdmap e150: 8 total, 8 up, 8 in 2024-09-11T07:20:09.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:09 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:20:09.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:09 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:20:09.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:09 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:20:09.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:09 smithi196 ceph-mon[30773]: pgmap v193: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:20:09.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:09 smithi196 ceph-mon[30773]: osdmap e150: 8 total, 8 up, 8 in 2024-09-11T07:20:09.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:09 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:20:09.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:09 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:20:09.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:09 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:20:10.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:10 smithi071 ceph-mon[27013]: osdmap e151: 8 total, 8 up, 8 in 2024-09-11T07:20:10.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:10 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/1096508658' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:20:10.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:10 smithi071 ceph-mon[27013]: from='client.34871 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:20:10.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:10 smithi071 ceph-mon[37727]: osdmap e151: 8 total, 8 up, 8 in 2024-09-11T07:20:10.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:10 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/1096508658' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:20:10.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:10 smithi071 ceph-mon[37727]: from='client.34871 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:20:10.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:10 smithi196 ceph-mon[30773]: osdmap e151: 8 total, 8 up, 8 in 2024-09-11T07:20:10.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:10 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/1096508658' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:20:10.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:10 smithi196 ceph-mon[30773]: from='client.34871 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:20:11.086 INFO:tasks.workunit.client.0.smithi071.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_create_snap PASSED [ 41%] 2024-09-11T07:20:11.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:11 smithi071 ceph-mon[27013]: pgmap v196: 196 pgs: 196 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:20:11.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:11 smithi071 ceph-mon[27013]: from='client.34871 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:20:11.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:11 smithi071 ceph-mon[27013]: osdmap e152: 8 total, 8 up, 8 in 2024-09-11T07:20:11.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:11 smithi071 ceph-mon[37727]: pgmap v196: 196 pgs: 196 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:20:11.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:11 smithi071 ceph-mon[37727]: from='client.34871 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:20:11.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:11 smithi071 ceph-mon[37727]: osdmap e152: 8 total, 8 up, 8 in 2024-09-11T07:20:11.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:11 smithi196 ceph-mon[30773]: pgmap v196: 196 pgs: 196 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:20:11.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:11 smithi196 ceph-mon[30773]: from='client.34871 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:20:11.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:11 smithi196 ceph-mon[30773]: osdmap e152: 8 total, 8 up, 8 in 2024-09-11T07:20:12.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:12 smithi071 ceph-mon[27013]: osdmap e153: 8 total, 8 up, 8 in 2024-09-11T07:20:12.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:12 smithi071 ceph-mon[37727]: osdmap e153: 8 total, 8 up, 8 in 2024-09-11T07:20:12.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:12 smithi196 ceph-mon[30773]: osdmap e153: 8 total, 8 up, 8 in 2024-09-11T07:20:13.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:13 smithi071 ceph-mon[27013]: pgmap v199: 164 pgs: 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:20:13.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:13 smithi071 ceph-mon[27013]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T07:20:13.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:13 smithi071 ceph-mon[27013]: osdmap e154: 8 total, 8 up, 8 in 2024-09-11T07:20:13.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:13 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/1933827797' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:20:13.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:13 smithi071 ceph-mon[37727]: pgmap v199: 164 pgs: 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:20:13.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:13 smithi071 ceph-mon[37727]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T07:20:13.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:13 smithi071 ceph-mon[37727]: osdmap e154: 8 total, 8 up, 8 in 2024-09-11T07:20:13.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:13 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/1933827797' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:20:13.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:13 smithi196 ceph-mon[30773]: pgmap v199: 164 pgs: 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:20:13.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:13 smithi196 ceph-mon[30773]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T07:20:13.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:13 smithi196 ceph-mon[30773]: osdmap e154: 8 total, 8 up, 8 in 2024-09-11T07:20:13.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:13 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/1933827797' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:20:14.102 INFO:tasks.workunit.client.0.smithi071.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_list_snaps_empty PASSED [ 42%] 2024-09-11T07:20:14.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:14 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/1933827797' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:20:14.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:14 smithi196 ceph-mon[30773]: osdmap e155: 8 total, 8 up, 8 in 2024-09-11T07:20:14.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:14 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:20:14.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:14 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/1933827797' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:20:14.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:14 smithi071 ceph-mon[27013]: osdmap e155: 8 total, 8 up, 8 in 2024-09-11T07:20:14.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:14 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:20:14.589 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:14 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/1933827797' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:20:14.589 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:14 smithi071 ceph-mon[37727]: osdmap e155: 8 total, 8 up, 8 in 2024-09-11T07:20:14.589 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:14 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:20:15.034 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 11 07:20:14 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-iscsi-iscsi-a[58197]: debug there is no tcmu-runner data available 2024-09-11T07:20:15.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:15 smithi196 ceph-mon[30773]: pgmap v202: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:20:15.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:15 smithi196 ceph-mon[30773]: osdmap e156: 8 total, 8 up, 8 in 2024-09-11T07:20:15.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:15 smithi071 ceph-mon[27013]: pgmap v202: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:20:15.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:15 smithi071 ceph-mon[27013]: osdmap e156: 8 total, 8 up, 8 in 2024-09-11T07:20:15.589 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:15 smithi071 ceph-mon[37727]: pgmap v202: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:20:15.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:15 smithi071 ceph-mon[37727]: osdmap e156: 8 total, 8 up, 8 in 2024-09-11T07:20:16.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:16 smithi196 ceph-mon[30773]: from='client.34442 v1:172.21.15.196:0/1952921461' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:20:16.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:16 smithi196 ceph-mon[30773]: osdmap e157: 8 total, 8 up, 8 in 2024-09-11T07:20:16.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:16 smithi071 ceph-mon[27013]: from='client.34442 v1:172.21.15.196:0/1952921461' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:20:16.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:16 smithi071 ceph-mon[27013]: osdmap e157: 8 total, 8 up, 8 in 2024-09-11T07:20:16.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:16 smithi071 ceph-mon[37727]: from='client.34442 v1:172.21.15.196:0/1952921461' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:20:16.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:16 smithi071 ceph-mon[37727]: osdmap e157: 8 total, 8 up, 8 in 2024-09-11T07:20:16.590 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:20:16 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: ::ffff:172.21.15.196 - - [11/Sep/2024:07:20:16] "GET /metrics HTTP/1.1" 200 224919 "" "Prometheus/2.43.0" 2024-09-11T07:20:17.035 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 07:20:16 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-x[32609]: ::ffff:172.21.15.196 - - [11/Sep/2024:07:20:16] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:20:17.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:17 smithi196 ceph-mon[30773]: pgmap v205: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:20:17.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:17 smithi196 ceph-mon[30773]: osdmap e158: 8 total, 8 up, 8 in 2024-09-11T07:20:17.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:17 smithi071 ceph-mon[27013]: pgmap v205: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:20:17.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:17 smithi071 ceph-mon[27013]: osdmap e158: 8 total, 8 up, 8 in 2024-09-11T07:20:17.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:17 smithi071 ceph-mon[37727]: pgmap v205: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:20:17.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:17 smithi071 ceph-mon[37727]: osdmap e158: 8 total, 8 up, 8 in 2024-09-11T07:20:18.372 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:18 smithi196 ceph-mon[30773]: osdmap e159: 8 total, 8 up, 8 in 2024-09-11T07:20:18.372 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:18 smithi196 ceph-mon[30773]: pgmap v208: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:20:18.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:18 smithi071 ceph-mon[27013]: osdmap e159: 8 total, 8 up, 8 in 2024-09-11T07:20:18.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:18 smithi071 ceph-mon[27013]: pgmap v208: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:20:18.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:18 smithi071 ceph-mon[37727]: osdmap e159: 8 total, 8 up, 8 in 2024-09-11T07:20:18.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:18 smithi071 ceph-mon[37727]: pgmap v208: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:20:19.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:19 smithi196 ceph-mon[30773]: osdmap e160: 8 total, 8 up, 8 in 2024-09-11T07:20:19.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:19 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/3266778823' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:20:19.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:19 smithi196 ceph-mon[30773]: from='client.34883 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:20:19.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:19 smithi196 ceph-mon[30773]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T07:20:19.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:19 smithi071 ceph-mon[27013]: osdmap e160: 8 total, 8 up, 8 in 2024-09-11T07:20:19.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:19 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/3266778823' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:20:19.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:19 smithi071 ceph-mon[27013]: from='client.34883 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:20:19.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:19 smithi071 ceph-mon[27013]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T07:20:19.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:19 smithi071 ceph-mon[37727]: osdmap e160: 8 total, 8 up, 8 in 2024-09-11T07:20:19.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:19 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/3266778823' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:20:19.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:19 smithi071 ceph-mon[37727]: from='client.34883 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:20:19.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:19 smithi071 ceph-mon[37727]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T07:20:20.130 INFO:tasks.workunit.client.0.smithi071.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_list_snaps PASSED [ 43%] 2024-09-11T07:20:20.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:20 smithi196 ceph-mon[30773]: from='client.34883 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:20:20.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:20 smithi196 ceph-mon[30773]: osdmap e161: 8 total, 8 up, 8 in 2024-09-11T07:20:20.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:20 smithi196 ceph-mon[30773]: pgmap v211: 196 pgs: 196 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:20:20.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:20 smithi196 ceph-mon[30773]: osdmap e162: 8 total, 8 up, 8 in 2024-09-11T07:20:20.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:20 smithi071 ceph-mon[27013]: from='client.34883 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:20:20.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:20 smithi071 ceph-mon[27013]: osdmap e161: 8 total, 8 up, 8 in 2024-09-11T07:20:20.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:20 smithi071 ceph-mon[27013]: pgmap v211: 196 pgs: 196 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:20:20.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:20 smithi071 ceph-mon[27013]: osdmap e162: 8 total, 8 up, 8 in 2024-09-11T07:20:20.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:20 smithi071 ceph-mon[37727]: from='client.34883 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:20:20.591 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:20 smithi071 ceph-mon[37727]: osdmap e161: 8 total, 8 up, 8 in 2024-09-11T07:20:20.591 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:20 smithi071 ceph-mon[37727]: pgmap v211: 196 pgs: 196 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:20:20.591 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:20 smithi071 ceph-mon[37727]: osdmap e162: 8 total, 8 up, 8 in 2024-09-11T07:20:22.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:22 smithi196 ceph-mon[30773]: osdmap e163: 8 total, 8 up, 8 in 2024-09-11T07:20:22.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:22 smithi196 ceph-mon[30773]: pgmap v214: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:20:22.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:22 smithi071 ceph-mon[27013]: osdmap e163: 8 total, 8 up, 8 in 2024-09-11T07:20:22.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:22 smithi071 ceph-mon[27013]: pgmap v214: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:20:22.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:22 smithi071 ceph-mon[37727]: osdmap e163: 8 total, 8 up, 8 in 2024-09-11T07:20:22.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:22 smithi071 ceph-mon[37727]: pgmap v214: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:20:23.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:23 smithi196 ceph-mon[30773]: osdmap e164: 8 total, 8 up, 8 in 2024-09-11T07:20:23.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:23 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/1735864329' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:20:23.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:23 smithi196 ceph-mon[30773]: from='client.34889 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:20:23.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:23 smithi071 ceph-mon[27013]: osdmap e164: 8 total, 8 up, 8 in 2024-09-11T07:20:23.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:23 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/1735864329' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:20:23.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:23 smithi071 ceph-mon[27013]: from='client.34889 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:20:23.589 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:23 smithi071 ceph-mon[37727]: osdmap e164: 8 total, 8 up, 8 in 2024-09-11T07:20:23.589 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:23 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/1735864329' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:20:23.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:23 smithi071 ceph-mon[37727]: from='client.34889 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:20:24.156 INFO:tasks.workunit.client.0.smithi071.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_lookup_snap PASSED [ 44%] 2024-09-11T07:20:24.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:24 smithi196 ceph-mon[30773]: from='client.34889 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:20:24.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:24 smithi196 ceph-mon[30773]: osdmap e165: 8 total, 8 up, 8 in 2024-09-11T07:20:24.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:24 smithi196 ceph-mon[30773]: pgmap v217: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:20:24.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:24 smithi196 ceph-mon[30773]: osdmap e166: 8 total, 8 up, 8 in 2024-09-11T07:20:24.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:24 smithi071 ceph-mon[27013]: from='client.34889 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:20:24.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:24 smithi071 ceph-mon[27013]: osdmap e165: 8 total, 8 up, 8 in 2024-09-11T07:20:24.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:24 smithi071 ceph-mon[27013]: pgmap v217: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:20:24.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:24 smithi071 ceph-mon[27013]: osdmap e166: 8 total, 8 up, 8 in 2024-09-11T07:20:24.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:24 smithi071 ceph-mon[37727]: from='client.34889 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:20:24.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:24 smithi071 ceph-mon[37727]: osdmap e165: 8 total, 8 up, 8 in 2024-09-11T07:20:24.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:24 smithi071 ceph-mon[37727]: pgmap v217: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:20:24.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:24 smithi071 ceph-mon[37727]: osdmap e166: 8 total, 8 up, 8 in 2024-09-11T07:20:25.035 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 11 07:20:24 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-iscsi-iscsi-a[58197]: debug there is no tcmu-runner data available 2024-09-11T07:20:25.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:25 smithi196 ceph-mon[30773]: from='client.34442 v1:172.21.15.196:0/1952921461' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:20:25.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:25 smithi071 ceph-mon[27013]: from='client.34442 v1:172.21.15.196:0/1952921461' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:20:25.589 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:25 smithi071 ceph-mon[37727]: from='client.34442 v1:172.21.15.196:0/1952921461' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:20:26.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:26 smithi196 ceph-mon[30773]: osdmap e167: 8 total, 8 up, 8 in 2024-09-11T07:20:26.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:26 smithi196 ceph-mon[30773]: pgmap v220: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:20:26.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:26 smithi196 ceph-mon[30773]: osdmap e168: 8 total, 8 up, 8 in 2024-09-11T07:20:26.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:26 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/2559382414' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:20:26.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:26 smithi071 ceph-mon[27013]: osdmap e167: 8 total, 8 up, 8 in 2024-09-11T07:20:26.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:26 smithi071 ceph-mon[27013]: pgmap v220: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:20:26.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:26 smithi071 ceph-mon[27013]: osdmap e168: 8 total, 8 up, 8 in 2024-09-11T07:20:26.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:26 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/2559382414' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:20:26.590 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:20:26 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: ::ffff:172.21.15.196 - - [11/Sep/2024:07:20:26] "GET /metrics HTTP/1.1" 200 224919 "" "Prometheus/2.43.0" 2024-09-11T07:20:26.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:26 smithi071 ceph-mon[37727]: osdmap e167: 8 total, 8 up, 8 in 2024-09-11T07:20:26.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:26 smithi071 ceph-mon[37727]: pgmap v220: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:20:26.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:26 smithi071 ceph-mon[37727]: osdmap e168: 8 total, 8 up, 8 in 2024-09-11T07:20:26.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:26 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/2559382414' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:20:27.035 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 07:20:26 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-x[32609]: ::ffff:172.21.15.196 - - [11/Sep/2024:07:20:26] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:20:28.186 INFO:tasks.workunit.client.0.smithi071.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_snap_timestamp PASSED [ 46%] 2024-09-11T07:20:28.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:28 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/2559382414' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:20:28.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:28 smithi196 ceph-mon[30773]: osdmap e169: 8 total, 8 up, 8 in 2024-09-11T07:20:28.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:28 smithi196 ceph-mon[30773]: pgmap v223: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:20:28.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:28 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/2559382414' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:20:28.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:28 smithi071 ceph-mon[27013]: osdmap e169: 8 total, 8 up, 8 in 2024-09-11T07:20:28.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:28 smithi071 ceph-mon[27013]: pgmap v223: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:20:28.589 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:28 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/2559382414' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:20:28.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:28 smithi071 ceph-mon[37727]: osdmap e169: 8 total, 8 up, 8 in 2024-09-11T07:20:28.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:28 smithi071 ceph-mon[37727]: pgmap v223: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:20:29.490 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:29 smithi071 ceph-mon[27013]: osdmap e170: 8 total, 8 up, 8 in 2024-09-11T07:20:29.491 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:29 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:20:29.491 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:29 smithi071 ceph-mon[37727]: osdmap e170: 8 total, 8 up, 8 in 2024-09-11T07:20:29.491 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:29 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:20:29.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:29 smithi196 ceph-mon[30773]: osdmap e170: 8 total, 8 up, 8 in 2024-09-11T07:20:29.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:29 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:20:30.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:30 smithi196 ceph-mon[30773]: osdmap e171: 8 total, 8 up, 8 in 2024-09-11T07:20:30.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:30 smithi196 ceph-mon[30773]: pgmap v226: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:20:30.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:30 smithi196 ceph-mon[30773]: osdmap e172: 8 total, 8 up, 8 in 2024-09-11T07:20:30.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:30 smithi071 ceph-mon[27013]: osdmap e171: 8 total, 8 up, 8 in 2024-09-11T07:20:30.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:30 smithi071 ceph-mon[27013]: pgmap v226: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:20:30.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:30 smithi071 ceph-mon[27013]: osdmap e172: 8 total, 8 up, 8 in 2024-09-11T07:20:30.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:30 smithi071 ceph-mon[37727]: osdmap e171: 8 total, 8 up, 8 in 2024-09-11T07:20:30.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:30 smithi071 ceph-mon[37727]: pgmap v226: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:20:30.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:30 smithi071 ceph-mon[37727]: osdmap e172: 8 total, 8 up, 8 in 2024-09-11T07:20:32.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:32 smithi196 ceph-mon[30773]: osdmap e173: 8 total, 8 up, 8 in 2024-09-11T07:20:32.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:32 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/728317726' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:20:32.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:32 smithi196 ceph-mon[30773]: from='client.24910 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:20:32.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:32 smithi196 ceph-mon[30773]: pgmap v229: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:20:32.589 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:32 smithi071 ceph-mon[37727]: osdmap e173: 8 total, 8 up, 8 in 2024-09-11T07:20:32.589 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:32 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/728317726' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:20:32.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:32 smithi071 ceph-mon[37727]: from='client.24910 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:20:32.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:32 smithi071 ceph-mon[37727]: pgmap v229: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:20:32.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:32 smithi071 ceph-mon[27013]: osdmap e173: 8 total, 8 up, 8 in 2024-09-11T07:20:32.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:32 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/728317726' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:20:32.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:32 smithi071 ceph-mon[27013]: from='client.24910 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:20:32.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:32 smithi071 ceph-mon[27013]: pgmap v229: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:20:33.200 INFO:tasks.workunit.client.0.smithi071.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_remove_snap PASSED [ 47%] 2024-09-11T07:20:33.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:33 smithi196 ceph-mon[30773]: from='client.24910 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:20:33.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:33 smithi196 ceph-mon[30773]: osdmap e174: 8 total, 8 up, 8 in 2024-09-11T07:20:33.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:33 smithi071 ceph-mon[27013]: from='client.24910 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:20:33.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:33 smithi071 ceph-mon[27013]: osdmap e174: 8 total, 8 up, 8 in 2024-09-11T07:20:33.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:33 smithi071 ceph-mon[37727]: from='client.24910 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:20:33.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:33 smithi071 ceph-mon[37727]: osdmap e174: 8 total, 8 up, 8 in 2024-09-11T07:20:34.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:34 smithi196 ceph-mon[30773]: osdmap e175: 8 total, 8 up, 8 in 2024-09-11T07:20:34.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:34 smithi196 ceph-mon[30773]: pgmap v232: 164 pgs: 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:20:34.589 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:34 smithi071 ceph-mon[37727]: osdmap e175: 8 total, 8 up, 8 in 2024-09-11T07:20:34.589 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:34 smithi071 ceph-mon[37727]: pgmap v232: 164 pgs: 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:20:34.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:34 smithi071 ceph-mon[27013]: osdmap e175: 8 total, 8 up, 8 in 2024-09-11T07:20:34.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:34 smithi071 ceph-mon[27013]: pgmap v232: 164 pgs: 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:20:35.035 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 11 07:20:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-iscsi-iscsi-a[58197]: debug there is no tcmu-runner data available 2024-09-11T07:20:35.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:35 smithi196 ceph-mon[30773]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T07:20:35.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:35 smithi196 ceph-mon[30773]: osdmap e176: 8 total, 8 up, 8 in 2024-09-11T07:20:35.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:35 smithi196 ceph-mon[30773]: from='client.34442 v1:172.21.15.196:0/1952921461' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:20:35.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:35 smithi071 ceph-mon[27013]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T07:20:35.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:35 smithi071 ceph-mon[27013]: osdmap e176: 8 total, 8 up, 8 in 2024-09-11T07:20:35.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:35 smithi071 ceph-mon[27013]: from='client.34442 v1:172.21.15.196:0/1952921461' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:20:35.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:35 smithi071 ceph-mon[37727]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T07:20:35.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:35 smithi071 ceph-mon[37727]: osdmap e176: 8 total, 8 up, 8 in 2024-09-11T07:20:35.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:35 smithi071 ceph-mon[37727]: from='client.34442 v1:172.21.15.196:0/1952921461' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:20:36.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:36 smithi196 ceph-mon[30773]: osdmap e177: 8 total, 8 up, 8 in 2024-09-11T07:20:36.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:36 smithi196 ceph-mon[30773]: pgmap v235: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:20:36.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:36 smithi071 ceph-mon[27013]: osdmap e177: 8 total, 8 up, 8 in 2024-09-11T07:20:36.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:36 smithi071 ceph-mon[27013]: pgmap v235: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:20:36.590 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:20:36 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: ::ffff:172.21.15.196 - - [11/Sep/2024:07:20:36] "GET /metrics HTTP/1.1" 200 222605 "" "Prometheus/2.43.0" 2024-09-11T07:20:36.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:36 smithi071 ceph-mon[37727]: osdmap e177: 8 total, 8 up, 8 in 2024-09-11T07:20:36.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:36 smithi071 ceph-mon[37727]: pgmap v235: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:20:37.035 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 07:20:36 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-x[32609]: ::ffff:172.21.15.196 - - [11/Sep/2024:07:20:36] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:20:37.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:37 smithi196 ceph-mon[30773]: osdmap e178: 8 total, 8 up, 8 in 2024-09-11T07:20:37.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:37 smithi071 ceph-mon[27013]: osdmap e178: 8 total, 8 up, 8 in 2024-09-11T07:20:37.589 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:37 smithi071 ceph-mon[37727]: osdmap e178: 8 total, 8 up, 8 in 2024-09-11T07:20:38.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:38 smithi196 ceph-mon[30773]: osdmap e179: 8 total, 8 up, 8 in 2024-09-11T07:20:38.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:38 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/3389779582' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:20:38.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:38 smithi196 ceph-mon[30773]: pgmap v238: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:20:38.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:38 smithi071 ceph-mon[27013]: osdmap e179: 8 total, 8 up, 8 in 2024-09-11T07:20:38.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:38 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/3389779582' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:20:38.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:38 smithi071 ceph-mon[27013]: pgmap v238: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:20:38.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:38 smithi071 ceph-mon[37727]: osdmap e179: 8 total, 8 up, 8 in 2024-09-11T07:20:38.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:38 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/3389779582' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:20:38.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:38 smithi071 ceph-mon[37727]: pgmap v238: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:20:39.248 INFO:tasks.workunit.client.0.smithi071.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_snap_rollback PASSED [ 48%] 2024-09-11T07:20:39.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:39 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/3389779582' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:20:39.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:39 smithi196 ceph-mon[30773]: osdmap e180: 8 total, 8 up, 8 in 2024-09-11T07:20:39.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:39 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/3389779582' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:20:39.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:39 smithi071 ceph-mon[27013]: osdmap e180: 8 total, 8 up, 8 in 2024-09-11T07:20:39.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:39 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/3389779582' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:20:39.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:39 smithi071 ceph-mon[37727]: osdmap e180: 8 total, 8 up, 8 in 2024-09-11T07:20:40.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:40 smithi196 ceph-mon[30773]: osdmap e181: 8 total, 8 up, 8 in 2024-09-11T07:20:40.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:40 smithi196 ceph-mon[30773]: pgmap v241: 164 pgs: 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:20:40.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:40 smithi071 ceph-mon[27013]: osdmap e181: 8 total, 8 up, 8 in 2024-09-11T07:20:40.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:40 smithi071 ceph-mon[27013]: pgmap v241: 164 pgs: 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:20:40.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:40 smithi071 ceph-mon[37727]: osdmap e181: 8 total, 8 up, 8 in 2024-09-11T07:20:40.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:40 smithi071 ceph-mon[37727]: pgmap v241: 164 pgs: 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:20:41.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:41 smithi196 ceph-mon[30773]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T07:20:41.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:41 smithi196 ceph-mon[30773]: osdmap e182: 8 total, 8 up, 8 in 2024-09-11T07:20:41.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:41 smithi071 ceph-mon[27013]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T07:20:41.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:41 smithi071 ceph-mon[27013]: osdmap e182: 8 total, 8 up, 8 in 2024-09-11T07:20:41.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:41 smithi071 ceph-mon[37727]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T07:20:41.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:41 smithi071 ceph-mon[37727]: osdmap e182: 8 total, 8 up, 8 in 2024-09-11T07:20:42.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:42 smithi196 ceph-mon[30773]: osdmap e183: 8 total, 8 up, 8 in 2024-09-11T07:20:42.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:42 smithi196 ceph-mon[30773]: pgmap v244: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:20:42.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:42 smithi071 ceph-mon[27013]: osdmap e183: 8 total, 8 up, 8 in 2024-09-11T07:20:42.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:42 smithi071 ceph-mon[27013]: pgmap v244: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:20:42.589 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:42 smithi071 ceph-mon[37727]: osdmap e183: 8 total, 8 up, 8 in 2024-09-11T07:20:42.589 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:42 smithi071 ceph-mon[37727]: pgmap v244: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:20:43.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:43 smithi196 ceph-mon[30773]: osdmap e184: 8 total, 8 up, 8 in 2024-09-11T07:20:43.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:43 smithi071 ceph-mon[27013]: osdmap e184: 8 total, 8 up, 8 in 2024-09-11T07:20:43.589 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:43 smithi071 ceph-mon[37727]: osdmap e184: 8 total, 8 up, 8 in 2024-09-11T07:20:44.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:44 smithi196 ceph-mon[30773]: osdmap e185: 8 total, 8 up, 8 in 2024-09-11T07:20:44.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:44 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/1609583723' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:20:44.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:44 smithi196 ceph-mon[30773]: from='client.34913 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:20:44.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:44 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:20:44.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:44 smithi196 ceph-mon[30773]: pgmap v247: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:20:44.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:44 smithi071 ceph-mon[27013]: osdmap e185: 8 total, 8 up, 8 in 2024-09-11T07:20:44.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:44 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/1609583723' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:20:44.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:44 smithi071 ceph-mon[27013]: from='client.34913 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:20:44.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:44 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:20:44.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:44 smithi071 ceph-mon[27013]: pgmap v247: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:20:44.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:44 smithi071 ceph-mon[37727]: osdmap e185: 8 total, 8 up, 8 in 2024-09-11T07:20:44.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:44 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/1609583723' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:20:44.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:44 smithi071 ceph-mon[37727]: from='client.34913 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:20:44.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:44 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:20:44.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:44 smithi071 ceph-mon[37727]: pgmap v247: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:20:45.035 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 11 07:20:44 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-iscsi-iscsi-a[58197]: debug there is no tcmu-runner data available 2024-09-11T07:20:45.298 INFO:tasks.workunit.client.0.smithi071.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_snap_read PASSED [ 49%] 2024-09-11T07:20:45.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:45 smithi071 ceph-mon[27013]: from='client.34913 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:20:45.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:45 smithi071 ceph-mon[27013]: osdmap e186: 8 total, 8 up, 8 in 2024-09-11T07:20:45.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:45 smithi071 ceph-mon[27013]: from='client.34442 v1:172.21.15.196:0/1952921461' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:20:45.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:45 smithi071 ceph-mon[37727]: from='client.34913 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:20:45.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:45 smithi071 ceph-mon[37727]: osdmap e186: 8 total, 8 up, 8 in 2024-09-11T07:20:45.591 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:45 smithi071 ceph-mon[37727]: from='client.34442 v1:172.21.15.196:0/1952921461' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:20:45.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:45 smithi196 ceph-mon[30773]: from='client.34913 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:20:45.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:45 smithi196 ceph-mon[30773]: osdmap e186: 8 total, 8 up, 8 in 2024-09-11T07:20:45.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:45 smithi196 ceph-mon[30773]: from='client.34442 v1:172.21.15.196:0/1952921461' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:20:46.553 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:46 smithi196 ceph-mon[30773]: osdmap e187: 8 total, 8 up, 8 in 2024-09-11T07:20:46.553 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:46 smithi196 ceph-mon[30773]: pgmap v250: 164 pgs: 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:20:46.589 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:46 smithi071 ceph-mon[37727]: osdmap e187: 8 total, 8 up, 8 in 2024-09-11T07:20:46.589 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:46 smithi071 ceph-mon[37727]: pgmap v250: 164 pgs: 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:20:46.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:46 smithi071 ceph-mon[27013]: osdmap e187: 8 total, 8 up, 8 in 2024-09-11T07:20:46.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:46 smithi071 ceph-mon[27013]: pgmap v250: 164 pgs: 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:20:46.590 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:20:46 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: ::ffff:172.21.15.196 - - [11/Sep/2024:07:20:46] "GET /metrics HTTP/1.1" 200 224961 "" "Prometheus/2.43.0" 2024-09-11T07:20:47.035 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 07:20:46 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-x[32609]: ::ffff:172.21.15.196 - - [11/Sep/2024:07:20:46] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:20:47.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:47 smithi071 ceph-mon[27013]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T07:20:47.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:47 smithi071 ceph-mon[27013]: osdmap e188: 8 total, 8 up, 8 in 2024-09-11T07:20:47.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:47 smithi071 ceph-mon[37727]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T07:20:47.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:47 smithi071 ceph-mon[37727]: osdmap e188: 8 total, 8 up, 8 in 2024-09-11T07:20:47.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:47 smithi196 ceph-mon[30773]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T07:20:47.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:47 smithi196 ceph-mon[30773]: osdmap e188: 8 total, 8 up, 8 in 2024-09-11T07:20:48.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:48 smithi071 ceph-mon[27013]: osdmap e189: 8 total, 8 up, 8 in 2024-09-11T07:20:48.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:48 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/1527687662' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:20:48.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:48 smithi071 ceph-mon[27013]: from='client.34919 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:20:48.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:48 smithi071 ceph-mon[27013]: pgmap v253: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:20:48.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:48 smithi071 ceph-mon[37727]: osdmap e189: 8 total, 8 up, 8 in 2024-09-11T07:20:48.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:48 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/1527687662' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:20:48.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:48 smithi071 ceph-mon[37727]: from='client.34919 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:20:48.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:48 smithi071 ceph-mon[37727]: pgmap v253: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:20:48.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:48 smithi196 ceph-mon[30773]: osdmap e189: 8 total, 8 up, 8 in 2024-09-11T07:20:48.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:48 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/1527687662' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:20:48.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:48 smithi196 ceph-mon[30773]: from='client.34919 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:20:48.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:48 smithi196 ceph-mon[30773]: pgmap v253: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:20:49.336 INFO:tasks.workunit.client.0.smithi071.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_set_omap PASSED [ 50%] 2024-09-11T07:20:49.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:49 smithi071 ceph-mon[27013]: from='client.34919 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:20:49.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:49 smithi071 ceph-mon[27013]: osdmap e190: 8 total, 8 up, 8 in 2024-09-11T07:20:49.589 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:49 smithi071 ceph-mon[37727]: from='client.34919 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:20:49.589 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:49 smithi071 ceph-mon[37727]: osdmap e190: 8 total, 8 up, 8 in 2024-09-11T07:20:49.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:49 smithi196 ceph-mon[30773]: from='client.34919 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:20:49.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:49 smithi196 ceph-mon[30773]: osdmap e190: 8 total, 8 up, 8 in 2024-09-11T07:20:50.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:50 smithi196 ceph-mon[30773]: osdmap e191: 8 total, 8 up, 8 in 2024-09-11T07:20:50.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:50 smithi196 ceph-mon[30773]: pgmap v256: 164 pgs: 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:20:50.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:50 smithi071 ceph-mon[27013]: osdmap e191: 8 total, 8 up, 8 in 2024-09-11T07:20:50.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:50 smithi071 ceph-mon[27013]: pgmap v256: 164 pgs: 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:20:50.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:50 smithi071 ceph-mon[37727]: osdmap e191: 8 total, 8 up, 8 in 2024-09-11T07:20:50.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:50 smithi071 ceph-mon[37727]: pgmap v256: 164 pgs: 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:20:51.784 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:51 smithi196 ceph-mon[30773]: osdmap e192: 8 total, 8 up, 8 in 2024-09-11T07:20:51.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:51 smithi071 ceph-mon[27013]: osdmap e192: 8 total, 8 up, 8 in 2024-09-11T07:20:51.839 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:51 smithi071 ceph-mon[37727]: osdmap e192: 8 total, 8 up, 8 in 2024-09-11T07:20:52.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:52 smithi196 ceph-mon[30773]: osdmap e193: 8 total, 8 up, 8 in 2024-09-11T07:20:52.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:52 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/1573097400' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:20:52.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:52 smithi196 ceph-mon[30773]: from='client.34925 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:20:52.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:52 smithi196 ceph-mon[30773]: pgmap v259: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:20:52.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:52 smithi071 ceph-mon[27013]: osdmap e193: 8 total, 8 up, 8 in 2024-09-11T07:20:52.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:52 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/1573097400' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:20:52.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:52 smithi071 ceph-mon[27013]: from='client.34925 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:20:52.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:52 smithi071 ceph-mon[27013]: pgmap v259: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:20:52.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:52 smithi071 ceph-mon[37727]: osdmap e193: 8 total, 8 up, 8 in 2024-09-11T07:20:52.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:52 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/1573097400' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:20:52.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:52 smithi071 ceph-mon[37727]: from='client.34925 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:20:52.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:52 smithi071 ceph-mon[37727]: pgmap v259: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:20:53.371 INFO:tasks.workunit.client.0.smithi071.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_set_omap_aio PASSED [ 51%] 2024-09-11T07:20:53.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:53 smithi196 ceph-mon[30773]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T07:20:53.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:53 smithi196 ceph-mon[30773]: from='client.34925 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:20:53.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:53 smithi196 ceph-mon[30773]: osdmap e194: 8 total, 8 up, 8 in 2024-09-11T07:20:53.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:53 smithi071 ceph-mon[27013]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T07:20:53.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:53 smithi071 ceph-mon[27013]: from='client.34925 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:20:53.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:53 smithi071 ceph-mon[27013]: osdmap e194: 8 total, 8 up, 8 in 2024-09-11T07:20:53.839 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:53 smithi071 ceph-mon[37727]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T07:20:53.839 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:53 smithi071 ceph-mon[37727]: from='client.34925 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:20:53.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:53 smithi071 ceph-mon[37727]: osdmap e194: 8 total, 8 up, 8 in 2024-09-11T07:20:54.772 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:54 smithi196 ceph-mon[30773]: osdmap e195: 8 total, 8 up, 8 in 2024-09-11T07:20:54.772 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:54 smithi196 ceph-mon[30773]: pgmap v262: 164 pgs: 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:20:54.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:54 smithi071 ceph-mon[27013]: osdmap e195: 8 total, 8 up, 8 in 2024-09-11T07:20:54.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:54 smithi071 ceph-mon[27013]: pgmap v262: 164 pgs: 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:20:54.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:54 smithi071 ceph-mon[37727]: osdmap e195: 8 total, 8 up, 8 in 2024-09-11T07:20:54.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:54 smithi071 ceph-mon[37727]: pgmap v262: 164 pgs: 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:20:55.034 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 11 07:20:54 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-iscsi-iscsi-a[58197]: debug there is no tcmu-runner data available 2024-09-11T07:20:55.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:55 smithi196 ceph-mon[30773]: osdmap e196: 8 total, 8 up, 8 in 2024-09-11T07:20:55.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:55 smithi196 ceph-mon[30773]: from='client.34442 v1:172.21.15.196:0/1952921461' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:20:55.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:55 smithi071 ceph-mon[27013]: osdmap e196: 8 total, 8 up, 8 in 2024-09-11T07:20:55.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:55 smithi071 ceph-mon[27013]: from='client.34442 v1:172.21.15.196:0/1952921461' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:20:55.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:55 smithi071 ceph-mon[37727]: osdmap e196: 8 total, 8 up, 8 in 2024-09-11T07:20:55.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:55 smithi071 ceph-mon[37727]: from='client.34442 v1:172.21.15.196:0/1952921461' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:20:56.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:56 smithi071 ceph-mon[27013]: osdmap e197: 8 total, 8 up, 8 in 2024-09-11T07:20:56.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:56 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/3284023513' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:20:56.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:56 smithi071 ceph-mon[27013]: pgmap v265: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:20:56.590 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:20:56 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: ::ffff:172.21.15.196 - - [11/Sep/2024:07:20:56] "GET /metrics HTTP/1.1" 200 224961 "" "Prometheus/2.43.0" 2024-09-11T07:20:56.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:56 smithi071 ceph-mon[37727]: osdmap e197: 8 total, 8 up, 8 in 2024-09-11T07:20:56.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:56 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/3284023513' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:20:56.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:56 smithi071 ceph-mon[37727]: pgmap v265: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:20:56.785 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 07:20:56 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-x[32609]: ::ffff:172.21.15.196 - - [11/Sep/2024:07:20:56] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:20:56.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:56 smithi196 ceph-mon[30773]: osdmap e197: 8 total, 8 up, 8 in 2024-09-11T07:20:56.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:56 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/3284023513' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:20:56.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:56 smithi196 ceph-mon[30773]: pgmap v265: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:20:57.406 INFO:tasks.workunit.client.0.smithi071.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_write_ops PASSED [ 52%] 2024-09-11T07:20:57.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:57 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/3284023513' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:20:57.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:57 smithi196 ceph-mon[30773]: osdmap e198: 8 total, 8 up, 8 in 2024-09-11T07:20:57.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:57 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/3284023513' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:20:57.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:57 smithi071 ceph-mon[27013]: osdmap e198: 8 total, 8 up, 8 in 2024-09-11T07:20:57.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:57 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/3284023513' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:20:57.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:57 smithi071 ceph-mon[37727]: osdmap e198: 8 total, 8 up, 8 in 2024-09-11T07:20:58.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:58 smithi196 ceph-mon[30773]: osdmap e199: 8 total, 8 up, 8 in 2024-09-11T07:20:58.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:58 smithi196 ceph-mon[30773]: pgmap v268: 164 pgs: 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:20:58.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:58 smithi071 ceph-mon[27013]: osdmap e199: 8 total, 8 up, 8 in 2024-09-11T07:20:58.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:58 smithi071 ceph-mon[27013]: pgmap v268: 164 pgs: 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:20:58.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:58 smithi071 ceph-mon[37727]: osdmap e199: 8 total, 8 up, 8 in 2024-09-11T07:20:58.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:58 smithi071 ceph-mon[37727]: pgmap v268: 164 pgs: 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:20:59.741 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:59 smithi071 ceph-mon[27013]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T07:20:59.741 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:59 smithi071 ceph-mon[27013]: osdmap e200: 8 total, 8 up, 8 in 2024-09-11T07:20:59.741 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:20:59 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:20:59.741 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:59 smithi071 ceph-mon[37727]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T07:20:59.741 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:59 smithi071 ceph-mon[37727]: osdmap e200: 8 total, 8 up, 8 in 2024-09-11T07:20:59.741 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:20:59 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:20:59.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:59 smithi196 ceph-mon[30773]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T07:20:59.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:59 smithi196 ceph-mon[30773]: osdmap e200: 8 total, 8 up, 8 in 2024-09-11T07:20:59.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:20:59 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:21:00.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:00 smithi196 ceph-mon[30773]: osdmap e201: 8 total, 8 up, 8 in 2024-09-11T07:21:00.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:00 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/2627163274' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:21:00.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:00 smithi196 ceph-mon[30773]: pgmap v271: 196 pgs: 196 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-11T07:21:00.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:00 smithi071 ceph-mon[27013]: osdmap e201: 8 total, 8 up, 8 in 2024-09-11T07:21:00.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:00 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/2627163274' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:21:00.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:00 smithi071 ceph-mon[27013]: pgmap v271: 196 pgs: 196 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-11T07:21:00.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:00 smithi071 ceph-mon[37727]: osdmap e201: 8 total, 8 up, 8 in 2024-09-11T07:21:00.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:00 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/2627163274' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:21:00.841 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:00 smithi071 ceph-mon[37727]: pgmap v271: 196 pgs: 196 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-11T07:21:01.441 INFO:tasks.workunit.client.0.smithi071.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_execute_op PASSED [ 53%] 2024-09-11T07:21:01.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:01 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/2627163274' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:21:01.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:01 smithi196 ceph-mon[30773]: osdmap e202: 8 total, 8 up, 8 in 2024-09-11T07:21:01.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:01 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/2627163274' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:21:01.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:01 smithi071 ceph-mon[27013]: osdmap e202: 8 total, 8 up, 8 in 2024-09-11T07:21:01.839 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:01 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/2627163274' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:21:01.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:01 smithi071 ceph-mon[37727]: osdmap e202: 8 total, 8 up, 8 in 2024-09-11T07:21:02.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:02 smithi196 ceph-mon[30773]: osdmap e203: 8 total, 8 up, 8 in 2024-09-11T07:21:02.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:02 smithi196 ceph-mon[30773]: pgmap v274: 164 pgs: 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:21:02.839 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:02 smithi071 ceph-mon[37727]: osdmap e203: 8 total, 8 up, 8 in 2024-09-11T07:21:02.839 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:02 smithi071 ceph-mon[37727]: pgmap v274: 164 pgs: 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:21:02.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:02 smithi071 ceph-mon[27013]: osdmap e203: 8 total, 8 up, 8 in 2024-09-11T07:21:02.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:02 smithi071 ceph-mon[27013]: pgmap v274: 164 pgs: 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:21:03.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:03 smithi196 ceph-mon[30773]: osdmap e204: 8 total, 8 up, 8 in 2024-09-11T07:21:03.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:03 smithi071 ceph-mon[27013]: osdmap e204: 8 total, 8 up, 8 in 2024-09-11T07:21:03.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:03 smithi071 ceph-mon[37727]: osdmap e204: 8 total, 8 up, 8 in 2024-09-11T07:21:04.775 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:04 smithi196 ceph-mon[30773]: osdmap e205: 8 total, 8 up, 8 in 2024-09-11T07:21:04.775 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:04 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/820786891' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:21:04.775 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:04 smithi196 ceph-mon[30773]: from='client.34943 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:21:04.775 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:04 smithi196 ceph-mon[30773]: pgmap v277: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:21:04.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:04 smithi071 ceph-mon[27013]: osdmap e205: 8 total, 8 up, 8 in 2024-09-11T07:21:04.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:04 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/820786891' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:21:04.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:04 smithi071 ceph-mon[27013]: from='client.34943 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:21:04.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:04 smithi071 ceph-mon[27013]: pgmap v277: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:21:04.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:04 smithi071 ceph-mon[37727]: osdmap e205: 8 total, 8 up, 8 in 2024-09-11T07:21:04.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:04 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/820786891' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:21:04.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:04 smithi071 ceph-mon[37727]: from='client.34943 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:21:04.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:04 smithi071 ceph-mon[37727]: pgmap v277: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:21:05.035 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 11 07:21:04 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-iscsi-iscsi-a[58197]: debug there is no tcmu-runner data available 2024-09-11T07:21:05.473 INFO:tasks.workunit.client.0.smithi071.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_writesame_op PASSED [ 55%] 2024-09-11T07:21:05.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:05 smithi196 ceph-mon[30773]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T07:21:05.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:05 smithi196 ceph-mon[30773]: from='client.34943 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:21:05.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:05 smithi196 ceph-mon[30773]: osdmap e206: 8 total, 8 up, 8 in 2024-09-11T07:21:05.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:05 smithi196 ceph-mon[30773]: from='client.34442 v1:172.21.15.196:0/1952921461' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:21:05.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:05 smithi071 ceph-mon[27013]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T07:21:05.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:05 smithi071 ceph-mon[27013]: from='client.34943 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:21:05.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:05 smithi071 ceph-mon[27013]: osdmap e206: 8 total, 8 up, 8 in 2024-09-11T07:21:05.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:05 smithi071 ceph-mon[27013]: from='client.34442 v1:172.21.15.196:0/1952921461' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:21:05.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:05 smithi071 ceph-mon[37727]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T07:21:05.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:05 smithi071 ceph-mon[37727]: from='client.34943 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:21:05.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:05 smithi071 ceph-mon[37727]: osdmap e206: 8 total, 8 up, 8 in 2024-09-11T07:21:05.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:05 smithi071 ceph-mon[37727]: from='client.34442 v1:172.21.15.196:0/1952921461' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:21:06.480 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:21:06 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: ::ffff:172.21.15.196 - - [11/Sep/2024:07:21:06] "GET /metrics HTTP/1.1" 200 225028 "" "Prometheus/2.43.0" 2024-09-11T07:21:06.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:06 smithi196 ceph-mon[30773]: osdmap e207: 8 total, 8 up, 8 in 2024-09-11T07:21:06.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:06 smithi196 ceph-mon[30773]: pgmap v280: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:21:06.785 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 07:21:06 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-x[32609]: ::ffff:172.21.15.196 - - [11/Sep/2024:07:21:06] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:21:06.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:06 smithi071 ceph-mon[27013]: osdmap e207: 8 total, 8 up, 8 in 2024-09-11T07:21:06.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:06 smithi071 ceph-mon[27013]: pgmap v280: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:21:06.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:06 smithi071 ceph-mon[37727]: osdmap e207: 8 total, 8 up, 8 in 2024-09-11T07:21:06.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:06 smithi071 ceph-mon[37727]: pgmap v280: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:21:07.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:07 smithi196 ceph-mon[30773]: osdmap e208: 8 total, 8 up, 8 in 2024-09-11T07:21:07.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:07 smithi071 ceph-mon[27013]: osdmap e208: 8 total, 8 up, 8 in 2024-09-11T07:21:07.839 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:07 smithi071 ceph-mon[37727]: osdmap e208: 8 total, 8 up, 8 in 2024-09-11T07:21:08.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:08 smithi196 ceph-mon[30773]: osdmap e209: 8 total, 8 up, 8 in 2024-09-11T07:21:08.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:08 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/1582039132' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:21:08.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:08 smithi196 ceph-mon[30773]: from='client.34949 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:21:08.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:08 smithi196 ceph-mon[30773]: pgmap v283: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:21:08.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:08 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:21:08.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:08 smithi071 ceph-mon[27013]: osdmap e209: 8 total, 8 up, 8 in 2024-09-11T07:21:08.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:08 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/1582039132' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:21:08.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:08 smithi071 ceph-mon[27013]: from='client.34949 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:21:08.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:08 smithi071 ceph-mon[27013]: pgmap v283: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:21:08.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:08 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:21:08.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:08 smithi071 ceph-mon[37727]: osdmap e209: 8 total, 8 up, 8 in 2024-09-11T07:21:08.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:08 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/1582039132' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:21:08.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:08 smithi071 ceph-mon[37727]: from='client.34949 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:21:08.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:08 smithi071 ceph-mon[37727]: pgmap v283: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:21:08.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:08 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:21:09.499 INFO:tasks.workunit.client.0.smithi071.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_get_omap_vals_by_keys PASSED [ 56%] 2024-09-11T07:21:09.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:09 smithi196 ceph-mon[30773]: from='client.34949 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:21:09.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:09 smithi196 ceph-mon[30773]: osdmap e210: 8 total, 8 up, 8 in 2024-09-11T07:21:09.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:09 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:21:09.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:09 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:21:09.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:09 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:21:09.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:09 smithi071 ceph-mon[27013]: from='client.34949 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:21:09.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:09 smithi071 ceph-mon[27013]: osdmap e210: 8 total, 8 up, 8 in 2024-09-11T07:21:09.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:09 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:21:09.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:09 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:21:09.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:09 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:21:09.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:09 smithi071 ceph-mon[37727]: from='client.34949 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:21:09.841 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:09 smithi071 ceph-mon[37727]: osdmap e210: 8 total, 8 up, 8 in 2024-09-11T07:21:09.841 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:09 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:21:09.841 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:09 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:21:09.841 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:09 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:21:10.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:10 smithi196 ceph-mon[30773]: osdmap e211: 8 total, 8 up, 8 in 2024-09-11T07:21:10.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:10 smithi196 ceph-mon[30773]: pgmap v286: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:21:10.839 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:10 smithi071 ceph-mon[37727]: osdmap e211: 8 total, 8 up, 8 in 2024-09-11T07:21:10.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:10 smithi071 ceph-mon[37727]: pgmap v286: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:21:10.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:10 smithi071 ceph-mon[27013]: osdmap e211: 8 total, 8 up, 8 in 2024-09-11T07:21:10.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:10 smithi071 ceph-mon[27013]: pgmap v286: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:21:11.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:11 smithi196 ceph-mon[30773]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T07:21:11.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:11 smithi196 ceph-mon[30773]: osdmap e212: 8 total, 8 up, 8 in 2024-09-11T07:21:11.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:11 smithi071 ceph-mon[27013]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T07:21:11.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:11 smithi071 ceph-mon[27013]: osdmap e212: 8 total, 8 up, 8 in 2024-09-11T07:21:11.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:11 smithi071 ceph-mon[37727]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T07:21:11.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:11 smithi071 ceph-mon[37727]: osdmap e212: 8 total, 8 up, 8 in 2024-09-11T07:21:12.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:12 smithi196 ceph-mon[30773]: osdmap e213: 8 total, 8 up, 8 in 2024-09-11T07:21:12.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:12 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/4063740129' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:21:12.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:12 smithi196 ceph-mon[30773]: pgmap v289: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:21:12.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:12 smithi071 ceph-mon[27013]: osdmap e213: 8 total, 8 up, 8 in 2024-09-11T07:21:12.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:12 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/4063740129' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:21:12.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:12 smithi071 ceph-mon[27013]: pgmap v289: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:21:12.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:12 smithi071 ceph-mon[37727]: osdmap e213: 8 total, 8 up, 8 in 2024-09-11T07:21:12.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:12 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/4063740129' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:21:12.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:12 smithi071 ceph-mon[37727]: pgmap v289: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:21:13.534 INFO:tasks.workunit.client.0.smithi071.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_get_omap_keys PASSED [ 57%] 2024-09-11T07:21:13.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:13 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/4063740129' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:21:13.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:13 smithi071 ceph-mon[27013]: osdmap e214: 8 total, 8 up, 8 in 2024-09-11T07:21:13.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:13 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/4063740129' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:21:13.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:13 smithi071 ceph-mon[37727]: osdmap e214: 8 total, 8 up, 8 in 2024-09-11T07:21:14.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:13 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/4063740129' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:21:14.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:13 smithi196 ceph-mon[30773]: osdmap e214: 8 total, 8 up, 8 in 2024-09-11T07:21:14.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:14 smithi071 ceph-mon[27013]: osdmap e215: 8 total, 8 up, 8 in 2024-09-11T07:21:14.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:14 smithi071 ceph-mon[27013]: pgmap v292: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:21:14.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:14 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:21:14.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:14 smithi071 ceph-mon[37727]: osdmap e215: 8 total, 8 up, 8 in 2024-09-11T07:21:14.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:14 smithi071 ceph-mon[37727]: pgmap v292: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:21:14.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:14 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:21:15.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:14 smithi196 ceph-mon[30773]: osdmap e215: 8 total, 8 up, 8 in 2024-09-11T07:21:15.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:14 smithi196 ceph-mon[30773]: pgmap v292: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:21:15.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:14 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:21:15.035 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 11 07:21:14 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-iscsi-iscsi-a[58197]: debug there is no tcmu-runner data available 2024-09-11T07:21:15.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:15 smithi071 ceph-mon[27013]: osdmap e216: 8 total, 8 up, 8 in 2024-09-11T07:21:15.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:15 smithi071 ceph-mon[27013]: from='client.34442 v1:172.21.15.196:0/1952921461' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:21:15.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:15 smithi071 ceph-mon[37727]: osdmap e216: 8 total, 8 up, 8 in 2024-09-11T07:21:15.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:15 smithi071 ceph-mon[37727]: from='client.34442 v1:172.21.15.196:0/1952921461' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:21:16.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:15 smithi196 ceph-mon[30773]: osdmap e216: 8 total, 8 up, 8 in 2024-09-11T07:21:16.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:15 smithi196 ceph-mon[30773]: from='client.34442 v1:172.21.15.196:0/1952921461' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:21:16.560 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:21:16 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: ::ffff:172.21.15.196 - - [11/Sep/2024:07:21:16] "GET /metrics HTTP/1.1" 200 222670 "" "Prometheus/2.43.0" 2024-09-11T07:21:16.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:16 smithi071 ceph-mon[27013]: osdmap e217: 8 total, 8 up, 8 in 2024-09-11T07:21:16.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:16 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/469895553' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:21:16.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:16 smithi071 ceph-mon[27013]: pgmap v295: 196 pgs: 30 creating+peering, 2 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:21:16.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:16 smithi071 ceph-mon[37727]: osdmap e217: 8 total, 8 up, 8 in 2024-09-11T07:21:16.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:16 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/469895553' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:21:16.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:16 smithi071 ceph-mon[37727]: pgmap v295: 196 pgs: 30 creating+peering, 2 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:21:17.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:16 smithi196 ceph-mon[30773]: osdmap e217: 8 total, 8 up, 8 in 2024-09-11T07:21:17.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:16 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/469895553' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:21:17.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:16 smithi196 ceph-mon[30773]: pgmap v295: 196 pgs: 30 creating+peering, 2 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:21:17.035 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 07:21:16 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-x[32609]: ::ffff:172.21.15.196 - - [11/Sep/2024:07:21:16] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:21:17.574 INFO:tasks.workunit.client.0.smithi071.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_clear_omap PASSED [ 58%] 2024-09-11T07:21:17.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:17 smithi071 ceph-mon[27013]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T07:21:17.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:17 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/469895553' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:21:17.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:17 smithi071 ceph-mon[27013]: osdmap e218: 8 total, 8 up, 8 in 2024-09-11T07:21:17.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:17 smithi071 ceph-mon[37727]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T07:21:17.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:17 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/469895553' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:21:17.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:17 smithi071 ceph-mon[37727]: osdmap e218: 8 total, 8 up, 8 in 2024-09-11T07:21:18.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:17 smithi196 ceph-mon[30773]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T07:21:18.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:17 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/469895553' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:21:18.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:17 smithi196 ceph-mon[30773]: osdmap e218: 8 total, 8 up, 8 in 2024-09-11T07:21:18.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:18 smithi071 ceph-mon[27013]: osdmap e219: 8 total, 8 up, 8 in 2024-09-11T07:21:18.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:18 smithi071 ceph-mon[27013]: pgmap v298: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:21:18.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:18 smithi071 ceph-mon[37727]: osdmap e219: 8 total, 8 up, 8 in 2024-09-11T07:21:18.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:18 smithi071 ceph-mon[37727]: pgmap v298: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:21:19.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:18 smithi196 ceph-mon[30773]: osdmap e219: 8 total, 8 up, 8 in 2024-09-11T07:21:19.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:18 smithi196 ceph-mon[30773]: pgmap v298: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:21:20.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:19 smithi196 ceph-mon[30773]: osdmap e220: 8 total, 8 up, 8 in 2024-09-11T07:21:20.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:19 smithi071 ceph-mon[27013]: osdmap e220: 8 total, 8 up, 8 in 2024-09-11T07:21:20.089 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:19 smithi071 ceph-mon[37727]: osdmap e220: 8 total, 8 up, 8 in 2024-09-11T07:21:21.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:20 smithi196 ceph-mon[30773]: osdmap e221: 8 total, 8 up, 8 in 2024-09-11T07:21:21.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:20 smithi196 ceph-mon[30773]: pgmap v301: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T07:21:21.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:20 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/1620225283' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:21:21.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:20 smithi071 ceph-mon[27013]: osdmap e221: 8 total, 8 up, 8 in 2024-09-11T07:21:21.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:20 smithi071 ceph-mon[27013]: pgmap v301: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T07:21:21.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:20 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/1620225283' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:21:21.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:20 smithi071 ceph-mon[37727]: osdmap e221: 8 total, 8 up, 8 in 2024-09-11T07:21:21.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:20 smithi071 ceph-mon[37727]: pgmap v301: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T07:21:21.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:20 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/1620225283' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:21:21.607 INFO:tasks.workunit.client.0.smithi071.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_remove_omap_ramge2 PASSED [ 59%] 2024-09-11T07:21:22.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:21 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/1620225283' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:21:22.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:21 smithi196 ceph-mon[30773]: osdmap e222: 8 total, 8 up, 8 in 2024-09-11T07:21:22.092 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:21 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/1620225283' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:21:22.093 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:21 smithi071 ceph-mon[27013]: osdmap e222: 8 total, 8 up, 8 in 2024-09-11T07:21:22.093 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:21 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/1620225283' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:21:22.093 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:21 smithi071 ceph-mon[37727]: osdmap e222: 8 total, 8 up, 8 in 2024-09-11T07:21:23.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:22 smithi196 ceph-mon[30773]: pgmap v304: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T07:21:23.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:22 smithi196 ceph-mon[30773]: osdmap e223: 8 total, 8 up, 8 in 2024-09-11T07:21:23.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:22 smithi196 ceph-mon[30773]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T07:21:23.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:22 smithi071 ceph-mon[27013]: pgmap v304: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T07:21:23.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:22 smithi071 ceph-mon[27013]: osdmap e223: 8 total, 8 up, 8 in 2024-09-11T07:21:23.090 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:22 smithi071 ceph-mon[27013]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T07:21:23.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:22 smithi071 ceph-mon[37727]: pgmap v304: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-11T07:21:23.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:22 smithi071 ceph-mon[37727]: osdmap e223: 8 total, 8 up, 8 in 2024-09-11T07:21:23.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:22 smithi071 ceph-mon[37727]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T07:21:24.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:23 smithi196 ceph-mon[30773]: osdmap e224: 8 total, 8 up, 8 in 2024-09-11T07:21:24.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:23 smithi071 ceph-mon[27013]: osdmap e224: 8 total, 8 up, 8 in 2024-09-11T07:21:24.089 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:23 smithi071 ceph-mon[37727]: osdmap e224: 8 total, 8 up, 8 in 2024-09-11T07:21:25.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:24 smithi196 ceph-mon[30773]: pgmap v306: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:21:25.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:24 smithi196 ceph-mon[30773]: osdmap e225: 8 total, 8 up, 8 in 2024-09-11T07:21:25.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:24 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/420824210' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:21:25.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:24 smithi196 ceph-mon[30773]: from='client.34973 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:21:25.035 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 11 07:21:24 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-iscsi-iscsi-a[58197]: debug there is no tcmu-runner data available 2024-09-11T07:21:25.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:24 smithi071 ceph-mon[27013]: pgmap v306: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:21:25.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:24 smithi071 ceph-mon[27013]: osdmap e225: 8 total, 8 up, 8 in 2024-09-11T07:21:25.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:24 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/420824210' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:21:25.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:24 smithi071 ceph-mon[27013]: from='client.34973 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:21:25.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:24 smithi071 ceph-mon[37727]: pgmap v306: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:21:25.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:24 smithi071 ceph-mon[37727]: osdmap e225: 8 total, 8 up, 8 in 2024-09-11T07:21:25.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:24 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/420824210' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:21:25.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:24 smithi071 ceph-mon[37727]: from='client.34973 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:21:25.643 INFO:tasks.workunit.client.0.smithi071.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_omap_cmp PASSED [ 60%] 2024-09-11T07:21:26.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:25 smithi196 ceph-mon[30773]: from='client.34973 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:21:26.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:25 smithi196 ceph-mon[30773]: osdmap e226: 8 total, 8 up, 8 in 2024-09-11T07:21:26.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:25 smithi196 ceph-mon[30773]: from='client.34442 v1:172.21.15.196:0/1952921461' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:21:26.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:25 smithi071 ceph-mon[27013]: from='client.34973 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:21:26.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:25 smithi071 ceph-mon[27013]: osdmap e226: 8 total, 8 up, 8 in 2024-09-11T07:21:26.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:25 smithi071 ceph-mon[27013]: from='client.34442 v1:172.21.15.196:0/1952921461' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:21:26.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:25 smithi071 ceph-mon[37727]: from='client.34973 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:21:26.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:25 smithi071 ceph-mon[37727]: osdmap e226: 8 total, 8 up, 8 in 2024-09-11T07:21:26.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:25 smithi071 ceph-mon[37727]: from='client.34442 v1:172.21.15.196:0/1952921461' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:21:26.589 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:21:26 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: ::ffff:172.21.15.196 - - [11/Sep/2024:07:21:26] "GET /metrics HTTP/1.1" 200 222670 "" "Prometheus/2.43.0" 2024-09-11T07:21:27.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:26 smithi196 ceph-mon[30773]: pgmap v309: 196 pgs: 196 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-09-11T07:21:27.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:26 smithi196 ceph-mon[30773]: osdmap e227: 8 total, 8 up, 8 in 2024-09-11T07:21:27.035 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 07:21:26 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-x[32609]: ::ffff:172.21.15.196 - - [11/Sep/2024:07:21:26] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:21:27.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:26 smithi071 ceph-mon[27013]: pgmap v309: 196 pgs: 196 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-09-11T07:21:27.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:26 smithi071 ceph-mon[27013]: osdmap e227: 8 total, 8 up, 8 in 2024-09-11T07:21:27.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:26 smithi071 ceph-mon[37727]: pgmap v309: 196 pgs: 196 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-09-11T07:21:27.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:26 smithi071 ceph-mon[37727]: osdmap e227: 8 total, 8 up, 8 in 2024-09-11T07:21:28.036 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:27 smithi196 ceph-mon[30773]: osdmap e228: 8 total, 8 up, 8 in 2024-09-11T07:21:28.089 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:27 smithi071 ceph-mon[37727]: osdmap e228: 8 total, 8 up, 8 in 2024-09-11T07:21:28.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:27 smithi071 ceph-mon[27013]: osdmap e228: 8 total, 8 up, 8 in 2024-09-11T07:21:29.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:28 smithi196 ceph-mon[30773]: pgmap v312: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:21:29.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:28 smithi196 ceph-mon[30773]: osdmap e229: 8 total, 8 up, 8 in 2024-09-11T07:21:29.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:28 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/2984577583' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:21:29.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:28 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:21:29.089 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:28 smithi071 ceph-mon[37727]: pgmap v312: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:21:29.089 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:28 smithi071 ceph-mon[37727]: osdmap e229: 8 total, 8 up, 8 in 2024-09-11T07:21:29.089 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:28 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/2984577583' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:21:29.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:28 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:21:29.090 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:28 smithi071 ceph-mon[27013]: pgmap v312: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:21:29.090 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:28 smithi071 ceph-mon[27013]: osdmap e229: 8 total, 8 up, 8 in 2024-09-11T07:21:29.090 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:28 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/2984577583' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:21:29.090 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:28 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:21:29.660 INFO:tasks.workunit.client.0.smithi071.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_cmpext_op PASSED [ 61%] 2024-09-11T07:21:29.991 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:29 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/2984577583' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:21:29.991 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:29 smithi071 ceph-mon[27013]: osdmap e230: 8 total, 8 up, 8 in 2024-09-11T07:21:29.991 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:29 smithi071 ceph-mon[27013]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T07:21:29.991 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:29 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/2984577583' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:21:29.992 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:29 smithi071 ceph-mon[37727]: osdmap e230: 8 total, 8 up, 8 in 2024-09-11T07:21:29.992 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:29 smithi071 ceph-mon[37727]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T07:21:30.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:29 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/2984577583' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:21:30.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:29 smithi196 ceph-mon[30773]: osdmap e230: 8 total, 8 up, 8 in 2024-09-11T07:21:30.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:29 smithi196 ceph-mon[30773]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T07:21:31.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:30 smithi196 ceph-mon[30773]: pgmap v315: 196 pgs: 196 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-09-11T07:21:31.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:30 smithi196 ceph-mon[30773]: osdmap e231: 8 total, 8 up, 8 in 2024-09-11T07:21:31.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:30 smithi071 ceph-mon[27013]: pgmap v315: 196 pgs: 196 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-09-11T07:21:31.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:30 smithi071 ceph-mon[27013]: osdmap e231: 8 total, 8 up, 8 in 2024-09-11T07:21:31.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:30 smithi071 ceph-mon[37727]: pgmap v315: 196 pgs: 196 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-09-11T07:21:31.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:30 smithi071 ceph-mon[37727]: osdmap e231: 8 total, 8 up, 8 in 2024-09-11T07:21:32.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:31 smithi196 ceph-mon[30773]: osdmap e232: 8 total, 8 up, 8 in 2024-09-11T07:21:32.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:31 smithi071 ceph-mon[27013]: osdmap e232: 8 total, 8 up, 8 in 2024-09-11T07:21:32.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:31 smithi071 ceph-mon[37727]: osdmap e232: 8 total, 8 up, 8 in 2024-09-11T07:21:33.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:32 smithi196 ceph-mon[30773]: pgmap v318: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:21:33.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:32 smithi196 ceph-mon[30773]: osdmap e233: 8 total, 8 up, 8 in 2024-09-11T07:21:33.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:32 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/142119473' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:21:33.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:32 smithi196 ceph-mon[30773]: from='client.34985 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:21:33.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:32 smithi071 ceph-mon[27013]: pgmap v318: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:21:33.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:32 smithi071 ceph-mon[27013]: osdmap e233: 8 total, 8 up, 8 in 2024-09-11T07:21:33.090 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:32 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/142119473' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:21:33.090 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:32 smithi071 ceph-mon[27013]: from='client.34985 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:21:33.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:32 smithi071 ceph-mon[37727]: pgmap v318: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:21:33.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:32 smithi071 ceph-mon[37727]: osdmap e233: 8 total, 8 up, 8 in 2024-09-11T07:21:33.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:32 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/142119473' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:21:33.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:32 smithi071 ceph-mon[37727]: from='client.34985 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:21:33.703 INFO:tasks.workunit.client.0.smithi071.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_xattrs_op PASSED [ 62%] 2024-09-11T07:21:34.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:33 smithi196 ceph-mon[30773]: from='client.34985 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:21:34.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:33 smithi196 ceph-mon[30773]: osdmap e234: 8 total, 8 up, 8 in 2024-09-11T07:21:34.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:33 smithi071 ceph-mon[27013]: from='client.34985 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:21:34.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:33 smithi071 ceph-mon[27013]: osdmap e234: 8 total, 8 up, 8 in 2024-09-11T07:21:34.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:33 smithi071 ceph-mon[37727]: from='client.34985 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:21:34.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:33 smithi071 ceph-mon[37727]: osdmap e234: 8 total, 8 up, 8 in 2024-09-11T07:21:35.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:34 smithi196 ceph-mon[30773]: pgmap v321: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:21:35.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:34 smithi196 ceph-mon[30773]: osdmap e235: 8 total, 8 up, 8 in 2024-09-11T07:21:35.035 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 11 07:21:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-iscsi-iscsi-a[58197]: debug there is no tcmu-runner data available 2024-09-11T07:21:35.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:34 smithi071 ceph-mon[27013]: pgmap v321: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:21:35.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:34 smithi071 ceph-mon[27013]: osdmap e235: 8 total, 8 up, 8 in 2024-09-11T07:21:35.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:34 smithi071 ceph-mon[37727]: pgmap v321: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:21:35.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:34 smithi071 ceph-mon[37727]: osdmap e235: 8 total, 8 up, 8 in 2024-09-11T07:21:36.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:35 smithi196 ceph-mon[30773]: osdmap e236: 8 total, 8 up, 8 in 2024-09-11T07:21:36.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:35 smithi196 ceph-mon[30773]: from='client.34442 v1:172.21.15.196:0/1952921461' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:21:36.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:35 smithi196 ceph-mon[30773]: osdmap e237: 8 total, 8 up, 8 in 2024-09-11T07:21:36.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:35 smithi071 ceph-mon[27013]: osdmap e236: 8 total, 8 up, 8 in 2024-09-11T07:21:36.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:35 smithi071 ceph-mon[27013]: from='client.34442 v1:172.21.15.196:0/1952921461' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:21:36.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:35 smithi071 ceph-mon[27013]: osdmap e237: 8 total, 8 up, 8 in 2024-09-11T07:21:36.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:35 smithi071 ceph-mon[37727]: osdmap e236: 8 total, 8 up, 8 in 2024-09-11T07:21:36.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:35 smithi071 ceph-mon[37727]: from='client.34442 v1:172.21.15.196:0/1952921461' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:21:36.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:35 smithi071 ceph-mon[37727]: osdmap e237: 8 total, 8 up, 8 in 2024-09-11T07:21:36.589 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:21:36 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: ::ffff:172.21.15.196 - - [11/Sep/2024:07:21:36] "GET /metrics HTTP/1.1" 200 225057 "" "Prometheus/2.43.0" 2024-09-11T07:21:37.035 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 07:21:36 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-x[32609]: ::ffff:172.21.15.196 - - [11/Sep/2024:07:21:36] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:21:37.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:36 smithi196 ceph-mon[30773]: pgmap v324: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:21:37.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:36 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/1570967260' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:21:37.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:36 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/1570967260' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:21:37.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:36 smithi196 ceph-mon[30773]: osdmap e238: 8 total, 8 up, 8 in 2024-09-11T07:21:37.090 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:36 smithi071 ceph-mon[27013]: pgmap v324: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:21:37.090 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:36 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/1570967260' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:21:37.090 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:36 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/1570967260' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:21:37.090 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:36 smithi071 ceph-mon[27013]: osdmap e238: 8 total, 8 up, 8 in 2024-09-11T07:21:37.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:36 smithi071 ceph-mon[37727]: pgmap v324: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:21:37.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:36 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/1570967260' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:21:37.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:36 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/1570967260' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:21:37.091 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:36 smithi071 ceph-mon[37727]: osdmap e238: 8 total, 8 up, 8 in 2024-09-11T07:21:37.722 INFO:tasks.workunit.client.0.smithi071.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_locator PASSED [ 64%] 2024-09-11T07:21:39.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:38 smithi196 ceph-mon[30773]: pgmap v327: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:21:39.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:38 smithi196 ceph-mon[30773]: osdmap e239: 8 total, 8 up, 8 in 2024-09-11T07:21:39.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:38 smithi071 ceph-mon[27013]: pgmap v327: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:21:39.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:38 smithi071 ceph-mon[27013]: osdmap e239: 8 total, 8 up, 8 in 2024-09-11T07:21:39.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:38 smithi071 ceph-mon[37727]: pgmap v327: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:21:39.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:38 smithi071 ceph-mon[37727]: osdmap e239: 8 total, 8 up, 8 in 2024-09-11T07:21:40.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:39 smithi196 ceph-mon[30773]: osdmap e240: 8 total, 8 up, 8 in 2024-09-11T07:21:40.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:39 smithi071 ceph-mon[27013]: osdmap e240: 8 total, 8 up, 8 in 2024-09-11T07:21:40.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:39 smithi071 ceph-mon[37727]: osdmap e240: 8 total, 8 up, 8 in 2024-09-11T07:21:41.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:40 smithi196 ceph-mon[30773]: pgmap v330: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:21:41.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:40 smithi196 ceph-mon[30773]: osdmap e241: 8 total, 8 up, 8 in 2024-09-11T07:21:41.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:40 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/127810402' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:21:41.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:40 smithi071 ceph-mon[27013]: pgmap v330: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:21:41.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:40 smithi071 ceph-mon[27013]: osdmap e241: 8 total, 8 up, 8 in 2024-09-11T07:21:41.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:40 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/127810402' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:21:41.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:40 smithi071 ceph-mon[37727]: pgmap v330: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:21:41.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:40 smithi071 ceph-mon[37727]: osdmap e241: 8 total, 8 up, 8 in 2024-09-11T07:21:41.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:40 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/127810402' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:21:41.754 INFO:tasks.workunit.client.0.smithi071.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_operate_aio_write_op PASSED [ 65%] 2024-09-11T07:21:42.034 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:41 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/127810402' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:21:42.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:41 smithi196 ceph-mon[30773]: osdmap e242: 8 total, 8 up, 8 in 2024-09-11T07:21:42.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:41 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/127810402' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:21:42.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:41 smithi071 ceph-mon[27013]: osdmap e242: 8 total, 8 up, 8 in 2024-09-11T07:21:42.089 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:41 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/127810402' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:21:42.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:41 smithi071 ceph-mon[37727]: osdmap e242: 8 total, 8 up, 8 in 2024-09-11T07:21:43.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:42 smithi196 ceph-mon[30773]: pgmap v333: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:21:43.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:42 smithi196 ceph-mon[30773]: osdmap e243: 8 total, 8 up, 8 in 2024-09-11T07:21:43.089 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:42 smithi071 ceph-mon[37727]: pgmap v333: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:21:43.089 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:42 smithi071 ceph-mon[37727]: osdmap e243: 8 total, 8 up, 8 in 2024-09-11T07:21:43.090 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:42 smithi071 ceph-mon[27013]: pgmap v333: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:21:43.090 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:42 smithi071 ceph-mon[27013]: osdmap e243: 8 total, 8 up, 8 in 2024-09-11T07:21:44.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:43 smithi196 ceph-mon[30773]: osdmap e244: 8 total, 8 up, 8 in 2024-09-11T07:21:44.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:43 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:21:44.089 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:43 smithi071 ceph-mon[37727]: osdmap e244: 8 total, 8 up, 8 in 2024-09-11T07:21:44.089 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:43 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:21:44.090 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:43 smithi071 ceph-mon[27013]: osdmap e244: 8 total, 8 up, 8 in 2024-09-11T07:21:44.090 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:43 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:21:45.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:44 smithi196 ceph-mon[30773]: pgmap v336: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:21:45.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:44 smithi196 ceph-mon[30773]: osdmap e245: 8 total, 8 up, 8 in 2024-09-11T07:21:45.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:44 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/3085062701' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:21:45.036 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:44 smithi196 ceph-mon[30773]: from='client.35003 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:21:45.036 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:44 smithi196 ceph-mon[30773]: from='client.35003 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:21:45.036 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:44 smithi196 ceph-mon[30773]: osdmap e246: 8 total, 8 up, 8 in 2024-09-11T07:21:45.036 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 11 07:21:44 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-iscsi-iscsi-a[58197]: debug there is no tcmu-runner data available 2024-09-11T07:21:45.089 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:44 smithi071 ceph-mon[37727]: pgmap v336: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:21:45.089 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:44 smithi071 ceph-mon[37727]: osdmap e245: 8 total, 8 up, 8 in 2024-09-11T07:21:45.089 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:44 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/3085062701' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:21:45.089 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:44 smithi071 ceph-mon[37727]: from='client.35003 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:21:45.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:44 smithi071 ceph-mon[37727]: from='client.35003 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:21:45.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:44 smithi071 ceph-mon[37727]: osdmap e246: 8 total, 8 up, 8 in 2024-09-11T07:21:45.090 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:44 smithi071 ceph-mon[27013]: pgmap v336: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:21:45.090 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:44 smithi071 ceph-mon[27013]: osdmap e245: 8 total, 8 up, 8 in 2024-09-11T07:21:45.090 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:44 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/3085062701' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:21:45.090 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:44 smithi071 ceph-mon[27013]: from='client.35003 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:21:45.090 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:44 smithi071 ceph-mon[27013]: from='client.35003 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:21:45.090 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:44 smithi071 ceph-mon[27013]: osdmap e246: 8 total, 8 up, 8 in 2024-09-11T07:21:45.759 INFO:tasks.workunit.client.0.smithi071.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_aio_write PASSED [ 66%] 2024-09-11T07:21:46.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:46 smithi196 ceph-mon[30773]: from='client.34442 v1:172.21.15.196:0/1952921461' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:21:46.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:46 smithi196 ceph-mon[30773]: osdmap e247: 8 total, 8 up, 8 in 2024-09-11T07:21:46.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:46 smithi071 ceph-mon[27013]: from='client.34442 v1:172.21.15.196:0/1952921461' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:21:46.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:46 smithi071 ceph-mon[27013]: osdmap e247: 8 total, 8 up, 8 in 2024-09-11T07:21:46.340 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:21:46 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: ::ffff:172.21.15.196 - - [11/Sep/2024:07:21:46] "GET /metrics HTTP/1.1" 200 225072 "" "Prometheus/2.43.0" 2024-09-11T07:21:46.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:46 smithi071 ceph-mon[37727]: from='client.34442 v1:172.21.15.196:0/1952921461' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:21:46.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:46 smithi071 ceph-mon[37727]: osdmap e247: 8 total, 8 up, 8 in 2024-09-11T07:21:47.019 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 07:21:46 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-x[32609]: ::ffff:172.21.15.196 - - [11/Sep/2024:07:21:46] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:21:47.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:47 smithi196 ceph-mon[30773]: pgmap v339: 196 pgs: 196 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T07:21:47.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:47 smithi196 ceph-mon[30773]: osdmap e248: 8 total, 8 up, 8 in 2024-09-11T07:21:47.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:47 smithi071 ceph-mon[27013]: pgmap v339: 196 pgs: 196 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T07:21:47.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:47 smithi071 ceph-mon[27013]: osdmap e248: 8 total, 8 up, 8 in 2024-09-11T07:21:47.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:47 smithi071 ceph-mon[37727]: pgmap v339: 196 pgs: 196 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-11T07:21:47.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:47 smithi071 ceph-mon[37727]: osdmap e248: 8 total, 8 up, 8 in 2024-09-11T07:21:49.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:48 smithi196 ceph-mon[30773]: pgmap v342: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:21:49.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:48 smithi196 ceph-mon[30773]: osdmap e249: 8 total, 8 up, 8 in 2024-09-11T07:21:49.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:48 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/2416398261' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:21:49.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:48 smithi071 ceph-mon[27013]: pgmap v342: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:21:49.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:48 smithi071 ceph-mon[27013]: osdmap e249: 8 total, 8 up, 8 in 2024-09-11T07:21:49.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:48 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/2416398261' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:21:49.089 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:48 smithi071 ceph-mon[37727]: pgmap v342: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:21:49.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:48 smithi071 ceph-mon[37727]: osdmap e249: 8 total, 8 up, 8 in 2024-09-11T07:21:49.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:48 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/2416398261' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:21:49.809 INFO:tasks.workunit.client.0.smithi071.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_aio_cmpext PASSED [ 67%] 2024-09-11T07:21:50.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:49 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/2416398261' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:21:50.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:49 smithi071 ceph-mon[27013]: osdmap e250: 8 total, 8 up, 8 in 2024-09-11T07:21:50.089 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:49 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/2416398261' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:21:50.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:49 smithi071 ceph-mon[37727]: osdmap e250: 8 total, 8 up, 8 in 2024-09-11T07:21:50.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:49 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/2416398261' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:21:50.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:49 smithi196 ceph-mon[30773]: osdmap e250: 8 total, 8 up, 8 in 2024-09-11T07:21:51.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:50 smithi071 ceph-mon[27013]: pgmap v345: 196 pgs: 196 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:21:51.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:50 smithi071 ceph-mon[27013]: osdmap e251: 8 total, 8 up, 8 in 2024-09-11T07:21:51.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:50 smithi071 ceph-mon[37727]: pgmap v345: 196 pgs: 196 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:21:51.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:50 smithi071 ceph-mon[37727]: osdmap e251: 8 total, 8 up, 8 in 2024-09-11T07:21:51.284 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:50 smithi196 ceph-mon[30773]: pgmap v345: 196 pgs: 196 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:21:51.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:50 smithi196 ceph-mon[30773]: osdmap e251: 8 total, 8 up, 8 in 2024-09-11T07:21:52.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:51 smithi071 ceph-mon[27013]: osdmap e252: 8 total, 8 up, 8 in 2024-09-11T07:21:52.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:51 smithi071 ceph-mon[27013]: osdmap e253: 8 total, 8 up, 8 in 2024-09-11T07:21:52.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:51 smithi071 ceph-mon[37727]: osdmap e252: 8 total, 8 up, 8 in 2024-09-11T07:21:52.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:51 smithi071 ceph-mon[37727]: osdmap e253: 8 total, 8 up, 8 in 2024-09-11T07:21:52.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:51 smithi196 ceph-mon[30773]: osdmap e252: 8 total, 8 up, 8 in 2024-09-11T07:21:52.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:51 smithi196 ceph-mon[30773]: osdmap e253: 8 total, 8 up, 8 in 2024-09-11T07:21:53.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:52 smithi071 ceph-mon[27013]: pgmap v348: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:21:53.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:52 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/2571258945' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:21:53.090 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:52 smithi071 ceph-mon[27013]: from='client.35015 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:21:53.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:52 smithi071 ceph-mon[37727]: pgmap v348: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:21:53.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:52 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/2571258945' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:21:53.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:52 smithi071 ceph-mon[37727]: from='client.35015 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:21:53.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:52 smithi196 ceph-mon[30773]: pgmap v348: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:21:53.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:52 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/2571258945' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:21:53.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:52 smithi196 ceph-mon[30773]: from='client.35015 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:21:53.827 INFO:tasks.workunit.client.0.smithi071.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_aio_rmxattr PASSED [ 68%] 2024-09-11T07:21:54.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:53 smithi071 ceph-mon[27013]: from='client.35015 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:21:54.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:53 smithi071 ceph-mon[27013]: osdmap e254: 8 total, 8 up, 8 in 2024-09-11T07:21:54.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:53 smithi071 ceph-mon[27013]: osdmap e255: 8 total, 8 up, 8 in 2024-09-11T07:21:54.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:53 smithi071 ceph-mon[37727]: from='client.35015 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:21:54.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:53 smithi071 ceph-mon[37727]: osdmap e254: 8 total, 8 up, 8 in 2024-09-11T07:21:54.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:53 smithi071 ceph-mon[37727]: osdmap e255: 8 total, 8 up, 8 in 2024-09-11T07:21:54.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:53 smithi196 ceph-mon[30773]: from='client.35015 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:21:54.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:53 smithi196 ceph-mon[30773]: osdmap e254: 8 total, 8 up, 8 in 2024-09-11T07:21:54.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:53 smithi196 ceph-mon[30773]: osdmap e255: 8 total, 8 up, 8 in 2024-09-11T07:21:55.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:54 smithi071 ceph-mon[27013]: pgmap v351: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:21:55.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:54 smithi071 ceph-mon[37727]: pgmap v351: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:21:55.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:54 smithi196 ceph-mon[30773]: pgmap v351: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:21:55.285 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 11 07:21:54 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-iscsi-iscsi-a[58197]: debug there is no tcmu-runner data available 2024-09-11T07:21:56.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:55 smithi071 ceph-mon[27013]: from='client.34442 v1:172.21.15.196:0/1952921461' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:21:56.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:55 smithi071 ceph-mon[27013]: osdmap e256: 8 total, 8 up, 8 in 2024-09-11T07:21:56.161 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:55 smithi071 ceph-mon[37727]: from='client.34442 v1:172.21.15.196:0/1952921461' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:21:56.161 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:55 smithi071 ceph-mon[37727]: osdmap e256: 8 total, 8 up, 8 in 2024-09-11T07:21:56.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:55 smithi196 ceph-mon[30773]: from='client.34442 v1:172.21.15.196:0/1952921461' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:21:56.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:55 smithi196 ceph-mon[30773]: osdmap e256: 8 total, 8 up, 8 in 2024-09-11T07:21:56.589 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:21:56 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: ::ffff:172.21.15.196 - - [11/Sep/2024:07:21:56] "GET /metrics HTTP/1.1" 200 225072 "" "Prometheus/2.43.0" 2024-09-11T07:21:56.847 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 07:21:56 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-x[32609]: ::ffff:172.21.15.196 - - [11/Sep/2024:07:21:56] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:21:57.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:56 smithi196 ceph-mon[30773]: pgmap v354: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:21:57.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:56 smithi196 ceph-mon[30773]: osdmap e257: 8 total, 8 up, 8 in 2024-09-11T07:21:57.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:56 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/3477568867' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:21:57.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:56 smithi196 ceph-mon[30773]: from='client.35021 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:21:57.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:56 smithi071 ceph-mon[27013]: pgmap v354: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:21:57.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:56 smithi071 ceph-mon[27013]: osdmap e257: 8 total, 8 up, 8 in 2024-09-11T07:21:57.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:56 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/3477568867' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:21:57.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:56 smithi071 ceph-mon[27013]: from='client.35021 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:21:57.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:56 smithi071 ceph-mon[37727]: pgmap v354: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:21:57.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:56 smithi071 ceph-mon[37727]: osdmap e257: 8 total, 8 up, 8 in 2024-09-11T07:21:57.341 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:56 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/3477568867' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:21:57.341 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:56 smithi071 ceph-mon[37727]: from='client.35021 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:21:57.869 INFO:tasks.workunit.client.0.smithi071.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_aio_write_no_comp_ref PASSED [ 69%] 2024-09-11T07:21:58.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:57 smithi196 ceph-mon[30773]: from='client.35021 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:21:58.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:57 smithi196 ceph-mon[30773]: osdmap e258: 8 total, 8 up, 8 in 2024-09-11T07:21:58.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:57 smithi071 ceph-mon[27013]: from='client.35021 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:21:58.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:57 smithi071 ceph-mon[27013]: osdmap e258: 8 total, 8 up, 8 in 2024-09-11T07:21:58.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:57 smithi071 ceph-mon[37727]: from='client.35021 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:21:58.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:57 smithi071 ceph-mon[37727]: osdmap e258: 8 total, 8 up, 8 in 2024-09-11T07:21:59.241 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:58 smithi071 ceph-mon[27013]: pgmap v357: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:21:59.241 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:58 smithi071 ceph-mon[27013]: osdmap e259: 8 total, 8 up, 8 in 2024-09-11T07:21:59.241 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:58 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:21:59.241 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:58 smithi071 ceph-mon[37727]: pgmap v357: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:21:59.241 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:58 smithi071 ceph-mon[37727]: osdmap e259: 8 total, 8 up, 8 in 2024-09-11T07:21:59.241 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:58 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:21:59.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:58 smithi196 ceph-mon[30773]: pgmap v357: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:21:59.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:58 smithi196 ceph-mon[30773]: osdmap e259: 8 total, 8 up, 8 in 2024-09-11T07:21:59.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:58 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:22:00.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:21:59 smithi196 ceph-mon[30773]: osdmap e260: 8 total, 8 up, 8 in 2024-09-11T07:22:00.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:21:59 smithi071 ceph-mon[27013]: osdmap e260: 8 total, 8 up, 8 in 2024-09-11T07:22:00.339 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:21:59 smithi071 ceph-mon[37727]: osdmap e260: 8 total, 8 up, 8 in 2024-09-11T07:22:01.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:00 smithi196 ceph-mon[30773]: pgmap v360: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:22:01.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:00 smithi196 ceph-mon[30773]: osdmap e261: 8 total, 8 up, 8 in 2024-09-11T07:22:01.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:00 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/502148788' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:22:01.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:00 smithi071 ceph-mon[27013]: pgmap v360: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:22:01.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:00 smithi071 ceph-mon[27013]: osdmap e261: 8 total, 8 up, 8 in 2024-09-11T07:22:01.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:00 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/502148788' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:22:01.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:00 smithi071 ceph-mon[37727]: pgmap v360: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:22:01.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:00 smithi071 ceph-mon[37727]: osdmap e261: 8 total, 8 up, 8 in 2024-09-11T07:22:01.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:00 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/502148788' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:22:01.891 INFO:tasks.workunit.client.0.smithi071.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_aio_append PASSED [ 70%] 2024-09-11T07:22:02.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:01 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/502148788' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:22:02.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:01 smithi196 ceph-mon[30773]: osdmap e262: 8 total, 8 up, 8 in 2024-09-11T07:22:02.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:01 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/502148788' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:22:02.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:01 smithi071 ceph-mon[27013]: osdmap e262: 8 total, 8 up, 8 in 2024-09-11T07:22:02.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:01 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/502148788' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:22:02.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:01 smithi071 ceph-mon[37727]: osdmap e262: 8 total, 8 up, 8 in 2024-09-11T07:22:03.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:02 smithi196 ceph-mon[30773]: pgmap v363: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:22:03.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:02 smithi196 ceph-mon[30773]: osdmap e263: 8 total, 8 up, 8 in 2024-09-11T07:22:03.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:02 smithi071 ceph-mon[27013]: pgmap v363: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:22:03.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:02 smithi071 ceph-mon[27013]: osdmap e263: 8 total, 8 up, 8 in 2024-09-11T07:22:03.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:02 smithi071 ceph-mon[37727]: pgmap v363: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:22:03.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:02 smithi071 ceph-mon[37727]: osdmap e263: 8 total, 8 up, 8 in 2024-09-11T07:22:04.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:03 smithi196 ceph-mon[30773]: osdmap e264: 8 total, 8 up, 8 in 2024-09-11T07:22:04.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:03 smithi196 ceph-mon[30773]: osdmap e265: 8 total, 8 up, 8 in 2024-09-11T07:22:04.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:03 smithi071 ceph-mon[27013]: osdmap e264: 8 total, 8 up, 8 in 2024-09-11T07:22:04.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:03 smithi071 ceph-mon[27013]: osdmap e265: 8 total, 8 up, 8 in 2024-09-11T07:22:04.339 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:03 smithi071 ceph-mon[37727]: osdmap e264: 8 total, 8 up, 8 in 2024-09-11T07:22:04.339 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:03 smithi071 ceph-mon[37727]: osdmap e265: 8 total, 8 up, 8 in 2024-09-11T07:22:05.285 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 11 07:22:04 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-iscsi-iscsi-a[58197]: debug there is no tcmu-runner data available 2024-09-11T07:22:05.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:04 smithi196 ceph-mon[30773]: pgmap v366: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:22:05.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:04 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/3698743431' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:22:05.286 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:04 smithi196 ceph-mon[30773]: from='client.35033 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:22:05.286 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:04 smithi196 ceph-mon[30773]: from='client.35033 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:22:05.286 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:04 smithi196 ceph-mon[30773]: osdmap e266: 8 total, 8 up, 8 in 2024-09-11T07:22:05.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:04 smithi071 ceph-mon[27013]: pgmap v366: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:22:05.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:04 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/3698743431' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:22:05.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:04 smithi071 ceph-mon[27013]: from='client.35033 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:22:05.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:04 smithi071 ceph-mon[27013]: from='client.35033 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:22:05.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:04 smithi071 ceph-mon[27013]: osdmap e266: 8 total, 8 up, 8 in 2024-09-11T07:22:05.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:04 smithi071 ceph-mon[37727]: pgmap v366: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:22:05.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:04 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/3698743431' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:22:05.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:04 smithi071 ceph-mon[37727]: from='client.35033 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:22:05.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:04 smithi071 ceph-mon[37727]: from='client.35033 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:22:05.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:04 smithi071 ceph-mon[37727]: osdmap e266: 8 total, 8 up, 8 in 2024-09-11T07:22:05.912 INFO:tasks.workunit.client.0.smithi071.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_aio_write_full PASSED [ 71%] 2024-09-11T07:22:06.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:05 smithi196 ceph-mon[30773]: from='client.34442 v1:172.21.15.196:0/1952921461' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:22:06.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:05 smithi196 ceph-mon[30773]: osdmap e267: 8 total, 8 up, 8 in 2024-09-11T07:22:06.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:05 smithi071 ceph-mon[27013]: from='client.34442 v1:172.21.15.196:0/1952921461' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:22:06.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:05 smithi071 ceph-mon[27013]: osdmap e267: 8 total, 8 up, 8 in 2024-09-11T07:22:06.339 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:22:06 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: ::ffff:172.21.15.196 - - [11/Sep/2024:07:22:06] "GET /metrics HTTP/1.1" 200 222727 "" "Prometheus/2.43.0" 2024-09-11T07:22:06.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:05 smithi071 ceph-mon[37727]: from='client.34442 v1:172.21.15.196:0/1952921461' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:22:06.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:05 smithi071 ceph-mon[37727]: osdmap e267: 8 total, 8 up, 8 in 2024-09-11T07:22:06.918 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 07:22:06 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-x[32609]: ::ffff:172.21.15.196 - - [11/Sep/2024:07:22:06] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:22:07.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:06 smithi196 ceph-mon[30773]: pgmap v369: 196 pgs: 196 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-11T07:22:07.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:06 smithi071 ceph-mon[27013]: pgmap v369: 196 pgs: 196 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-11T07:22:07.339 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:06 smithi071 ceph-mon[37727]: pgmap v369: 196 pgs: 196 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-11T07:22:08.186 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:07 smithi196 ceph-mon[30773]: osdmap e268: 8 total, 8 up, 8 in 2024-09-11T07:22:08.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:07 smithi071 ceph-mon[27013]: osdmap e268: 8 total, 8 up, 8 in 2024-09-11T07:22:08.339 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:07 smithi071 ceph-mon[37727]: osdmap e268: 8 total, 8 up, 8 in 2024-09-11T07:22:09.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:08 smithi196 ceph-mon[30773]: pgmap v372: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:22:09.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:08 smithi196 ceph-mon[30773]: osdmap e269: 8 total, 8 up, 8 in 2024-09-11T07:22:09.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:08 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/2560947581' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:22:09.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:08 smithi196 ceph-mon[30773]: from='client.35039 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:22:09.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:08 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:22:09.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:08 smithi071 ceph-mon[27013]: pgmap v372: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:22:09.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:08 smithi071 ceph-mon[27013]: osdmap e269: 8 total, 8 up, 8 in 2024-09-11T07:22:09.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:08 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/2560947581' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:22:09.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:08 smithi071 ceph-mon[27013]: from='client.35039 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:22:09.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:08 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:22:09.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:08 smithi071 ceph-mon[37727]: pgmap v372: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:22:09.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:08 smithi071 ceph-mon[37727]: osdmap e269: 8 total, 8 up, 8 in 2024-09-11T07:22:09.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:08 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/2560947581' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:22:09.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:08 smithi071 ceph-mon[37727]: from='client.35039 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:22:09.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:08 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:22:09.943 INFO:tasks.workunit.client.0.smithi071.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_aio_writesame PASSED [ 73%] 2024-09-11T07:22:10.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:09 smithi196 ceph-mon[30773]: from='client.35039 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:22:10.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:09 smithi196 ceph-mon[30773]: osdmap e270: 8 total, 8 up, 8 in 2024-09-11T07:22:10.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:09 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:22:10.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:09 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:22:10.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:09 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:22:10.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:09 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:22:10.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:09 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:22:10.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:09 smithi071 ceph-mon[27013]: from='client.35039 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:22:10.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:09 smithi071 ceph-mon[27013]: osdmap e270: 8 total, 8 up, 8 in 2024-09-11T07:22:10.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:09 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:22:10.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:09 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:22:10.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:09 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:22:10.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:09 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:22:10.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:09 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:22:10.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:09 smithi071 ceph-mon[37727]: from='client.35039 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:22:10.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:09 smithi071 ceph-mon[37727]: osdmap e270: 8 total, 8 up, 8 in 2024-09-11T07:22:10.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:09 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:22:10.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:09 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:22:10.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:09 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:22:10.341 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:09 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:22:10.341 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:09 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:22:11.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:10 smithi196 ceph-mon[30773]: pgmap v375: 196 pgs: 196 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:22:11.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:10 smithi196 ceph-mon[30773]: osdmap e271: 8 total, 8 up, 8 in 2024-09-11T07:22:11.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:10 smithi071 ceph-mon[27013]: pgmap v375: 196 pgs: 196 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:22:11.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:10 smithi071 ceph-mon[27013]: osdmap e271: 8 total, 8 up, 8 in 2024-09-11T07:22:11.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:10 smithi071 ceph-mon[37727]: pgmap v375: 196 pgs: 196 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:22:11.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:10 smithi071 ceph-mon[37727]: osdmap e271: 8 total, 8 up, 8 in 2024-09-11T07:22:12.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:11 smithi196 ceph-mon[30773]: osdmap e272: 8 total, 8 up, 8 in 2024-09-11T07:22:12.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:11 smithi071 ceph-mon[27013]: osdmap e272: 8 total, 8 up, 8 in 2024-09-11T07:22:12.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:11 smithi071 ceph-mon[37727]: osdmap e272: 8 total, 8 up, 8 in 2024-09-11T07:22:13.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:12 smithi196 ceph-mon[30773]: pgmap v378: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:22:13.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:12 smithi196 ceph-mon[30773]: osdmap e273: 8 total, 8 up, 8 in 2024-09-11T07:22:13.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:12 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/4006440235' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:22:13.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:12 smithi196 ceph-mon[30773]: from='client.35045 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:22:13.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:12 smithi071 ceph-mon[27013]: pgmap v378: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:22:13.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:12 smithi071 ceph-mon[27013]: osdmap e273: 8 total, 8 up, 8 in 2024-09-11T07:22:13.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:12 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/4006440235' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:22:13.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:12 smithi071 ceph-mon[27013]: from='client.35045 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:22:13.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:12 smithi071 ceph-mon[37727]: pgmap v378: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:22:13.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:12 smithi071 ceph-mon[37727]: osdmap e273: 8 total, 8 up, 8 in 2024-09-11T07:22:13.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:12 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/4006440235' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:22:13.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:12 smithi071 ceph-mon[37727]: from='client.35045 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:22:13.979 INFO:tasks.workunit.client.0.smithi071.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_aio_stat PASSED [ 74%] 2024-09-11T07:22:14.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:13 smithi196 ceph-mon[30773]: from='client.35045 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:22:14.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:13 smithi196 ceph-mon[30773]: osdmap e274: 8 total, 8 up, 8 in 2024-09-11T07:22:14.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:13 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:22:14.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:13 smithi071 ceph-mon[27013]: from='client.35045 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:22:14.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:13 smithi071 ceph-mon[27013]: osdmap e274: 8 total, 8 up, 8 in 2024-09-11T07:22:14.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:13 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:22:14.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:13 smithi071 ceph-mon[37727]: from='client.35045 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:22:14.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:13 smithi071 ceph-mon[37727]: osdmap e274: 8 total, 8 up, 8 in 2024-09-11T07:22:14.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:13 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:22:15.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:14 smithi196 ceph-mon[30773]: pgmap v381: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:22:15.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:14 smithi196 ceph-mon[30773]: osdmap e275: 8 total, 8 up, 8 in 2024-09-11T07:22:15.285 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 11 07:22:14 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-iscsi-iscsi-a[58197]: debug there is no tcmu-runner data available 2024-09-11T07:22:15.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:14 smithi071 ceph-mon[27013]: pgmap v381: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:22:15.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:14 smithi071 ceph-mon[27013]: osdmap e275: 8 total, 8 up, 8 in 2024-09-11T07:22:15.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:14 smithi071 ceph-mon[37727]: pgmap v381: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:22:15.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:14 smithi071 ceph-mon[37727]: osdmap e275: 8 total, 8 up, 8 in 2024-09-11T07:22:16.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:15 smithi196 ceph-mon[30773]: from='client.34442 v1:172.21.15.196:0/1952921461' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:22:16.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:15 smithi196 ceph-mon[30773]: osdmap e276: 8 total, 8 up, 8 in 2024-09-11T07:22:16.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:15 smithi196 ceph-mon[30773]: osdmap e277: 8 total, 8 up, 8 in 2024-09-11T07:22:16.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:15 smithi071 ceph-mon[27013]: from='client.34442 v1:172.21.15.196:0/1952921461' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:22:16.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:15 smithi071 ceph-mon[27013]: osdmap e276: 8 total, 8 up, 8 in 2024-09-11T07:22:16.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:15 smithi071 ceph-mon[27013]: osdmap e277: 8 total, 8 up, 8 in 2024-09-11T07:22:16.340 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:22:16 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: ::ffff:172.21.15.196 - - [11/Sep/2024:07:22:16] "GET /metrics HTTP/1.1" 200 225081 "" "Prometheus/2.43.0" 2024-09-11T07:22:16.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:15 smithi071 ceph-mon[37727]: from='client.34442 v1:172.21.15.196:0/1952921461' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:22:16.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:15 smithi071 ceph-mon[37727]: osdmap e276: 8 total, 8 up, 8 in 2024-09-11T07:22:16.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:15 smithi071 ceph-mon[37727]: osdmap e277: 8 total, 8 up, 8 in 2024-09-11T07:22:17.003 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 07:22:16 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-x[32609]: ::ffff:172.21.15.196 - - [11/Sep/2024:07:22:16] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:22:17.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:17 smithi196 ceph-mon[30773]: pgmap v384: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:22:17.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:17 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/581636342' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:22:17.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:17 smithi071 ceph-mon[27013]: pgmap v384: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:22:17.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:17 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/581636342' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:22:17.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:17 smithi071 ceph-mon[37727]: pgmap v384: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:22:17.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:17 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/581636342' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:22:18.008 INFO:tasks.workunit.client.0.smithi071.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_aio_remove PASSED [ 75%] 2024-09-11T07:22:18.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:18 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/581636342' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:22:18.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:18 smithi196 ceph-mon[30773]: osdmap e278: 8 total, 8 up, 8 in 2024-09-11T07:22:18.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:18 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/581636342' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:22:18.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:18 smithi071 ceph-mon[27013]: osdmap e278: 8 total, 8 up, 8 in 2024-09-11T07:22:18.339 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:18 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/581636342' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:22:18.339 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:18 smithi071 ceph-mon[37727]: osdmap e278: 8 total, 8 up, 8 in 2024-09-11T07:22:19.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:19 smithi196 ceph-mon[30773]: pgmap v387: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:22:19.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:19 smithi196 ceph-mon[30773]: osdmap e279: 8 total, 8 up, 8 in 2024-09-11T07:22:19.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:19 smithi071 ceph-mon[27013]: pgmap v387: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:22:19.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:19 smithi071 ceph-mon[27013]: osdmap e279: 8 total, 8 up, 8 in 2024-09-11T07:22:19.339 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:19 smithi071 ceph-mon[37727]: pgmap v387: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:22:19.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:19 smithi071 ceph-mon[37727]: osdmap e279: 8 total, 8 up, 8 in 2024-09-11T07:22:20.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:20 smithi196 ceph-mon[30773]: osdmap e280: 8 total, 8 up, 8 in 2024-09-11T07:22:20.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:20 smithi071 ceph-mon[27013]: osdmap e280: 8 total, 8 up, 8 in 2024-09-11T07:22:20.339 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:20 smithi071 ceph-mon[37727]: osdmap e280: 8 total, 8 up, 8 in 2024-09-11T07:22:21.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:21 smithi196 ceph-mon[30773]: pgmap v390: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:22:21.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:21 smithi196 ceph-mon[30773]: osdmap e281: 8 total, 8 up, 8 in 2024-09-11T07:22:21.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:21 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/456413735' entity='client.admin' cmd=[{"prefix": "osd map", "pool": "test_pool", "object": "foo", "format": "json"}]: dispatch 2024-09-11T07:22:21.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:21 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/456413735' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noup"}]: dispatch 2024-09-11T07:22:21.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:21 smithi196 ceph-mon[30773]: from='client.35057 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noup"}]: dispatch 2024-09-11T07:22:21.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:21 smithi071 ceph-mon[27013]: pgmap v390: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:22:21.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:21 smithi071 ceph-mon[27013]: osdmap e281: 8 total, 8 up, 8 in 2024-09-11T07:22:21.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:21 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/456413735' entity='client.admin' cmd=[{"prefix": "osd map", "pool": "test_pool", "object": "foo", "format": "json"}]: dispatch 2024-09-11T07:22:21.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:21 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/456413735' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noup"}]: dispatch 2024-09-11T07:22:21.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:21 smithi071 ceph-mon[27013]: from='client.35057 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noup"}]: dispatch 2024-09-11T07:22:21.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:21 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mon-a[27009]: 2024-09-11T07:22:21.039+0000 7fea9125f640 -1 mon.a@0(leader).osd e282 definitely_dead 0 2024-09-11T07:22:21.341 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:21 smithi071 ceph-mon[37727]: pgmap v390: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:22:21.341 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:21 smithi071 ceph-mon[37727]: osdmap e281: 8 total, 8 up, 8 in 2024-09-11T07:22:21.341 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:21 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/456413735' entity='client.admin' cmd=[{"prefix": "osd map", "pool": "test_pool", "object": "foo", "format": "json"}]: dispatch 2024-09-11T07:22:21.341 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:21 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/456413735' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noup"}]: dispatch 2024-09-11T07:22:21.341 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:21 smithi071 ceph-mon[37727]: from='client.35057 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noup"}]: dispatch 2024-09-11T07:22:22.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:22 smithi196 ceph-mon[30773]: Health check failed: noup flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:22:22.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:22 smithi196 ceph-mon[30773]: from='client.35057 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noup"}]': finished 2024-09-11T07:22:22.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:22 smithi196 ceph-mon[30773]: osdmap e282: 8 total, 8 up, 8 in 2024-09-11T07:22:22.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:22 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/456413735' entity='client.admin' cmd=[{"prefix": "osd down", "ids": ["6", "1", "2"]}]: dispatch 2024-09-11T07:22:22.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:22 smithi196 ceph-mon[30773]: from='client.35057 ' entity='client.admin' cmd=[{"prefix": "osd down", "ids": ["6", "1", "2"]}]: dispatch 2024-09-11T07:22:22.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:22 smithi071 ceph-mon[27013]: Health check failed: noup flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:22:22.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:22 smithi071 ceph-mon[27013]: from='client.35057 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noup"}]': finished 2024-09-11T07:22:22.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:22 smithi071 ceph-mon[27013]: osdmap e282: 8 total, 8 up, 8 in 2024-09-11T07:22:22.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:22 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/456413735' entity='client.admin' cmd=[{"prefix": "osd down", "ids": ["6", "1", "2"]}]: dispatch 2024-09-11T07:22:22.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:22 smithi071 ceph-mon[27013]: from='client.35057 ' entity='client.admin' cmd=[{"prefix": "osd down", "ids": ["6", "1", "2"]}]: dispatch 2024-09-11T07:22:22.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:22 smithi071 ceph-mon[37727]: Health check failed: noup flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:22:22.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:22 smithi071 ceph-mon[37727]: from='client.35057 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noup"}]': finished 2024-09-11T07:22:22.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:22 smithi071 ceph-mon[37727]: osdmap e282: 8 total, 8 up, 8 in 2024-09-11T07:22:22.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:22 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/456413735' entity='client.admin' cmd=[{"prefix": "osd down", "ids": ["6", "1", "2"]}]: dispatch 2024-09-11T07:22:22.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:22 smithi071 ceph-mon[37727]: from='client.35057 ' entity='client.admin' cmd=[{"prefix": "osd down", "ids": ["6", "1", "2"]}]: dispatch 2024-09-11T07:22:23.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:23 smithi071 ceph-mon[27013]: pgmap v393: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:22:23.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:23 smithi071 ceph-mon[27013]: Health check failed: 3 osds down (OSD_DOWN) 2024-09-11T07:22:23.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:23 smithi071 ceph-mon[27013]: from='client.35057 ' entity='client.admin' cmd='[{"prefix": "osd down", "ids": ["6", "1", "2"]}]': finished 2024-09-11T07:22:23.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:23 smithi071 ceph-mon[27013]: osdmap e283: 8 total, 5 up, 8 in 2024-09-11T07:22:23.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:23 smithi071 ceph-mon[37727]: pgmap v393: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:22:23.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:23 smithi071 ceph-mon[37727]: Health check failed: 3 osds down (OSD_DOWN) 2024-09-11T07:22:23.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:23 smithi071 ceph-mon[37727]: from='client.35057 ' entity='client.admin' cmd='[{"prefix": "osd down", "ids": ["6", "1", "2"]}]': finished 2024-09-11T07:22:23.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:23 smithi071 ceph-mon[37727]: osdmap e283: 8 total, 5 up, 8 in 2024-09-11T07:22:23.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:23 smithi196 ceph-mon[30773]: pgmap v393: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:22:23.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:23 smithi196 ceph-mon[30773]: Health check failed: 3 osds down (OSD_DOWN) 2024-09-11T07:22:23.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:23 smithi196 ceph-mon[30773]: from='client.35057 ' entity='client.admin' cmd='[{"prefix": "osd down", "ids": ["6", "1", "2"]}]': finished 2024-09-11T07:22:23.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:23 smithi196 ceph-mon[30773]: osdmap e283: 8 total, 5 up, 8 in 2024-09-11T07:22:24.052 INFO:journalctl@ceph.osd.1.smithi071.stdout:Sep 11 07:22:23 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-1[47504]: 2024-09-11T07:22:23.728+0000 7fb83af4d640 -1 osd.1 284 osdmap NOUP flag is set, waiting for it to clear 2024-09-11T07:22:24.339 INFO:journalctl@ceph.osd.1.smithi071.stdout:Sep 11 07:22:24 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-1[47504]: 2024-09-11T07:22:24.056+0000 7fb82d566640 -1 osd.1 285 osdmap NOUP flag is set, waiting for it to clear 2024-09-11T07:22:24.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:24 smithi071 ceph-mon[27013]: osdmap e284: 8 total, 5 up, 8 in 2024-09-11T07:22:24.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:24 smithi071 ceph-mon[27013]: Monitor daemon marked osd.1 down, but it is still running 2024-09-11T07:22:24.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:24 smithi071 ceph-mon[27013]: map e284 wrongly marked me down at e283 2024-09-11T07:22:24.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:24 smithi071 ceph-mon[27013]: osd.1 marked itself dead as of e284 2024-09-11T07:22:24.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:24 smithi071 ceph-mon[37727]: osdmap e284: 8 total, 5 up, 8 in 2024-09-11T07:22:24.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:24 smithi071 ceph-mon[37727]: Monitor daemon marked osd.1 down, but it is still running 2024-09-11T07:22:24.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:24 smithi071 ceph-mon[37727]: map e284 wrongly marked me down at e283 2024-09-11T07:22:24.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:24 smithi071 ceph-mon[37727]: osd.1 marked itself dead as of e284 2024-09-11T07:22:24.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:24 smithi196 ceph-mon[30773]: osdmap e284: 8 total, 5 up, 8 in 2024-09-11T07:22:24.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:24 smithi196 ceph-mon[30773]: Monitor daemon marked osd.1 down, but it is still running 2024-09-11T07:22:24.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:24 smithi196 ceph-mon[30773]: map e284 wrongly marked me down at e283 2024-09-11T07:22:24.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:24 smithi196 ceph-mon[30773]: osd.1 marked itself dead as of e284 2024-09-11T07:22:25.261 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:25 smithi196 ceph-mon[30773]: pgmap v396: 196 pgs: 10 stale+creating+peering, 54 stale+active+clean, 22 creating+peering, 110 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:22:25.261 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:25 smithi196 ceph-mon[30773]: osdmap e285: 8 total, 5 up, 8 in 2024-09-11T07:22:25.261 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:25 smithi196 ceph-mon[30773]: osd.2 marked itself dead as of e285 2024-09-11T07:22:25.262 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:25 smithi196 ceph-mon[30773]: osd.6 marked itself dead as of e285 2024-09-11T07:22:25.262 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 11 07:22:24 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-iscsi-iscsi-a[58197]: debug there is no tcmu-runner data available 2024-09-11T07:22:25.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:25 smithi071 ceph-mon[27013]: pgmap v396: 196 pgs: 10 stale+creating+peering, 54 stale+active+clean, 22 creating+peering, 110 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:22:25.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:25 smithi071 ceph-mon[27013]: osdmap e285: 8 total, 5 up, 8 in 2024-09-11T07:22:25.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:25 smithi071 ceph-mon[27013]: osd.2 marked itself dead as of e285 2024-09-11T07:22:25.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:25 smithi071 ceph-mon[27013]: osd.6 marked itself dead as of e285 2024-09-11T07:22:25.340 INFO:journalctl@ceph.osd.1.smithi071.stdout:Sep 11 07:22:25 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-1[47504]: 2024-09-11T07:22:25.057+0000 7fb82d566640 -1 osd.1 286 osdmap NOUP flag is set, waiting for it to clear 2024-09-11T07:22:25.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:25 smithi071 ceph-mon[37727]: pgmap v396: 196 pgs: 10 stale+creating+peering, 54 stale+active+clean, 22 creating+peering, 110 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:22:25.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:25 smithi071 ceph-mon[37727]: osdmap e285: 8 total, 5 up, 8 in 2024-09-11T07:22:25.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:25 smithi071 ceph-mon[37727]: osd.2 marked itself dead as of e285 2024-09-11T07:22:25.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:25 smithi071 ceph-mon[37727]: osd.6 marked itself dead as of e285 2024-09-11T07:22:25.341 INFO:journalctl@ceph.osd.2.smithi071.stdout:Sep 11 07:22:25 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-2[53217]: 2024-09-11T07:22:25.153+0000 7fdcb492a640 -1 osd.2 286 osdmap NOUP flag is set, waiting for it to clear 2024-09-11T07:22:25.535 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 07:22:25 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-6[47249]: 2024-09-11T07:22:25.258+0000 7f4953431640 -1 osd.6 286 osdmap NOUP flag is set, waiting for it to clear 2024-09-11T07:22:26.339 INFO:journalctl@ceph.osd.1.smithi071.stdout:Sep 11 07:22:26 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-1[47504]: 2024-09-11T07:22:26.060+0000 7fb835d84640 -1 osd.1 287 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T07:22:26.340 INFO:journalctl@ceph.osd.2.smithi071.stdout:Sep 11 07:22:26 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-2[53217]: 2024-09-11T07:22:26.060+0000 7fdcb1765640 -1 osd.2 287 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T07:22:26.340 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:22:26 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: ::ffff:172.21.15.196 - - [11/Sep/2024:07:22:26] "GET /metrics HTTP/1.1" 200 225081 "" "Prometheus/2.43.0" 2024-09-11T07:22:26.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:26 smithi071 ceph-mon[37727]: from='client.34442 v1:172.21.15.196:0/1952921461' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:22:26.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:26 smithi071 ceph-mon[37727]: Monitor daemon marked osd.2 down, but it is still running 2024-09-11T07:22:26.341 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:26 smithi071 ceph-mon[37727]: map e285 wrongly marked me down at e283 2024-09-11T07:22:26.341 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:26 smithi071 ceph-mon[37727]: Monitor daemon marked osd.6 down, but it is still running 2024-09-11T07:22:26.341 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:26 smithi071 ceph-mon[37727]: map e285 wrongly marked me down at e283 2024-09-11T07:22:26.341 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:26 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/456413735' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:22:26.341 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:26 smithi071 ceph-mon[37727]: from='client.35057 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:22:26.341 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:26 smithi071 ceph-mon[37727]: osdmap e286: 8 total, 5 up, 8 in 2024-09-11T07:22:26.341 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:26 smithi071 ceph-mon[37727]: Health check cleared: OSDMAP_FLAGS (was: noup flag(s) set) 2024-09-11T07:22:26.341 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:26 smithi071 ceph-mon[37727]: from='client.35057 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:22:26.341 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:26 smithi071 ceph-mon[27013]: from='client.34442 v1:172.21.15.196:0/1952921461' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:22:26.342 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:26 smithi071 ceph-mon[27013]: Monitor daemon marked osd.2 down, but it is still running 2024-09-11T07:22:26.342 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:26 smithi071 ceph-mon[27013]: map e285 wrongly marked me down at e283 2024-09-11T07:22:26.342 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:26 smithi071 ceph-mon[27013]: Monitor daemon marked osd.6 down, but it is still running 2024-09-11T07:22:26.342 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:26 smithi071 ceph-mon[27013]: map e285 wrongly marked me down at e283 2024-09-11T07:22:26.342 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:26 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/456413735' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:22:26.342 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:26 smithi071 ceph-mon[27013]: from='client.35057 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:22:26.342 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:26 smithi071 ceph-mon[27013]: osdmap e286: 8 total, 5 up, 8 in 2024-09-11T07:22:26.342 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:26 smithi071 ceph-mon[27013]: Health check cleared: OSDMAP_FLAGS (was: noup flag(s) set) 2024-09-11T07:22:26.342 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:26 smithi071 ceph-mon[27013]: from='client.35057 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:22:26.535 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 07:22:26 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-6[47249]: 2024-09-11T07:22:26.058+0000 7f494da67640 -1 osd.6 287 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T07:22:26.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:26 smithi196 ceph-mon[30773]: from='client.34442 v1:172.21.15.196:0/1952921461' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:22:26.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:26 smithi196 ceph-mon[30773]: Monitor daemon marked osd.2 down, but it is still running 2024-09-11T07:22:26.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:26 smithi196 ceph-mon[30773]: map e285 wrongly marked me down at e283 2024-09-11T07:22:26.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:26 smithi196 ceph-mon[30773]: Monitor daemon marked osd.6 down, but it is still running 2024-09-11T07:22:26.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:26 smithi196 ceph-mon[30773]: map e285 wrongly marked me down at e283 2024-09-11T07:22:26.536 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:26 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/456413735' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:22:26.536 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:26 smithi196 ceph-mon[30773]: from='client.35057 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:22:26.536 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:26 smithi196 ceph-mon[30773]: osdmap e286: 8 total, 5 up, 8 in 2024-09-11T07:22:26.536 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:26 smithi196 ceph-mon[30773]: Health check cleared: OSDMAP_FLAGS (was: noup flag(s) set) 2024-09-11T07:22:26.536 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:26 smithi196 ceph-mon[30773]: from='client.35057 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:22:27.035 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 07:22:26 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-x[32609]: ::ffff:172.21.15.196 - - [11/Sep/2024:07:22:26] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:22:27.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:27 smithi071 ceph-mon[27013]: pgmap v399: 196 pgs: 18 active+undersized, 8 undersized+degraded+peered+wait, 20 active+undersized+degraded+wait, 3 stale+active+clean, 8 active+undersized+degraded, 18 undersized+peered+wait, 74 active+undersized+wait, 47 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 255 B/s wr, 0 op/s; 190/591 objects degraded (32.149%) 2024-09-11T07:22:27.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:27 smithi071 ceph-mon[27013]: osdmap e287: 8 total, 5 up, 8 in 2024-09-11T07:22:27.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:27 smithi071 ceph-mon[27013]: Health check failed: Reduced data availability: 5 pgs inactive (PG_AVAILABILITY) 2024-09-11T07:22:27.339 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:27 smithi071 ceph-mon[27013]: Health check failed: Degraded data redundancy: 190/591 objects degraded (32.149%), 36 pgs degraded (PG_DEGRADED) 2024-09-11T07:22:27.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:27 smithi071 ceph-mon[27013]: Health check cleared: OSD_DOWN (was: 3 osds down) 2024-09-11T07:22:27.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:27 smithi071 ceph-mon[27013]: osd.1 v1:172.21.15.71:6805/2000038894 boot 2024-09-11T07:22:27.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:27 smithi071 ceph-mon[27013]: osd.6 v1:172.21.15.196:6808/307456429 boot 2024-09-11T07:22:27.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:27 smithi071 ceph-mon[27013]: osd.2 v1:172.21.15.71:6809/3105396883 boot 2024-09-11T07:22:27.340 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:27 smithi071 ceph-mon[27013]: osdmap e288: 8 total, 8 up, 8 in 2024-09-11T07:22:27.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:27 smithi071 ceph-mon[37727]: pgmap v399: 196 pgs: 18 active+undersized, 8 undersized+degraded+peered+wait, 20 active+undersized+degraded+wait, 3 stale+active+clean, 8 active+undersized+degraded, 18 undersized+peered+wait, 74 active+undersized+wait, 47 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 255 B/s wr, 0 op/s; 190/591 objects degraded (32.149%) 2024-09-11T07:22:27.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:27 smithi071 ceph-mon[37727]: osdmap e287: 8 total, 5 up, 8 in 2024-09-11T07:22:27.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:27 smithi071 ceph-mon[37727]: Health check failed: Reduced data availability: 5 pgs inactive (PG_AVAILABILITY) 2024-09-11T07:22:27.340 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:27 smithi071 ceph-mon[37727]: Health check failed: Degraded data redundancy: 190/591 objects degraded (32.149%), 36 pgs degraded (PG_DEGRADED) 2024-09-11T07:22:27.341 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:27 smithi071 ceph-mon[37727]: Health check cleared: OSD_DOWN (was: 3 osds down) 2024-09-11T07:22:27.341 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:27 smithi071 ceph-mon[37727]: osd.1 v1:172.21.15.71:6805/2000038894 boot 2024-09-11T07:22:27.341 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:27 smithi071 ceph-mon[37727]: osd.6 v1:172.21.15.196:6808/307456429 boot 2024-09-11T07:22:27.341 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:27 smithi071 ceph-mon[37727]: osd.2 v1:172.21.15.71:6809/3105396883 boot 2024-09-11T07:22:27.341 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:27 smithi071 ceph-mon[37727]: osdmap e288: 8 total, 8 up, 8 in 2024-09-11T07:22:27.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:27 smithi196 ceph-mon[30773]: pgmap v399: 196 pgs: 18 active+undersized, 8 undersized+degraded+peered+wait, 20 active+undersized+degraded+wait, 3 stale+active+clean, 8 active+undersized+degraded, 18 undersized+peered+wait, 74 active+undersized+wait, 47 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 255 B/s wr, 0 op/s; 190/591 objects degraded (32.149%) 2024-09-11T07:22:27.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:27 smithi196 ceph-mon[30773]: osdmap e287: 8 total, 5 up, 8 in 2024-09-11T07:22:27.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:27 smithi196 ceph-mon[30773]: Health check failed: Reduced data availability: 5 pgs inactive (PG_AVAILABILITY) 2024-09-11T07:22:27.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:27 smithi196 ceph-mon[30773]: Health check failed: Degraded data redundancy: 190/591 objects degraded (32.149%), 36 pgs degraded (PG_DEGRADED) 2024-09-11T07:22:27.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:27 smithi196 ceph-mon[30773]: Health check cleared: OSD_DOWN (was: 3 osds down) 2024-09-11T07:22:27.536 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:27 smithi196 ceph-mon[30773]: osd.1 v1:172.21.15.71:6805/2000038894 boot 2024-09-11T07:22:27.536 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:27 smithi196 ceph-mon[30773]: osd.6 v1:172.21.15.196:6808/307456429 boot 2024-09-11T07:22:27.536 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:27 smithi196 ceph-mon[30773]: osd.2 v1:172.21.15.71:6809/3105396883 boot 2024-09-11T07:22:27.536 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:27 smithi196 ceph-mon[30773]: osdmap e288: 8 total, 8 up, 8 in 2024-09-11T07:22:28.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:28 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T07:22:28.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:28 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T07:22:28.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:28 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T07:22:28.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:28 smithi196 ceph-mon[30773]: osdmap e289: 8 total, 8 up, 8 in 2024-09-11T07:22:28.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:28 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T07:22:28.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:28 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T07:22:28.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:28 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T07:22:28.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:28 smithi071 ceph-mon[27013]: osdmap e289: 8 total, 8 up, 8 in 2024-09-11T07:22:28.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:28 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T07:22:28.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:28 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T07:22:28.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:28 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T07:22:28.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:28 smithi071 ceph-mon[37727]: osdmap e289: 8 total, 8 up, 8 in 2024-09-11T07:22:29.491 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:29 smithi071 ceph-mon[27013]: pgmap v402: 196 pgs: 18 active+undersized, 8 undersized+degraded+peered+wait, 20 active+undersized+degraded+wait, 19 stale+active+clean, 8 active+undersized+degraded, 18 undersized+peered+wait, 74 active+undersized+wait, 31 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 255 B/s wr, 0 op/s; 190/591 objects degraded (32.149%) 2024-09-11T07:22:29.491 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:29 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/456413735' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:22:29.491 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:29 smithi071 ceph-mon[27013]: from='client.35057 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:22:29.491 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:29 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:22:29.491 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:29 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:22:29.492 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:29 smithi071 ceph-mon[37727]: pgmap v402: 196 pgs: 18 active+undersized, 8 undersized+degraded+peered+wait, 20 active+undersized+degraded+wait, 19 stale+active+clean, 8 active+undersized+degraded, 18 undersized+peered+wait, 74 active+undersized+wait, 31 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 255 B/s wr, 0 op/s; 190/591 objects degraded (32.149%) 2024-09-11T07:22:29.492 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:29 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/456413735' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:22:29.492 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:29 smithi071 ceph-mon[37727]: from='client.35057 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:22:29.492 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:29 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:22:29.492 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:29 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:22:29.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:29 smithi196 ceph-mon[30773]: pgmap v402: 196 pgs: 18 active+undersized, 8 undersized+degraded+peered+wait, 20 active+undersized+degraded+wait, 19 stale+active+clean, 8 active+undersized+degraded, 18 undersized+peered+wait, 74 active+undersized+wait, 31 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 255 B/s wr, 0 op/s; 190/591 objects degraded (32.149%) 2024-09-11T07:22:29.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:29 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/456413735' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:22:29.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:29 smithi196 ceph-mon[30773]: from='client.35057 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:22:29.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:29 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:22:29.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:29 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:22:30.116 INFO:tasks.workunit.client.0.smithi071.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_aio_read_wait_for_complete PASSED [ 76%] 2024-09-11T07:22:30.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:30 smithi196 ceph-mon[30773]: from='client.35057 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:22:30.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:30 smithi196 ceph-mon[30773]: osdmap e290: 8 total, 8 up, 8 in 2024-09-11T07:22:30.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:30 smithi071 ceph-mon[27013]: from='client.35057 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:22:30.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:30 smithi071 ceph-mon[27013]: osdmap e290: 8 total, 8 up, 8 in 2024-09-11T07:22:30.589 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:30 smithi071 ceph-mon[37727]: from='client.35057 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:22:30.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:30 smithi071 ceph-mon[37727]: osdmap e290: 8 total, 8 up, 8 in 2024-09-11T07:22:31.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:31 smithi196 ceph-mon[30773]: pgmap v405: 196 pgs: 196 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-09-11T07:22:31.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:31 smithi196 ceph-mon[30773]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 5 pgs inactive) 2024-09-11T07:22:31.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:31 smithi196 ceph-mon[30773]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 190/591 objects degraded (32.149%), 36 pgs degraded) 2024-09-11T07:22:31.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:31 smithi196 ceph-mon[30773]: osdmap e291: 8 total, 8 up, 8 in 2024-09-11T07:22:31.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:31 smithi071 ceph-mon[27013]: pgmap v405: 196 pgs: 196 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-09-11T07:22:31.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:31 smithi071 ceph-mon[27013]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 5 pgs inactive) 2024-09-11T07:22:31.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:31 smithi071 ceph-mon[27013]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 190/591 objects degraded (32.149%), 36 pgs degraded) 2024-09-11T07:22:31.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:31 smithi071 ceph-mon[27013]: osdmap e291: 8 total, 8 up, 8 in 2024-09-11T07:22:31.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:31 smithi071 ceph-mon[37727]: pgmap v405: 196 pgs: 196 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-09-11T07:22:31.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:31 smithi071 ceph-mon[37727]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 5 pgs inactive) 2024-09-11T07:22:31.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:31 smithi071 ceph-mon[37727]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 190/591 objects degraded (32.149%), 36 pgs degraded) 2024-09-11T07:22:31.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:31 smithi071 ceph-mon[37727]: osdmap e291: 8 total, 8 up, 8 in 2024-09-11T07:22:32.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:32 smithi196 ceph-mon[30773]: osdmap e292: 8 total, 8 up, 8 in 2024-09-11T07:22:32.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:32 smithi071 ceph-mon[27013]: osdmap e292: 8 total, 8 up, 8 in 2024-09-11T07:22:32.589 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:32 smithi071 ceph-mon[37727]: osdmap e292: 8 total, 8 up, 8 in 2024-09-11T07:22:33.536 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:33 smithi196 ceph-mon[30773]: pgmap v408: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:22:33.537 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:33 smithi196 ceph-mon[30773]: osdmap e293: 8 total, 8 up, 8 in 2024-09-11T07:22:33.537 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:33 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/1714986802' entity='client.admin' cmd=[{"prefix": "osd map", "pool": "test_pool", "object": "foo", "format": "json"}]: dispatch 2024-09-11T07:22:33.537 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:33 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/1714986802' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noup"}]: dispatch 2024-09-11T07:22:33.589 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:33 smithi071 ceph-mon[37727]: pgmap v408: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:22:33.589 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:33 smithi071 ceph-mon[37727]: osdmap e293: 8 total, 8 up, 8 in 2024-09-11T07:22:33.589 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:33 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/1714986802' entity='client.admin' cmd=[{"prefix": "osd map", "pool": "test_pool", "object": "foo", "format": "json"}]: dispatch 2024-09-11T07:22:33.589 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:33 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/1714986802' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noup"}]: dispatch 2024-09-11T07:22:33.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:33 smithi071 ceph-mon[27013]: pgmap v408: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:22:33.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:33 smithi071 ceph-mon[27013]: osdmap e293: 8 total, 8 up, 8 in 2024-09-11T07:22:33.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:33 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/1714986802' entity='client.admin' cmd=[{"prefix": "osd map", "pool": "test_pool", "object": "foo", "format": "json"}]: dispatch 2024-09-11T07:22:33.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:33 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/1714986802' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noup"}]: dispatch 2024-09-11T07:22:33.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:33 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mon-a[27009]: 2024-09-11T07:22:33.129+0000 7fea9125f640 -1 mon.a@0(leader).osd e294 definitely_dead 0 2024-09-11T07:22:34.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:34 smithi196 ceph-mon[30773]: Health check failed: noup flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:22:34.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:34 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/1714986802' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noup"}]': finished 2024-09-11T07:22:34.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:34 smithi196 ceph-mon[30773]: osdmap e294: 8 total, 8 up, 8 in 2024-09-11T07:22:34.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:34 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/1714986802' entity='client.admin' cmd=[{"prefix": "osd down", "ids": ["5", "2", "4"]}]: dispatch 2024-09-11T07:22:34.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:34 smithi071 ceph-mon[27013]: Health check failed: noup flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:22:34.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:34 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/1714986802' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noup"}]': finished 2024-09-11T07:22:34.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:34 smithi071 ceph-mon[27013]: osdmap e294: 8 total, 8 up, 8 in 2024-09-11T07:22:34.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:34 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/1714986802' entity='client.admin' cmd=[{"prefix": "osd down", "ids": ["5", "2", "4"]}]: dispatch 2024-09-11T07:22:34.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:34 smithi071 ceph-mon[37727]: Health check failed: noup flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:22:34.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:34 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/1714986802' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noup"}]': finished 2024-09-11T07:22:34.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:34 smithi071 ceph-mon[37727]: osdmap e294: 8 total, 8 up, 8 in 2024-09-11T07:22:34.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:34 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/1714986802' entity='client.admin' cmd=[{"prefix": "osd down", "ids": ["5", "2", "4"]}]: dispatch 2024-09-11T07:22:35.127 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 11 07:22:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-iscsi-iscsi-a[58197]: debug there is no tcmu-runner data available 2024-09-11T07:22:35.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:35 smithi196 ceph-mon[30773]: pgmap v411: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:22:35.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:35 smithi196 ceph-mon[30773]: Health check failed: 3 osds down (OSD_DOWN) 2024-09-11T07:22:35.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:35 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/1714986802' entity='client.admin' cmd='[{"prefix": "osd down", "ids": ["5", "2", "4"]}]': finished 2024-09-11T07:22:35.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:35 smithi196 ceph-mon[30773]: osdmap e295: 8 total, 5 up, 8 in 2024-09-11T07:22:35.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:35 smithi071 ceph-mon[27013]: pgmap v411: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:22:35.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:35 smithi071 ceph-mon[27013]: Health check failed: 3 osds down (OSD_DOWN) 2024-09-11T07:22:35.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:35 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/1714986802' entity='client.admin' cmd='[{"prefix": "osd down", "ids": ["5", "2", "4"]}]': finished 2024-09-11T07:22:35.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:35 smithi071 ceph-mon[27013]: osdmap e295: 8 total, 5 up, 8 in 2024-09-11T07:22:35.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:35 smithi071 ceph-mon[37727]: pgmap v411: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:22:35.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:35 smithi071 ceph-mon[37727]: Health check failed: 3 osds down (OSD_DOWN) 2024-09-11T07:22:35.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:35 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/1714986802' entity='client.admin' cmd='[{"prefix": "osd down", "ids": ["5", "2", "4"]}]': finished 2024-09-11T07:22:35.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:35 smithi071 ceph-mon[37727]: osdmap e295: 8 total, 5 up, 8 in 2024-09-11T07:22:36.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:36 smithi196 ceph-mon[30773]: from='client.34442 v1:172.21.15.196:0/1952921461' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:22:36.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:36 smithi196 ceph-mon[30773]: osdmap e296: 8 total, 5 up, 8 in 2024-09-11T07:22:36.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:36 smithi196 ceph-mon[30773]: Monitor daemon marked osd.5 down, but it is still running 2024-09-11T07:22:36.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:36 smithi196 ceph-mon[30773]: map e296 wrongly marked me down at e295 2024-09-11T07:22:36.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:36 smithi196 ceph-mon[30773]: osd.5 marked itself dead as of e296 2024-09-11T07:22:36.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:36 smithi196 ceph-mon[30773]: osd.2 marked itself dead as of e296 2024-09-11T07:22:36.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:36 smithi196 ceph-mon[30773]: osd.4 marked itself dead as of e296 2024-09-11T07:22:36.589 INFO:journalctl@ceph.osd.2.smithi071.stdout:Sep 11 07:22:36 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-2[53217]: 2024-09-11T07:22:36.204+0000 7fdcb614f640 -1 osd.2 297 osdmap NOUP flag is set, waiting for it to clear 2024-09-11T07:22:36.589 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:22:36 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: ::ffff:172.21.15.196 - - [11/Sep/2024:07:22:36] "GET /metrics HTTP/1.1" 200 225285 "" "Prometheus/2.43.0" 2024-09-11T07:22:36.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:36 smithi071 ceph-mon[27013]: from='client.34442 v1:172.21.15.196:0/1952921461' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:22:36.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:36 smithi071 ceph-mon[27013]: osdmap e296: 8 total, 5 up, 8 in 2024-09-11T07:22:36.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:36 smithi071 ceph-mon[27013]: Monitor daemon marked osd.5 down, but it is still running 2024-09-11T07:22:36.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:36 smithi071 ceph-mon[27013]: map e296 wrongly marked me down at e295 2024-09-11T07:22:36.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:36 smithi071 ceph-mon[27013]: osd.5 marked itself dead as of e296 2024-09-11T07:22:36.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:36 smithi071 ceph-mon[27013]: osd.2 marked itself dead as of e296 2024-09-11T07:22:36.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:36 smithi071 ceph-mon[27013]: osd.4 marked itself dead as of e296 2024-09-11T07:22:36.591 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:36 smithi071 ceph-mon[37727]: from='client.34442 v1:172.21.15.196:0/1952921461' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:22:36.591 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:36 smithi071 ceph-mon[37727]: osdmap e296: 8 total, 5 up, 8 in 2024-09-11T07:22:36.591 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:36 smithi071 ceph-mon[37727]: Monitor daemon marked osd.5 down, but it is still running 2024-09-11T07:22:36.591 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:36 smithi071 ceph-mon[37727]: map e296 wrongly marked me down at e295 2024-09-11T07:22:36.591 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:36 smithi071 ceph-mon[37727]: osd.5 marked itself dead as of e296 2024-09-11T07:22:36.591 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:36 smithi071 ceph-mon[37727]: osd.2 marked itself dead as of e296 2024-09-11T07:22:36.591 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:36 smithi071 ceph-mon[37727]: osd.4 marked itself dead as of e296 2024-09-11T07:22:37.035 INFO:journalctl@ceph.osd.4.smithi196.stdout:Sep 11 07:22:36 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-4[35524]: 2024-09-11T07:22:36.554+0000 7f8cfe730640 -1 osd.4 297 osdmap NOUP flag is set, waiting for it to clear 2024-09-11T07:22:37.035 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 11 07:22:36 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-5[41502]: 2024-09-11T07:22:36.755+0000 7fc908be6640 -1 osd.5 297 osdmap NOUP flag is set, waiting for it to clear 2024-09-11T07:22:37.035 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 07:22:36 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-x[32609]: ::ffff:172.21.15.196 - - [11/Sep/2024:07:22:36] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:22:37.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:37 smithi196 ceph-mon[30773]: pgmap v414: 196 pgs: 122 peering, 74 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 255 B/s wr, 0 op/s 2024-09-11T07:22:37.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:37 smithi196 ceph-mon[30773]: Monitor daemon marked osd.2 down, but it is still running 2024-09-11T07:22:37.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:37 smithi196 ceph-mon[30773]: map e296 wrongly marked me down at e295 2024-09-11T07:22:37.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:37 smithi196 ceph-mon[30773]: Monitor daemon marked osd.4 down, but it is still running 2024-09-11T07:22:37.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:37 smithi196 ceph-mon[30773]: map e296 wrongly marked me down at e295 2024-09-11T07:22:37.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:37 smithi196 ceph-mon[30773]: Health check failed: Reduced data availability: 5 pgs inactive, 10 pgs peering (PG_AVAILABILITY) 2024-09-11T07:22:37.536 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:37 smithi196 ceph-mon[30773]: osdmap e297: 8 total, 5 up, 8 in 2024-09-11T07:22:37.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:37 smithi071 ceph-mon[27013]: pgmap v414: 196 pgs: 122 peering, 74 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 255 B/s wr, 0 op/s 2024-09-11T07:22:37.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:37 smithi071 ceph-mon[27013]: Monitor daemon marked osd.2 down, but it is still running 2024-09-11T07:22:37.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:37 smithi071 ceph-mon[27013]: map e296 wrongly marked me down at e295 2024-09-11T07:22:37.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:37 smithi071 ceph-mon[27013]: Monitor daemon marked osd.4 down, but it is still running 2024-09-11T07:22:37.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:37 smithi071 ceph-mon[27013]: map e296 wrongly marked me down at e295 2024-09-11T07:22:37.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:37 smithi071 ceph-mon[27013]: Health check failed: Reduced data availability: 5 pgs inactive, 10 pgs peering (PG_AVAILABILITY) 2024-09-11T07:22:37.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:37 smithi071 ceph-mon[27013]: osdmap e297: 8 total, 5 up, 8 in 2024-09-11T07:22:37.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:37 smithi071 ceph-mon[37727]: pgmap v414: 196 pgs: 122 peering, 74 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 255 B/s wr, 0 op/s 2024-09-11T07:22:37.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:37 smithi071 ceph-mon[37727]: Monitor daemon marked osd.2 down, but it is still running 2024-09-11T07:22:37.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:37 smithi071 ceph-mon[37727]: map e296 wrongly marked me down at e295 2024-09-11T07:22:37.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:37 smithi071 ceph-mon[37727]: Monitor daemon marked osd.4 down, but it is still running 2024-09-11T07:22:37.591 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:37 smithi071 ceph-mon[37727]: map e296 wrongly marked me down at e295 2024-09-11T07:22:37.591 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:37 smithi071 ceph-mon[37727]: Health check failed: Reduced data availability: 5 pgs inactive, 10 pgs peering (PG_AVAILABILITY) 2024-09-11T07:22:37.591 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:37 smithi071 ceph-mon[37727]: osdmap e297: 8 total, 5 up, 8 in 2024-09-11T07:22:38.437 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:38 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/1714986802' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:22:38.437 INFO:journalctl@ceph.osd.4.smithi196.stdout:Sep 11 07:22:38 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-4[35524]: 2024-09-11T07:22:38.139+0000 7f8cfa569640 -1 osd.4 298 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T07:22:38.437 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 11 07:22:38 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-5[41502]: 2024-09-11T07:22:38.139+0000 7fc9021f8640 -1 osd.5 298 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T07:22:38.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:38 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/1714986802' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:22:38.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:38 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/1714986802' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:22:38.590 INFO:journalctl@ceph.osd.2.smithi071.stdout:Sep 11 07:22:38 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-2[53217]: 2024-09-11T07:22:38.140+0000 7fdcb1765640 -1 osd.2 298 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T07:22:39.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:39 smithi196 ceph-mon[30773]: pgmap v416: 196 pgs: 36 stale+active+clean, 122 peering, 38 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 227 B/s wr, 0 op/s 2024-09-11T07:22:39.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:39 smithi196 ceph-mon[30773]: Health check cleared: OSDMAP_FLAGS (was: noup flag(s) set) 2024-09-11T07:22:39.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:39 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/1714986802' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:22:39.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:39 smithi196 ceph-mon[30773]: osdmap e298: 8 total, 5 up, 8 in 2024-09-11T07:22:39.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:39 smithi071 ceph-mon[27013]: pgmap v416: 196 pgs: 36 stale+active+clean, 122 peering, 38 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 227 B/s wr, 0 op/s 2024-09-11T07:22:39.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:39 smithi071 ceph-mon[27013]: Health check cleared: OSDMAP_FLAGS (was: noup flag(s) set) 2024-09-11T07:22:39.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:39 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/1714986802' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:22:39.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:39 smithi071 ceph-mon[27013]: osdmap e298: 8 total, 5 up, 8 in 2024-09-11T07:22:39.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:39 smithi071 ceph-mon[37727]: pgmap v416: 196 pgs: 36 stale+active+clean, 122 peering, 38 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 227 B/s wr, 0 op/s 2024-09-11T07:22:39.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:39 smithi071 ceph-mon[37727]: Health check cleared: OSDMAP_FLAGS (was: noup flag(s) set) 2024-09-11T07:22:39.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:39 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/1714986802' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:22:39.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:39 smithi071 ceph-mon[37727]: osdmap e298: 8 total, 5 up, 8 in 2024-09-11T07:22:40.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:40 smithi196 ceph-mon[30773]: Health check cleared: OSD_DOWN (was: 3 osds down) 2024-09-11T07:22:40.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:40 smithi196 ceph-mon[30773]: osd.5 v1:172.21.15.196:6804/3113061101 boot 2024-09-11T07:22:40.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:40 smithi196 ceph-mon[30773]: osd.4 v1:172.21.15.196:6800/4211374096 boot 2024-09-11T07:22:40.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:40 smithi196 ceph-mon[30773]: osd.2 v1:172.21.15.71:6809/3105396883 boot 2024-09-11T07:22:40.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:40 smithi196 ceph-mon[30773]: osdmap e299: 8 total, 8 up, 8 in 2024-09-11T07:22:40.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:40 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T07:22:40.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:40 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T07:22:40.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:40 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T07:22:40.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:40 smithi071 ceph-mon[27013]: Health check cleared: OSD_DOWN (was: 3 osds down) 2024-09-11T07:22:40.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:40 smithi071 ceph-mon[27013]: osd.5 v1:172.21.15.196:6804/3113061101 boot 2024-09-11T07:22:40.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:40 smithi071 ceph-mon[27013]: osd.4 v1:172.21.15.196:6800/4211374096 boot 2024-09-11T07:22:40.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:40 smithi071 ceph-mon[27013]: osd.2 v1:172.21.15.71:6809/3105396883 boot 2024-09-11T07:22:40.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:40 smithi071 ceph-mon[27013]: osdmap e299: 8 total, 8 up, 8 in 2024-09-11T07:22:40.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:40 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T07:22:40.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:40 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T07:22:40.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:40 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T07:22:40.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:40 smithi071 ceph-mon[37727]: Health check cleared: OSD_DOWN (was: 3 osds down) 2024-09-11T07:22:40.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:40 smithi071 ceph-mon[37727]: osd.5 v1:172.21.15.196:6804/3113061101 boot 2024-09-11T07:22:40.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:40 smithi071 ceph-mon[37727]: osd.4 v1:172.21.15.196:6800/4211374096 boot 2024-09-11T07:22:40.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:40 smithi071 ceph-mon[37727]: osd.2 v1:172.21.15.71:6809/3105396883 boot 2024-09-11T07:22:40.591 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:40 smithi071 ceph-mon[37727]: osdmap e299: 8 total, 8 up, 8 in 2024-09-11T07:22:40.591 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:40 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-11T07:22:40.591 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:40 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T07:22:40.591 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:40 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-11T07:22:41.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:41 smithi196 ceph-mon[30773]: pgmap v419: 196 pgs: 158 peering, 38 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.8 KiB/s rd, 1 op/s; 24 B/s, 0 objects/s recovering 2024-09-11T07:22:41.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:41 smithi196 ceph-mon[30773]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 5 pgs inactive, 10 pgs peering) 2024-09-11T07:22:41.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:41 smithi196 ceph-mon[30773]: osdmap e300: 8 total, 8 up, 8 in 2024-09-11T07:22:41.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:41 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/1714986802' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:22:41.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:41 smithi071 ceph-mon[27013]: pgmap v419: 196 pgs: 158 peering, 38 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.8 KiB/s rd, 1 op/s; 24 B/s, 0 objects/s recovering 2024-09-11T07:22:41.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:41 smithi071 ceph-mon[27013]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 5 pgs inactive, 10 pgs peering) 2024-09-11T07:22:41.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:41 smithi071 ceph-mon[27013]: osdmap e300: 8 total, 8 up, 8 in 2024-09-11T07:22:41.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:41 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/1714986802' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:22:41.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:41 smithi071 ceph-mon[37727]: pgmap v419: 196 pgs: 158 peering, 38 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.8 KiB/s rd, 1 op/s; 24 B/s, 0 objects/s recovering 2024-09-11T07:22:41.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:41 smithi071 ceph-mon[37727]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 5 pgs inactive, 10 pgs peering) 2024-09-11T07:22:41.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:41 smithi071 ceph-mon[37727]: osdmap e300: 8 total, 8 up, 8 in 2024-09-11T07:22:41.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:41 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/1714986802' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:22:42.167 INFO:tasks.workunit.client.0.smithi071.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_aio_read_wait_for_complete_and_cb PASSED [ 77%] 2024-09-11T07:22:42.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:42 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/1714986802' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:22:42.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:42 smithi196 ceph-mon[30773]: osdmap e301: 8 total, 8 up, 8 in 2024-09-11T07:22:42.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:42 smithi196 ceph-mon[30773]: pgmap v422: 196 pgs: 158 peering, 38 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 1 op/s; 27 B/s, 0 objects/s recovering 2024-09-11T07:22:42.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:42 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/1714986802' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:22:42.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:42 smithi071 ceph-mon[27013]: osdmap e301: 8 total, 8 up, 8 in 2024-09-11T07:22:42.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:42 smithi071 ceph-mon[27013]: pgmap v422: 196 pgs: 158 peering, 38 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 1 op/s; 27 B/s, 0 objects/s recovering 2024-09-11T07:22:42.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:42 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/1714986802' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:22:42.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:42 smithi071 ceph-mon[37727]: osdmap e301: 8 total, 8 up, 8 in 2024-09-11T07:22:42.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:42 smithi071 ceph-mon[37727]: pgmap v422: 196 pgs: 158 peering, 38 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 1 op/s; 27 B/s, 0 objects/s recovering 2024-09-11T07:22:43.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:43 smithi196 ceph-mon[30773]: osdmap e302: 8 total, 8 up, 8 in 2024-09-11T07:22:43.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:43 smithi071 ceph-mon[27013]: osdmap e302: 8 total, 8 up, 8 in 2024-09-11T07:22:43.589 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:43 smithi071 ceph-mon[37727]: osdmap e302: 8 total, 8 up, 8 in 2024-09-11T07:22:44.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:44 smithi196 ceph-mon[30773]: osdmap e303: 8 total, 8 up, 8 in 2024-09-11T07:22:44.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:44 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/667161110' entity='client.admin' cmd=[{"prefix": "osd map", "pool": "test_pool", "object": "bar", "format": "json"}]: dispatch 2024-09-11T07:22:44.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:44 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/667161110' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noup"}]: dispatch 2024-09-11T07:22:44.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:44 smithi196 ceph-mon[30773]: from='client.35069 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noup"}]: dispatch 2024-09-11T07:22:44.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:44 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:22:44.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:44 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:22:44.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:44 smithi196 ceph-mon[30773]: pgmap v425: 196 pgs: 32 unknown, 131 peering, 33 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:22:44.589 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:44 smithi071 ceph-mon[37727]: osdmap e303: 8 total, 8 up, 8 in 2024-09-11T07:22:44.589 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:44 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/667161110' entity='client.admin' cmd=[{"prefix": "osd map", "pool": "test_pool", "object": "bar", "format": "json"}]: dispatch 2024-09-11T07:22:44.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:44 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/667161110' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noup"}]: dispatch 2024-09-11T07:22:44.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:44 smithi071 ceph-mon[37727]: from='client.35069 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noup"}]: dispatch 2024-09-11T07:22:44.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:44 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:22:44.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:44 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:22:44.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:44 smithi071 ceph-mon[37727]: pgmap v425: 196 pgs: 32 unknown, 131 peering, 33 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:22:44.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:44 smithi071 ceph-mon[27013]: osdmap e303: 8 total, 8 up, 8 in 2024-09-11T07:22:44.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:44 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/667161110' entity='client.admin' cmd=[{"prefix": "osd map", "pool": "test_pool", "object": "bar", "format": "json"}]: dispatch 2024-09-11T07:22:44.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:44 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/667161110' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noup"}]: dispatch 2024-09-11T07:22:44.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:44 smithi071 ceph-mon[27013]: from='client.35069 ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noup"}]: dispatch 2024-09-11T07:22:44.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:44 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:22:44.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:44 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:22:44.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:44 smithi071 ceph-mon[27013]: pgmap v425: 196 pgs: 32 unknown, 131 peering, 33 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:22:44.591 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:44 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mon-a[27009]: 2024-09-11T07:22:44.188+0000 7fea9125f640 -1 mon.a@0(leader).osd e304 definitely_dead 0 2024-09-11T07:22:45.180 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 11 07:22:44 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-iscsi-iscsi-a[58197]: debug there is no tcmu-runner data available 2024-09-11T07:22:45.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:45 smithi196 ceph-mon[30773]: Health check failed: noup flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:22:45.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:45 smithi196 ceph-mon[30773]: from='client.35069 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noup"}]': finished 2024-09-11T07:22:45.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:45 smithi196 ceph-mon[30773]: osdmap e304: 8 total, 8 up, 8 in 2024-09-11T07:22:45.536 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:45 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/667161110' entity='client.admin' cmd=[{"prefix": "osd down", "ids": ["1", "6", "4"]}]: dispatch 2024-09-11T07:22:45.536 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:45 smithi196 ceph-mon[30773]: from='client.35069 ' entity='client.admin' cmd=[{"prefix": "osd down", "ids": ["1", "6", "4"]}]: dispatch 2024-09-11T07:22:45.536 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:45 smithi196 ceph-mon[30773]: from='client.34442 v1:172.21.15.196:0/1952921461' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:22:45.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:45 smithi071 ceph-mon[27013]: Health check failed: noup flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:22:45.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:45 smithi071 ceph-mon[27013]: from='client.35069 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noup"}]': finished 2024-09-11T07:22:45.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:45 smithi071 ceph-mon[27013]: osdmap e304: 8 total, 8 up, 8 in 2024-09-11T07:22:45.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:45 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/667161110' entity='client.admin' cmd=[{"prefix": "osd down", "ids": ["1", "6", "4"]}]: dispatch 2024-09-11T07:22:45.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:45 smithi071 ceph-mon[27013]: from='client.35069 ' entity='client.admin' cmd=[{"prefix": "osd down", "ids": ["1", "6", "4"]}]: dispatch 2024-09-11T07:22:45.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:45 smithi071 ceph-mon[27013]: from='client.34442 v1:172.21.15.196:0/1952921461' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:22:45.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:45 smithi071 ceph-mon[37727]: Health check failed: noup flag(s) set (OSDMAP_FLAGS) 2024-09-11T07:22:45.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:45 smithi071 ceph-mon[37727]: from='client.35069 ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noup"}]': finished 2024-09-11T07:22:45.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:45 smithi071 ceph-mon[37727]: osdmap e304: 8 total, 8 up, 8 in 2024-09-11T07:22:45.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:45 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/667161110' entity='client.admin' cmd=[{"prefix": "osd down", "ids": ["1", "6", "4"]}]: dispatch 2024-09-11T07:22:45.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:45 smithi071 ceph-mon[37727]: from='client.35069 ' entity='client.admin' cmd=[{"prefix": "osd down", "ids": ["1", "6", "4"]}]: dispatch 2024-09-11T07:22:45.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:45 smithi071 ceph-mon[37727]: from='client.34442 v1:172.21.15.196:0/1952921461' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:22:46.535 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 07:22:46 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-6[47249]: 2024-09-11T07:22:46.235+0000 7f4954c56640 -1 osd.6 306 osdmap NOUP flag is set, waiting for it to clear 2024-09-11T07:22:46.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:46 smithi196 ceph-mon[30773]: Health check failed: 3 osds down (OSD_DOWN) 2024-09-11T07:22:46.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:46 smithi196 ceph-mon[30773]: from='client.35069 ' entity='client.admin' cmd='[{"prefix": "osd down", "ids": ["1", "6", "4"]}]': finished 2024-09-11T07:22:46.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:46 smithi196 ceph-mon[30773]: osdmap e305: 8 total, 5 up, 8 in 2024-09-11T07:22:46.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:46 smithi196 ceph-mon[30773]: Monitor daemon marked osd.6 down, but it is still running 2024-09-11T07:22:46.536 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:46 smithi196 ceph-mon[30773]: map e305 wrongly marked me down at e305 2024-09-11T07:22:46.536 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:46 smithi196 ceph-mon[30773]: osd.6 marked itself dead as of e305 2024-09-11T07:22:46.536 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:46 smithi196 ceph-mon[30773]: pgmap v428: 196 pgs: 61 stale+active+clean, 135 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:22:46.589 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:22:46 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: ::ffff:172.21.15.196 - - [11/Sep/2024:07:22:46] "GET /metrics HTTP/1.1" 200 225230 "" "Prometheus/2.43.0" 2024-09-11T07:22:46.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:46 smithi071 ceph-mon[37727]: Health check failed: 3 osds down (OSD_DOWN) 2024-09-11T07:22:46.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:46 smithi071 ceph-mon[37727]: from='client.35069 ' entity='client.admin' cmd='[{"prefix": "osd down", "ids": ["1", "6", "4"]}]': finished 2024-09-11T07:22:46.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:46 smithi071 ceph-mon[37727]: osdmap e305: 8 total, 5 up, 8 in 2024-09-11T07:22:46.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:46 smithi071 ceph-mon[37727]: Monitor daemon marked osd.6 down, but it is still running 2024-09-11T07:22:46.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:46 smithi071 ceph-mon[37727]: map e305 wrongly marked me down at e305 2024-09-11T07:22:46.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:46 smithi071 ceph-mon[37727]: osd.6 marked itself dead as of e305 2024-09-11T07:22:46.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:46 smithi071 ceph-mon[37727]: pgmap v428: 196 pgs: 61 stale+active+clean, 135 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:22:46.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:46 smithi071 ceph-mon[27013]: Health check failed: 3 osds down (OSD_DOWN) 2024-09-11T07:22:46.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:46 smithi071 ceph-mon[27013]: from='client.35069 ' entity='client.admin' cmd='[{"prefix": "osd down", "ids": ["1", "6", "4"]}]': finished 2024-09-11T07:22:46.591 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:46 smithi071 ceph-mon[27013]: osdmap e305: 8 total, 5 up, 8 in 2024-09-11T07:22:46.591 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:46 smithi071 ceph-mon[27013]: Monitor daemon marked osd.6 down, but it is still running 2024-09-11T07:22:46.591 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:46 smithi071 ceph-mon[27013]: map e305 wrongly marked me down at e305 2024-09-11T07:22:46.591 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:46 smithi071 ceph-mon[27013]: osd.6 marked itself dead as of e305 2024-09-11T07:22:46.591 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:46 smithi071 ceph-mon[27013]: pgmap v428: 196 pgs: 61 stale+active+clean, 135 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:22:47.035 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 07:22:46 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-x[32609]: ::ffff:172.21.15.196 - - [11/Sep/2024:07:22:46] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:22:47.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:47 smithi196 ceph-mon[30773]: osdmap e306: 8 total, 5 up, 8 in 2024-09-11T07:22:47.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:47 smithi071 ceph-mon[27013]: osdmap e306: 8 total, 5 up, 8 in 2024-09-11T07:22:47.589 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:47 smithi071 ceph-mon[37727]: osdmap e306: 8 total, 5 up, 8 in 2024-09-11T07:22:48.035 INFO:journalctl@ceph.osd.4.smithi196.stdout:Sep 11 07:22:47 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-4[35524]: 2024-09-11T07:22:47.625+0000 7f8cfff55640 -1 osd.4 306 osdmap NOUP flag is set, waiting for it to clear 2024-09-11T07:22:48.089 INFO:journalctl@ceph.osd.1.smithi071.stdout:Sep 11 07:22:47 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-1[47504]: 2024-09-11T07:22:47.770+0000 7fb83c772640 -1 osd.1 306 osdmap NOUP flag is set, waiting for it to clear 2024-09-11T07:22:48.535 INFO:journalctl@ceph.osd.4.smithi196.stdout:Sep 11 07:22:48 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-4[35524]: 2024-09-11T07:22:48.202+0000 7f8cf1d4b640 -1 osd.4 307 osdmap NOUP flag is set, waiting for it to clear 2024-09-11T07:22:48.535 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 07:22:48 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-6[47249]: 2024-09-11T07:22:48.202+0000 7f4945249640 -1 osd.6 307 osdmap NOUP flag is set, waiting for it to clear 2024-09-11T07:22:48.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:48 smithi196 ceph-mon[30773]: osd.1 marked itself dead as of e306 2024-09-11T07:22:48.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:48 smithi196 ceph-mon[30773]: Monitor daemon marked osd.4 down, but it is still running 2024-09-11T07:22:48.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:48 smithi196 ceph-mon[30773]: map e306 wrongly marked me down at e305 2024-09-11T07:22:48.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:48 smithi196 ceph-mon[30773]: osd.4 marked itself dead as of e306 2024-09-11T07:22:48.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:48 smithi196 ceph-mon[30773]: pgmap v430: 196 pgs: 75 stale+active+clean, 121 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:22:48.589 INFO:journalctl@ceph.osd.1.smithi071.stdout:Sep 11 07:22:48 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-1[47504]: 2024-09-11T07:22:48.209+0000 7fb82d566640 -1 osd.1 307 osdmap NOUP flag is set, waiting for it to clear 2024-09-11T07:22:48.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:48 smithi071 ceph-mon[37727]: osd.1 marked itself dead as of e306 2024-09-11T07:22:48.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:48 smithi071 ceph-mon[37727]: Monitor daemon marked osd.4 down, but it is still running 2024-09-11T07:22:48.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:48 smithi071 ceph-mon[37727]: map e306 wrongly marked me down at e305 2024-09-11T07:22:48.591 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:48 smithi071 ceph-mon[37727]: osd.4 marked itself dead as of e306 2024-09-11T07:22:48.591 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:48 smithi071 ceph-mon[37727]: pgmap v430: 196 pgs: 75 stale+active+clean, 121 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:22:48.591 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:48 smithi071 ceph-mon[27013]: osd.1 marked itself dead as of e306 2024-09-11T07:22:48.591 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:48 smithi071 ceph-mon[27013]: Monitor daemon marked osd.4 down, but it is still running 2024-09-11T07:22:48.591 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:48 smithi071 ceph-mon[27013]: map e306 wrongly marked me down at e305 2024-09-11T07:22:48.592 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:48 smithi071 ceph-mon[27013]: osd.4 marked itself dead as of e306 2024-09-11T07:22:48.592 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:48 smithi071 ceph-mon[27013]: pgmap v430: 196 pgs: 75 stale+active+clean, 121 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:22:49.535 INFO:journalctl@ceph.osd.4.smithi196.stdout:Sep 11 07:22:49 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-4[35524]: 2024-09-11T07:22:49.208+0000 7f8cfa569640 -1 osd.4 308 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T07:22:49.535 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 07:22:49 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-6[47249]: 2024-09-11T07:22:49.206+0000 7f494da67640 -1 osd.6 308 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T07:22:49.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:49 smithi196 ceph-mon[30773]: Monitor daemon marked osd.1 down, but it is still running 2024-09-11T07:22:49.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:49 smithi196 ceph-mon[30773]: map e306 wrongly marked me down at e305 2024-09-11T07:22:49.536 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:49 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/667161110' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:22:49.536 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:49 smithi196 ceph-mon[30773]: from='client.35069 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:22:49.536 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:49 smithi196 ceph-mon[30773]: osdmap e307: 8 total, 5 up, 8 in 2024-09-11T07:22:49.589 INFO:journalctl@ceph.osd.1.smithi071.stdout:Sep 11 07:22:49 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-1[47504]: 2024-09-11T07:22:49.209+0000 7fb835d84640 -1 osd.1 308 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T07:22:49.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:49 smithi071 ceph-mon[27013]: Monitor daemon marked osd.1 down, but it is still running 2024-09-11T07:22:49.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:49 smithi071 ceph-mon[27013]: map e306 wrongly marked me down at e305 2024-09-11T07:22:49.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:49 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/667161110' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:22:49.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:49 smithi071 ceph-mon[27013]: from='client.35069 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:22:49.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:49 smithi071 ceph-mon[27013]: osdmap e307: 8 total, 5 up, 8 in 2024-09-11T07:22:49.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:49 smithi071 ceph-mon[37727]: Monitor daemon marked osd.1 down, but it is still running 2024-09-11T07:22:49.591 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:49 smithi071 ceph-mon[37727]: map e306 wrongly marked me down at e305 2024-09-11T07:22:49.591 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:49 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/667161110' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:22:49.591 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:49 smithi071 ceph-mon[37727]: from='client.35069 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:22:49.591 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:49 smithi071 ceph-mon[37727]: osdmap e307: 8 total, 5 up, 8 in 2024-09-11T07:22:50.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:50 smithi196 ceph-mon[30773]: Health check cleared: OSDMAP_FLAGS (was: noup flag(s) set) 2024-09-11T07:22:50.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:50 smithi196 ceph-mon[30773]: from='client.35069 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:22:50.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:50 smithi196 ceph-mon[30773]: osdmap e308: 8 total, 5 up, 8 in 2024-09-11T07:22:50.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:50 smithi196 ceph-mon[30773]: pgmap v433: 196 pgs: 31 undersized+peered, 78 active+undersized, 6 stale+active+clean, 18 undersized+degraded+peered, 19 active+undersized+degraded, 44 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 219/588 objects degraded (37.245%) 2024-09-11T07:22:50.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:50 smithi071 ceph-mon[27013]: Health check cleared: OSDMAP_FLAGS (was: noup flag(s) set) 2024-09-11T07:22:50.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:50 smithi071 ceph-mon[27013]: from='client.35069 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:22:50.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:50 smithi071 ceph-mon[27013]: osdmap e308: 8 total, 5 up, 8 in 2024-09-11T07:22:50.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:50 smithi071 ceph-mon[27013]: pgmap v433: 196 pgs: 31 undersized+peered, 78 active+undersized, 6 stale+active+clean, 18 undersized+degraded+peered, 19 active+undersized+degraded, 44 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 219/588 objects degraded (37.245%) 2024-09-11T07:22:50.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:50 smithi071 ceph-mon[37727]: Health check cleared: OSDMAP_FLAGS (was: noup flag(s) set) 2024-09-11T07:22:50.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:50 smithi071 ceph-mon[37727]: from='client.35069 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:22:50.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:50 smithi071 ceph-mon[37727]: osdmap e308: 8 total, 5 up, 8 in 2024-09-11T07:22:50.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:50 smithi071 ceph-mon[37727]: pgmap v433: 196 pgs: 31 undersized+peered, 78 active+undersized, 6 stale+active+clean, 18 undersized+degraded+peered, 19 active+undersized+degraded, 44 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 219/588 objects degraded (37.245%) 2024-09-11T07:22:51.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:51 smithi196 ceph-mon[30773]: Health check cleared: OSD_DOWN (was: 3 osds down) 2024-09-11T07:22:51.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:51 smithi196 ceph-mon[30773]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-09-11T07:22:51.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:51 smithi196 ceph-mon[30773]: Health check failed: Degraded data redundancy: 219/588 objects degraded (37.245%), 37 pgs degraded (PG_DEGRADED) 2024-09-11T07:22:51.536 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:51 smithi196 ceph-mon[30773]: osd.6 v1:172.21.15.196:6808/307456429 boot 2024-09-11T07:22:51.536 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:51 smithi196 ceph-mon[30773]: osd.4 v1:172.21.15.196:6800/4211374096 boot 2024-09-11T07:22:51.536 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:51 smithi196 ceph-mon[30773]: osd.1 v1:172.21.15.71:6805/2000038894 boot 2024-09-11T07:22:51.536 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:51 smithi196 ceph-mon[30773]: osdmap e309: 8 total, 8 up, 8 in 2024-09-11T07:22:51.536 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:51 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T07:22:51.536 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:51 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T07:22:51.536 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:51 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T07:22:51.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:51 smithi071 ceph-mon[27013]: Health check cleared: OSD_DOWN (was: 3 osds down) 2024-09-11T07:22:51.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:51 smithi071 ceph-mon[27013]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-09-11T07:22:51.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:51 smithi071 ceph-mon[27013]: Health check failed: Degraded data redundancy: 219/588 objects degraded (37.245%), 37 pgs degraded (PG_DEGRADED) 2024-09-11T07:22:51.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:51 smithi071 ceph-mon[27013]: osd.6 v1:172.21.15.196:6808/307456429 boot 2024-09-11T07:22:51.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:51 smithi071 ceph-mon[27013]: osd.4 v1:172.21.15.196:6800/4211374096 boot 2024-09-11T07:22:51.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:51 smithi071 ceph-mon[27013]: osd.1 v1:172.21.15.71:6805/2000038894 boot 2024-09-11T07:22:51.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:51 smithi071 ceph-mon[27013]: osdmap e309: 8 total, 8 up, 8 in 2024-09-11T07:22:51.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:51 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T07:22:51.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:51 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T07:22:51.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:51 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T07:22:51.591 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:51 smithi071 ceph-mon[37727]: Health check cleared: OSD_DOWN (was: 3 osds down) 2024-09-11T07:22:51.591 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:51 smithi071 ceph-mon[37727]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-09-11T07:22:51.591 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:51 smithi071 ceph-mon[37727]: Health check failed: Degraded data redundancy: 219/588 objects degraded (37.245%), 37 pgs degraded (PG_DEGRADED) 2024-09-11T07:22:51.591 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:51 smithi071 ceph-mon[37727]: osd.6 v1:172.21.15.196:6808/307456429 boot 2024-09-11T07:22:51.591 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:51 smithi071 ceph-mon[37727]: osd.4 v1:172.21.15.196:6800/4211374096 boot 2024-09-11T07:22:51.591 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:51 smithi071 ceph-mon[37727]: osd.1 v1:172.21.15.71:6805/2000038894 boot 2024-09-11T07:22:51.591 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:51 smithi071 ceph-mon[37727]: osdmap e309: 8 total, 8 up, 8 in 2024-09-11T07:22:51.591 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:51 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-11T07:22:51.591 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:51 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-11T07:22:51.591 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:51 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-11T07:22:52.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:52 smithi196 ceph-mon[30773]: osdmap e310: 8 total, 8 up, 8 in 2024-09-11T07:22:52.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:52 smithi196 ceph-mon[30773]: pgmap v436: 196 pgs: 31 undersized+peered, 78 active+undersized, 6 stale+active+clean, 18 undersized+degraded+peered, 19 active+undersized+degraded, 44 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 219/588 objects degraded (37.245%) 2024-09-11T07:22:52.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:52 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/667161110' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:22:52.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:52 smithi196 ceph-mon[30773]: from='client.35069 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:22:52.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:52 smithi071 ceph-mon[27013]: osdmap e310: 8 total, 8 up, 8 in 2024-09-11T07:22:52.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:52 smithi071 ceph-mon[27013]: pgmap v436: 196 pgs: 31 undersized+peered, 78 active+undersized, 6 stale+active+clean, 18 undersized+degraded+peered, 19 active+undersized+degraded, 44 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 219/588 objects degraded (37.245%) 2024-09-11T07:22:52.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:52 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/667161110' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:22:52.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:52 smithi071 ceph-mon[27013]: from='client.35069 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:22:52.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:52 smithi071 ceph-mon[37727]: osdmap e310: 8 total, 8 up, 8 in 2024-09-11T07:22:52.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:52 smithi071 ceph-mon[37727]: pgmap v436: 196 pgs: 31 undersized+peered, 78 active+undersized, 6 stale+active+clean, 18 undersized+degraded+peered, 19 active+undersized+degraded, 44 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 219/588 objects degraded (37.245%) 2024-09-11T07:22:52.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:52 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/667161110' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:22:52.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:52 smithi071 ceph-mon[37727]: from='client.35069 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:22:53.224 INFO:tasks.workunit.client.0.smithi071.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_aio_read_wait_for_complete_and_cb_error PASSED [ 78%] 2024-09-11T07:22:53.534 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:53 smithi196 ceph-mon[30773]: from='client.35069 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:22:53.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:53 smithi196 ceph-mon[30773]: osdmap e311: 8 total, 8 up, 8 in 2024-09-11T07:22:53.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:53 smithi071 ceph-mon[27013]: from='client.35069 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:22:53.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:53 smithi071 ceph-mon[27013]: osdmap e311: 8 total, 8 up, 8 in 2024-09-11T07:22:53.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:53 smithi071 ceph-mon[37727]: from='client.35069 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:22:53.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:53 smithi071 ceph-mon[37727]: osdmap e311: 8 total, 8 up, 8 in 2024-09-11T07:22:54.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:54 smithi196 ceph-mon[30773]: osdmap e312: 8 total, 8 up, 8 in 2024-09-11T07:22:54.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:54 smithi196 ceph-mon[30773]: pgmap v439: 164 pgs: 24 undersized+peered, 62 active+undersized, 5 stale+active+clean, 18 undersized+degraded+peered, 19 active+undersized+degraded, 36 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 219/588 objects degraded (37.245%) 2024-09-11T07:22:54.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:54 smithi071 ceph-mon[27013]: osdmap e312: 8 total, 8 up, 8 in 2024-09-11T07:22:54.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:54 smithi071 ceph-mon[27013]: pgmap v439: 164 pgs: 24 undersized+peered, 62 active+undersized, 5 stale+active+clean, 18 undersized+degraded+peered, 19 active+undersized+degraded, 36 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 219/588 objects degraded (37.245%) 2024-09-11T07:22:54.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:54 smithi071 ceph-mon[37727]: osdmap e312: 8 total, 8 up, 8 in 2024-09-11T07:22:54.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:54 smithi071 ceph-mon[37727]: pgmap v439: 164 pgs: 24 undersized+peered, 62 active+undersized, 5 stale+active+clean, 18 undersized+degraded+peered, 19 active+undersized+degraded, 36 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 219/588 objects degraded (37.245%) 2024-09-11T07:22:55.242 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 11 07:22:54 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-iscsi-iscsi-a[58197]: debug there is no tcmu-runner data available 2024-09-11T07:22:55.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:55 smithi196 ceph-mon[30773]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T07:22:55.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:55 smithi196 ceph-mon[30773]: osdmap e313: 8 total, 8 up, 8 in 2024-09-11T07:22:55.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:55 smithi196 ceph-mon[30773]: from='client.34442 v1:172.21.15.196:0/1952921461' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:22:55.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:55 smithi071 ceph-mon[27013]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T07:22:55.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:55 smithi071 ceph-mon[27013]: osdmap e313: 8 total, 8 up, 8 in 2024-09-11T07:22:55.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:55 smithi071 ceph-mon[27013]: from='client.34442 v1:172.21.15.196:0/1952921461' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:22:55.589 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:55 smithi071 ceph-mon[37727]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T07:22:55.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:55 smithi071 ceph-mon[37727]: osdmap e313: 8 total, 8 up, 8 in 2024-09-11T07:22:55.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:55 smithi071 ceph-mon[37727]: from='client.34442 v1:172.21.15.196:0/1952921461' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:22:56.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:56 smithi196 ceph-mon[30773]: osdmap e314: 8 total, 8 up, 8 in 2024-09-11T07:22:56.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:56 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/1504418917' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:22:56.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:56 smithi196 ceph-mon[30773]: from='client.35075 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:22:56.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:56 smithi196 ceph-mon[30773]: pgmap v442: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 93 KiB/s rd, 93 op/s; 27 B/s, 0 objects/s recovering 2024-09-11T07:22:56.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:56 smithi071 ceph-mon[27013]: osdmap e314: 8 total, 8 up, 8 in 2024-09-11T07:22:56.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:56 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/1504418917' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:22:56.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:56 smithi071 ceph-mon[27013]: from='client.35075 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:22:56.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:56 smithi071 ceph-mon[27013]: pgmap v442: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 93 KiB/s rd, 93 op/s; 27 B/s, 0 objects/s recovering 2024-09-11T07:22:56.590 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:22:56 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: ::ffff:172.21.15.196 - - [11/Sep/2024:07:22:56] "GET /metrics HTTP/1.1" 200 225230 "" "Prometheus/2.43.0" 2024-09-11T07:22:56.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:56 smithi071 ceph-mon[37727]: osdmap e314: 8 total, 8 up, 8 in 2024-09-11T07:22:56.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:56 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/1504418917' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:22:56.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:56 smithi071 ceph-mon[37727]: from='client.35075 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:22:56.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:56 smithi071 ceph-mon[37727]: pgmap v442: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 93 KiB/s rd, 93 op/s; 27 B/s, 0 objects/s recovering 2024-09-11T07:22:57.035 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 07:22:56 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-x[32609]: ::ffff:172.21.15.196 - - [11/Sep/2024:07:22:56] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:22:57.255 INFO:tasks.workunit.client.0.smithi071.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_lock PASSED [ 79%] 2024-09-11T07:22:57.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:57 smithi196 ceph-mon[30773]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-09-11T07:22:57.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:57 smithi196 ceph-mon[30773]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 219/588 objects degraded (37.245%), 37 pgs degraded) 2024-09-11T07:22:57.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:57 smithi196 ceph-mon[30773]: from='client.35075 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:22:57.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:57 smithi196 ceph-mon[30773]: osdmap e315: 8 total, 8 up, 8 in 2024-09-11T07:22:57.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:57 smithi071 ceph-mon[27013]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-09-11T07:22:57.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:57 smithi071 ceph-mon[27013]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 219/588 objects degraded (37.245%), 37 pgs degraded) 2024-09-11T07:22:57.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:57 smithi071 ceph-mon[27013]: from='client.35075 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:22:57.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:57 smithi071 ceph-mon[27013]: osdmap e315: 8 total, 8 up, 8 in 2024-09-11T07:22:57.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:57 smithi071 ceph-mon[37727]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-09-11T07:22:57.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:57 smithi071 ceph-mon[37727]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 219/588 objects degraded (37.245%), 37 pgs degraded) 2024-09-11T07:22:57.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:57 smithi071 ceph-mon[37727]: from='client.35075 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:22:57.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:57 smithi071 ceph-mon[37727]: osdmap e315: 8 total, 8 up, 8 in 2024-09-11T07:22:58.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:58 smithi196 ceph-mon[30773]: osdmap e316: 8 total, 8 up, 8 in 2024-09-11T07:22:58.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:58 smithi196 ceph-mon[30773]: pgmap v445: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 93 KiB/s rd, 93 op/s; 27 B/s, 0 objects/s recovering 2024-09-11T07:22:58.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:58 smithi071 ceph-mon[27013]: osdmap e316: 8 total, 8 up, 8 in 2024-09-11T07:22:58.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:58 smithi071 ceph-mon[27013]: pgmap v445: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 93 KiB/s rd, 93 op/s; 27 B/s, 0 objects/s recovering 2024-09-11T07:22:58.589 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:58 smithi071 ceph-mon[37727]: osdmap e316: 8 total, 8 up, 8 in 2024-09-11T07:22:58.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:58 smithi071 ceph-mon[37727]: pgmap v445: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 93 KiB/s rd, 93 op/s; 27 B/s, 0 objects/s recovering 2024-09-11T07:22:59.534 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:59 smithi196 ceph-mon[30773]: osdmap e317: 8 total, 8 up, 8 in 2024-09-11T07:22:59.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:22:59 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:22:59.589 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:59 smithi071 ceph-mon[37727]: osdmap e317: 8 total, 8 up, 8 in 2024-09-11T07:22:59.589 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:22:59 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:22:59.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:59 smithi071 ceph-mon[27013]: osdmap e317: 8 total, 8 up, 8 in 2024-09-11T07:22:59.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:22:59 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:23:00.534 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:00 smithi196 ceph-mon[30773]: osdmap e318: 8 total, 8 up, 8 in 2024-09-11T07:23:00.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:00 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/991457376' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:23:00.535 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:00 smithi196 ceph-mon[30773]: pgmap v448: 196 pgs: 196 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-11T07:23:00.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:00 smithi071 ceph-mon[27013]: osdmap e318: 8 total, 8 up, 8 in 2024-09-11T07:23:00.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:00 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/991457376' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:23:00.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:00 smithi071 ceph-mon[27013]: pgmap v448: 196 pgs: 196 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-11T07:23:00.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:00 smithi071 ceph-mon[37727]: osdmap e318: 8 total, 8 up, 8 in 2024-09-11T07:23:00.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:00 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/991457376' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:23:00.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:00 smithi071 ceph-mon[37727]: pgmap v448: 196 pgs: 196 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-11T07:23:01.287 INFO:tasks.workunit.client.0.smithi071.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_execute PASSED [ 80%] 2024-09-11T07:23:01.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:01 smithi071 ceph-mon[27013]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T07:23:01.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:01 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/991457376' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:23:01.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:01 smithi071 ceph-mon[27013]: osdmap e319: 8 total, 8 up, 8 in 2024-09-11T07:23:01.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:01 smithi071 ceph-mon[37727]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T07:23:01.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:01 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/991457376' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:23:01.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:01 smithi071 ceph-mon[37727]: osdmap e319: 8 total, 8 up, 8 in 2024-09-11T07:23:01.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:01 smithi196 ceph-mon[30773]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T07:23:01.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:01 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/991457376' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:23:01.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:01 smithi196 ceph-mon[30773]: osdmap e319: 8 total, 8 up, 8 in 2024-09-11T07:23:02.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:02 smithi071 ceph-mon[27013]: osdmap e320: 8 total, 8 up, 8 in 2024-09-11T07:23:02.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:02 smithi071 ceph-mon[27013]: pgmap v451: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:23:02.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:02 smithi071 ceph-mon[37727]: osdmap e320: 8 total, 8 up, 8 in 2024-09-11T07:23:02.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:02 smithi071 ceph-mon[37727]: pgmap v451: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:23:02.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:02 smithi196 ceph-mon[30773]: osdmap e320: 8 total, 8 up, 8 in 2024-09-11T07:23:02.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:02 smithi196 ceph-mon[30773]: pgmap v451: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:23:03.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:03 smithi071 ceph-mon[27013]: osdmap e321: 8 total, 8 up, 8 in 2024-09-11T07:23:03.589 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:03 smithi071 ceph-mon[37727]: osdmap e321: 8 total, 8 up, 8 in 2024-09-11T07:23:03.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:03 smithi196 ceph-mon[30773]: osdmap e321: 8 total, 8 up, 8 in 2024-09-11T07:23:04.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:04 smithi071 ceph-mon[27013]: osdmap e322: 8 total, 8 up, 8 in 2024-09-11T07:23:04.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:04 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/1422226232' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:23:04.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:04 smithi071 ceph-mon[27013]: pgmap v454: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:23:04.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:04 smithi071 ceph-mon[37727]: osdmap e322: 8 total, 8 up, 8 in 2024-09-11T07:23:04.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:04 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/1422226232' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:23:04.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:04 smithi071 ceph-mon[37727]: pgmap v454: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:23:04.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:04 smithi196 ceph-mon[30773]: osdmap e322: 8 total, 8 up, 8 in 2024-09-11T07:23:04.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:04 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/1422226232' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:23:04.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:04 smithi196 ceph-mon[30773]: pgmap v454: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:23:05.285 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 11 07:23:04 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-iscsi-iscsi-a[58197]: debug there is no tcmu-runner data available 2024-09-11T07:23:05.318 INFO:tasks.workunit.client.0.smithi071.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_aio_execute PASSED [ 82%] 2024-09-11T07:23:05.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:05 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/1422226232' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:23:05.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:05 smithi071 ceph-mon[27013]: osdmap e323: 8 total, 8 up, 8 in 2024-09-11T07:23:05.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:05 smithi071 ceph-mon[27013]: from='client.34442 v1:172.21.15.196:0/1952921461' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:23:05.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:05 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/1422226232' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:23:05.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:05 smithi071 ceph-mon[37727]: osdmap e323: 8 total, 8 up, 8 in 2024-09-11T07:23:05.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:05 smithi071 ceph-mon[37727]: from='client.34442 v1:172.21.15.196:0/1952921461' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:23:05.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:05 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/1422226232' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:23:05.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:05 smithi196 ceph-mon[30773]: osdmap e323: 8 total, 8 up, 8 in 2024-09-11T07:23:05.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:05 smithi196 ceph-mon[30773]: from='client.34442 v1:172.21.15.196:0/1952921461' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:23:06.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:06 smithi071 ceph-mon[27013]: osdmap e324: 8 total, 8 up, 8 in 2024-09-11T07:23:06.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:06 smithi071 ceph-mon[27013]: pgmap v457: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:23:06.590 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:23:06 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: ::ffff:172.21.15.196 - - [11/Sep/2024:07:23:06] "GET /metrics HTTP/1.1" 200 225219 "" "Prometheus/2.43.0" 2024-09-11T07:23:06.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:06 smithi071 ceph-mon[37727]: osdmap e324: 8 total, 8 up, 8 in 2024-09-11T07:23:06.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:06 smithi071 ceph-mon[37727]: pgmap v457: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:23:06.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:06 smithi196 ceph-mon[30773]: osdmap e324: 8 total, 8 up, 8 in 2024-09-11T07:23:06.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:06 smithi196 ceph-mon[30773]: pgmap v457: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:23:06.785 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 07:23:06 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-x[32609]: ::ffff:172.21.15.196 - - [11/Sep/2024:07:23:06] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:23:07.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:07 smithi071 ceph-mon[27013]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T07:23:07.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:07 smithi071 ceph-mon[27013]: osdmap e325: 8 total, 8 up, 8 in 2024-09-11T07:23:07.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:07 smithi071 ceph-mon[37727]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T07:23:07.590 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:07 smithi071 ceph-mon[37727]: osdmap e325: 8 total, 8 up, 8 in 2024-09-11T07:23:07.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:07 smithi196 ceph-mon[30773]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T07:23:07.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:07 smithi196 ceph-mon[30773]: osdmap e325: 8 total, 8 up, 8 in 2024-09-11T07:23:08.686 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:08 smithi196 ceph-mon[30773]: osdmap e326: 8 total, 8 up, 8 in 2024-09-11T07:23:08.686 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:08 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/1004501032' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:23:08.686 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:08 smithi196 ceph-mon[30773]: pgmap v460: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:23:08.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:08 smithi071 ceph-mon[27013]: osdmap e326: 8 total, 8 up, 8 in 2024-09-11T07:23:08.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:08 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/1004501032' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:23:08.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:08 smithi071 ceph-mon[27013]: pgmap v460: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:23:08.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:08 smithi071 ceph-mon[37727]: osdmap e326: 8 total, 8 up, 8 in 2024-09-11T07:23:08.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:08 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/1004501032' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:23:08.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:08 smithi071 ceph-mon[37727]: pgmap v460: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:23:09.353 INFO:tasks.workunit.client.0.smithi071.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_aio_setxattr PASSED [ 83%] 2024-09-11T07:23:09.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:09 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/1004501032' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:23:09.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:09 smithi196 ceph-mon[30773]: osdmap e327: 8 total, 8 up, 8 in 2024-09-11T07:23:09.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:09 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/1004501032' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:23:09.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:09 smithi071 ceph-mon[27013]: osdmap e327: 8 total, 8 up, 8 in 2024-09-11T07:23:09.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:09 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/1004501032' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:23:09.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:09 smithi071 ceph-mon[37727]: osdmap e327: 8 total, 8 up, 8 in 2024-09-11T07:23:10.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:10 smithi196 ceph-mon[30773]: osdmap e328: 8 total, 8 up, 8 in 2024-09-11T07:23:10.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:10 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:23:10.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:10 smithi196 ceph-mon[30773]: pgmap v463: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:23:10.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:10 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:23:10.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:10 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:23:10.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:10 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:23:10.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:10 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:23:10.786 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:10 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:23:10.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:10 smithi071 ceph-mon[27013]: osdmap e328: 8 total, 8 up, 8 in 2024-09-11T07:23:10.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:10 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:23:10.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:10 smithi071 ceph-mon[27013]: pgmap v463: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:23:10.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:10 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:23:10.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:10 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:23:10.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:10 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:23:10.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:10 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:23:10.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:10 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:23:10.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:10 smithi071 ceph-mon[37727]: osdmap e328: 8 total, 8 up, 8 in 2024-09-11T07:23:10.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:10 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-11T07:23:10.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:10 smithi071 ceph-mon[37727]: pgmap v463: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:23:10.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:10 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:23:10.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:10 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:23:10.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:10 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-11T07:23:10.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:10 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-11T07:23:10.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:10 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' 2024-09-11T07:23:11.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:11 smithi196 ceph-mon[30773]: osdmap e329: 8 total, 8 up, 8 in 2024-09-11T07:23:11.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:11 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/887958423' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T07:23:11.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:11 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/887958423' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test_pool","app": "app1"}]: dispatch 2024-09-11T07:23:11.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:11 smithi196 ceph-mon[30773]: from='client.35099 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test_pool","app": "app1"}]: dispatch 2024-09-11T07:23:11.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:11 smithi071 ceph-mon[27013]: osdmap e329: 8 total, 8 up, 8 in 2024-09-11T07:23:11.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:11 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/887958423' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T07:23:11.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:11 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/887958423' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test_pool","app": "app1"}]: dispatch 2024-09-11T07:23:11.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:11 smithi071 ceph-mon[27013]: from='client.35099 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test_pool","app": "app1"}]: dispatch 2024-09-11T07:23:11.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:11 smithi071 ceph-mon[37727]: osdmap e329: 8 total, 8 up, 8 in 2024-09-11T07:23:11.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:11 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/887958423' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-11T07:23:11.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:11 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/887958423' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test_pool","app": "app1"}]: dispatch 2024-09-11T07:23:11.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:11 smithi071 ceph-mon[37727]: from='client.35099 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test_pool","app": "app1"}]: dispatch 2024-09-11T07:23:12.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:12 smithi196 ceph-mon[30773]: from='client.35099 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test_pool","app": "app1"}]': finished 2024-09-11T07:23:12.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:12 smithi196 ceph-mon[30773]: osdmap e330: 8 total, 8 up, 8 in 2024-09-11T07:23:12.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:12 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/887958423' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test_pool","app": "app2"}]: dispatch 2024-09-11T07:23:12.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:12 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/887958423' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test_pool","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-09-11T07:23:12.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:12 smithi196 ceph-mon[30773]: from='client.35099 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test_pool","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-09-11T07:23:12.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:12 smithi196 ceph-mon[30773]: pgmap v466: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:23:12.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:12 smithi071 ceph-mon[27013]: from='client.35099 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test_pool","app": "app1"}]': finished 2024-09-11T07:23:12.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:12 smithi071 ceph-mon[27013]: osdmap e330: 8 total, 8 up, 8 in 2024-09-11T07:23:12.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:12 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/887958423' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test_pool","app": "app2"}]: dispatch 2024-09-11T07:23:12.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:12 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/887958423' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test_pool","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-09-11T07:23:12.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:12 smithi071 ceph-mon[27013]: from='client.35099 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test_pool","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-09-11T07:23:12.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:12 smithi071 ceph-mon[27013]: pgmap v466: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:23:12.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:12 smithi071 ceph-mon[37727]: from='client.35099 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test_pool","app": "app1"}]': finished 2024-09-11T07:23:12.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:12 smithi071 ceph-mon[37727]: osdmap e330: 8 total, 8 up, 8 in 2024-09-11T07:23:12.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:12 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/887958423' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test_pool","app": "app2"}]: dispatch 2024-09-11T07:23:12.841 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:12 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/887958423' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test_pool","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-09-11T07:23:12.841 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:12 smithi071 ceph-mon[37727]: from='client.35099 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test_pool","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-09-11T07:23:12.841 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:12 smithi071 ceph-mon[37727]: pgmap v466: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:23:13.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:13 smithi196 ceph-mon[30773]: from='client.35099 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test_pool","app": "app2","yes_i_really_mean_it": true}]': finished 2024-09-11T07:23:13.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:13 smithi196 ceph-mon[30773]: osdmap e331: 8 total, 8 up, 8 in 2024-09-11T07:23:13.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:13 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/887958423' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"test_pool","app":"dne","key":"key","value":"key"}]: dispatch 2024-09-11T07:23:13.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:13 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/887958423' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"test_pool","app":"app1","key":"key1","value":"val1"}]: dispatch 2024-09-11T07:23:13.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:13 smithi196 ceph-mon[30773]: from='client.35099 ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"test_pool","app":"app1","key":"key1","value":"val1"}]: dispatch 2024-09-11T07:23:13.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:13 smithi071 ceph-mon[27013]: from='client.35099 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test_pool","app": "app2","yes_i_really_mean_it": true}]': finished 2024-09-11T07:23:13.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:13 smithi071 ceph-mon[27013]: osdmap e331: 8 total, 8 up, 8 in 2024-09-11T07:23:13.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:13 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/887958423' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"test_pool","app":"dne","key":"key","value":"key"}]: dispatch 2024-09-11T07:23:13.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:13 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/887958423' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"test_pool","app":"app1","key":"key1","value":"val1"}]: dispatch 2024-09-11T07:23:13.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:13 smithi071 ceph-mon[27013]: from='client.35099 ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"test_pool","app":"app1","key":"key1","value":"val1"}]: dispatch 2024-09-11T07:23:13.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:13 smithi071 ceph-mon[37727]: from='client.35099 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test_pool","app": "app2","yes_i_really_mean_it": true}]': finished 2024-09-11T07:23:13.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:13 smithi071 ceph-mon[37727]: osdmap e331: 8 total, 8 up, 8 in 2024-09-11T07:23:13.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:13 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/887958423' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"test_pool","app":"dne","key":"key","value":"key"}]: dispatch 2024-09-11T07:23:13.841 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:13 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/887958423' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"test_pool","app":"app1","key":"key1","value":"val1"}]: dispatch 2024-09-11T07:23:13.841 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:13 smithi071 ceph-mon[37727]: from='client.35099 ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"test_pool","app":"app1","key":"key1","value":"val1"}]: dispatch 2024-09-11T07:23:14.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:14 smithi196 ceph-mon[30773]: from='client.35099 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"test_pool","app":"app1","key":"key1","value":"val1"}]': finished 2024-09-11T07:23:14.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:14 smithi196 ceph-mon[30773]: osdmap e332: 8 total, 8 up, 8 in 2024-09-11T07:23:14.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:14 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/887958423' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"test_pool","app":"app1","key":"key2","value":"val2"}]: dispatch 2024-09-11T07:23:14.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:14 smithi196 ceph-mon[30773]: from='client.35099 ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"test_pool","app":"app1","key":"key2","value":"val2"}]: dispatch 2024-09-11T07:23:14.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:14 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:23:14.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:14 smithi196 ceph-mon[30773]: pgmap v469: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:23:14.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:14 smithi071 ceph-mon[27013]: from='client.35099 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"test_pool","app":"app1","key":"key1","value":"val1"}]': finished 2024-09-11T07:23:14.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:14 smithi071 ceph-mon[27013]: osdmap e332: 8 total, 8 up, 8 in 2024-09-11T07:23:14.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:14 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/887958423' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"test_pool","app":"app1","key":"key2","value":"val2"}]: dispatch 2024-09-11T07:23:14.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:14 smithi071 ceph-mon[27013]: from='client.35099 ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"test_pool","app":"app1","key":"key2","value":"val2"}]: dispatch 2024-09-11T07:23:14.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:14 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:23:14.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:14 smithi071 ceph-mon[27013]: pgmap v469: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:23:14.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:14 smithi071 ceph-mon[37727]: from='client.35099 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"test_pool","app":"app1","key":"key1","value":"val1"}]': finished 2024-09-11T07:23:14.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:14 smithi071 ceph-mon[37727]: osdmap e332: 8 total, 8 up, 8 in 2024-09-11T07:23:14.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:14 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/887958423' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"test_pool","app":"app1","key":"key2","value":"val2"}]: dispatch 2024-09-11T07:23:14.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:14 smithi071 ceph-mon[37727]: from='client.35099 ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"test_pool","app":"app1","key":"key2","value":"val2"}]: dispatch 2024-09-11T07:23:14.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:14 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:23:14.841 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:14 smithi071 ceph-mon[37727]: pgmap v469: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:23:15.285 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 11 07:23:14 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-iscsi-iscsi-a[58197]: debug there is no tcmu-runner data available 2024-09-11T07:23:15.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:15 smithi196 ceph-mon[30773]: from='client.35099 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"test_pool","app":"app1","key":"key2","value":"val2"}]': finished 2024-09-11T07:23:15.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:15 smithi196 ceph-mon[30773]: osdmap e333: 8 total, 8 up, 8 in 2024-09-11T07:23:15.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:15 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/887958423' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"test_pool","app":"app2","key":"key1","value":"val1"}]: dispatch 2024-09-11T07:23:15.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:15 smithi196 ceph-mon[30773]: from='client.35099 ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"test_pool","app":"app2","key":"key1","value":"val1"}]: dispatch 2024-09-11T07:23:15.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:15 smithi196 ceph-mon[30773]: from='client.34442 v1:172.21.15.196:0/1952921461' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:23:15.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:15 smithi071 ceph-mon[27013]: from='client.35099 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"test_pool","app":"app1","key":"key2","value":"val2"}]': finished 2024-09-11T07:23:15.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:15 smithi071 ceph-mon[27013]: osdmap e333: 8 total, 8 up, 8 in 2024-09-11T07:23:15.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:15 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/887958423' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"test_pool","app":"app2","key":"key1","value":"val1"}]: dispatch 2024-09-11T07:23:15.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:15 smithi071 ceph-mon[27013]: from='client.35099 ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"test_pool","app":"app2","key":"key1","value":"val1"}]: dispatch 2024-09-11T07:23:15.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:15 smithi071 ceph-mon[27013]: from='client.34442 v1:172.21.15.196:0/1952921461' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:23:15.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:15 smithi071 ceph-mon[37727]: from='client.35099 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"test_pool","app":"app1","key":"key2","value":"val2"}]': finished 2024-09-11T07:23:15.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:15 smithi071 ceph-mon[37727]: osdmap e333: 8 total, 8 up, 8 in 2024-09-11T07:23:15.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:15 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/887958423' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"test_pool","app":"app2","key":"key1","value":"val1"}]: dispatch 2024-09-11T07:23:15.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:15 smithi071 ceph-mon[37727]: from='client.35099 ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"test_pool","app":"app2","key":"key1","value":"val1"}]: dispatch 2024-09-11T07:23:15.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:15 smithi071 ceph-mon[37727]: from='client.34442 v1:172.21.15.196:0/1952921461' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:23:16.413 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:23:16 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: ::ffff:172.21.15.196 - - [11/Sep/2024:07:23:16] "GET /metrics HTTP/1.1" 200 225220 "" "Prometheus/2.43.0" 2024-09-11T07:23:16.785 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 07:23:16 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-x[32609]: ::ffff:172.21.15.196 - - [11/Sep/2024:07:23:16] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:23:16.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:16 smithi196 ceph-mon[30773]: from='client.35099 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"test_pool","app":"app2","key":"key1","value":"val1"}]': finished 2024-09-11T07:23:16.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:16 smithi196 ceph-mon[30773]: osdmap e334: 8 total, 8 up, 8 in 2024-09-11T07:23:16.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:16 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/887958423' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"test_pool","app":"app1","key":"key1"}]: dispatch 2024-09-11T07:23:16.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:16 smithi196 ceph-mon[30773]: from='client.35099 ' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"test_pool","app":"app1","key":"key1"}]: dispatch 2024-09-11T07:23:16.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:16 smithi196 ceph-mon[30773]: pgmap v472: 196 pgs: 196 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:23:16.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:16 smithi071 ceph-mon[27013]: from='client.35099 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"test_pool","app":"app2","key":"key1","value":"val1"}]': finished 2024-09-11T07:23:16.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:16 smithi071 ceph-mon[27013]: osdmap e334: 8 total, 8 up, 8 in 2024-09-11T07:23:16.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:16 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/887958423' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"test_pool","app":"app1","key":"key1"}]: dispatch 2024-09-11T07:23:16.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:16 smithi071 ceph-mon[27013]: from='client.35099 ' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"test_pool","app":"app1","key":"key1"}]: dispatch 2024-09-11T07:23:16.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:16 smithi071 ceph-mon[27013]: pgmap v472: 196 pgs: 196 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:23:16.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:16 smithi071 ceph-mon[37727]: from='client.35099 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"test_pool","app":"app2","key":"key1","value":"val1"}]': finished 2024-09-11T07:23:16.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:16 smithi071 ceph-mon[37727]: osdmap e334: 8 total, 8 up, 8 in 2024-09-11T07:23:16.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:16 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/887958423' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"test_pool","app":"app1","key":"key1"}]: dispatch 2024-09-11T07:23:16.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:16 smithi071 ceph-mon[37727]: from='client.35099 ' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"test_pool","app":"app1","key":"key1"}]: dispatch 2024-09-11T07:23:16.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:16 smithi071 ceph-mon[37727]: pgmap v472: 196 pgs: 196 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:23:17.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:17 smithi196 ceph-mon[30773]: from='client.35099 ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"test_pool","app":"app1","key":"key1"}]': finished 2024-09-11T07:23:17.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:17 smithi196 ceph-mon[30773]: osdmap e335: 8 total, 8 up, 8 in 2024-09-11T07:23:17.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:17 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/887958423' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:23:17.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:17 smithi196 ceph-mon[30773]: from='client.35099 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:23:17.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:17 smithi071 ceph-mon[27013]: from='client.35099 ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"test_pool","app":"app1","key":"key1"}]': finished 2024-09-11T07:23:17.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:17 smithi071 ceph-mon[27013]: osdmap e335: 8 total, 8 up, 8 in 2024-09-11T07:23:17.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:17 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/887958423' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:23:17.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:17 smithi071 ceph-mon[27013]: from='client.35099 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:23:17.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:17 smithi071 ceph-mon[37727]: from='client.35099 ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"test_pool","app":"app1","key":"key1"}]': finished 2024-09-11T07:23:17.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:17 smithi071 ceph-mon[37727]: osdmap e335: 8 total, 8 up, 8 in 2024-09-11T07:23:17.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:17 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/887958423' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:23:17.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:17 smithi071 ceph-mon[37727]: from='client.35099 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:23:18.427 INFO:tasks.workunit.client.0.smithi071.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_applications PASSED [ 84%] 2024-09-11T07:23:18.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:18 smithi196 ceph-mon[30773]: from='client.35099 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:23:18.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:18 smithi196 ceph-mon[30773]: osdmap e336: 8 total, 8 up, 8 in 2024-09-11T07:23:18.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:18 smithi196 ceph-mon[30773]: pgmap v475: 196 pgs: 196 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:23:18.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:18 smithi071 ceph-mon[27013]: from='client.35099 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:23:18.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:18 smithi071 ceph-mon[27013]: osdmap e336: 8 total, 8 up, 8 in 2024-09-11T07:23:18.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:18 smithi071 ceph-mon[27013]: pgmap v475: 196 pgs: 196 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:23:18.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:18 smithi071 ceph-mon[37727]: from='client.35099 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:23:18.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:18 smithi071 ceph-mon[37727]: osdmap e336: 8 total, 8 up, 8 in 2024-09-11T07:23:18.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:18 smithi071 ceph-mon[37727]: pgmap v475: 196 pgs: 196 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:23:19.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:19 smithi196 ceph-mon[30773]: osdmap e337: 8 total, 8 up, 8 in 2024-09-11T07:23:19.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:19 smithi071 ceph-mon[27013]: osdmap e337: 8 total, 8 up, 8 in 2024-09-11T07:23:19.839 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:19 smithi071 ceph-mon[37727]: osdmap e337: 8 total, 8 up, 8 in 2024-09-11T07:23:20.788 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:20 smithi196 ceph-mon[30773]: osdmap e338: 8 total, 8 up, 8 in 2024-09-11T07:23:20.789 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:20 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/2706988963' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:23:20.789 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:20 smithi196 ceph-mon[30773]: pgmap v478: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:23:20.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:20 smithi071 ceph-mon[27013]: osdmap e338: 8 total, 8 up, 8 in 2024-09-11T07:23:20.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:20 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/2706988963' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:23:20.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:20 smithi071 ceph-mon[27013]: pgmap v478: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:23:20.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:20 smithi071 ceph-mon[37727]: osdmap e338: 8 total, 8 up, 8 in 2024-09-11T07:23:20.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:20 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/2706988963' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:23:20.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:20 smithi071 ceph-mon[37727]: pgmap v478: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:23:21.458 INFO:tasks.workunit.client.0.smithi071.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_service_daemon PASSED [ 85%] 2024-09-11T07:23:21.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:21 smithi196 ceph-mon[30773]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T07:23:21.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:21 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/2706988963' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:23:21.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:21 smithi196 ceph-mon[30773]: osdmap e339: 8 total, 8 up, 8 in 2024-09-11T07:23:21.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:21 smithi071 ceph-mon[27013]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T07:23:21.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:21 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/2706988963' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:23:21.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:21 smithi071 ceph-mon[27013]: osdmap e339: 8 total, 8 up, 8 in 2024-09-11T07:23:21.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:21 smithi071 ceph-mon[37727]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T07:23:21.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:21 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/2706988963' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:23:21.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:21 smithi071 ceph-mon[37727]: osdmap e339: 8 total, 8 up, 8 in 2024-09-11T07:23:22.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:22 smithi196 ceph-mon[30773]: osdmap e340: 8 total, 8 up, 8 in 2024-09-11T07:23:22.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:22 smithi196 ceph-mon[30773]: pgmap v481: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:23:22.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:22 smithi071 ceph-mon[27013]: osdmap e340: 8 total, 8 up, 8 in 2024-09-11T07:23:22.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:22 smithi071 ceph-mon[27013]: pgmap v481: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:23:22.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:22 smithi071 ceph-mon[37727]: osdmap e340: 8 total, 8 up, 8 in 2024-09-11T07:23:22.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:22 smithi071 ceph-mon[37727]: pgmap v481: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:23:23.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:23 smithi196 ceph-mon[30773]: osdmap e341: 8 total, 8 up, 8 in 2024-09-11T07:23:23.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:23 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/2117263357' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:23:23.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:23 smithi196 ceph-mon[30773]: from='client.35111 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:23:23.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:23 smithi071 ceph-mon[27013]: osdmap e341: 8 total, 8 up, 8 in 2024-09-11T07:23:23.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:23 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/2117263357' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:23:23.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:23 smithi071 ceph-mon[27013]: from='client.35111 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:23:23.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:23 smithi071 ceph-mon[37727]: osdmap e341: 8 total, 8 up, 8 in 2024-09-11T07:23:23.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:23 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/2117263357' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:23:23.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:23 smithi071 ceph-mon[37727]: from='client.35111 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:23:24.490 INFO:tasks.workunit.client.0.smithi071.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_alignment PASSED [ 86%] 2024-09-11T07:23:24.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:24 smithi196 ceph-mon[30773]: from='client.35111 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:23:24.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:24 smithi196 ceph-mon[30773]: osdmap e342: 8 total, 8 up, 8 in 2024-09-11T07:23:24.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:24 smithi196 ceph-mon[30773]: pgmap v484: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:23:24.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:24 smithi071 ceph-mon[27013]: from='client.35111 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:23:24.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:24 smithi071 ceph-mon[27013]: osdmap e342: 8 total, 8 up, 8 in 2024-09-11T07:23:24.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:24 smithi071 ceph-mon[27013]: pgmap v484: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:23:24.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:24 smithi071 ceph-mon[37727]: from='client.35111 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:23:24.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:24 smithi071 ceph-mon[37727]: osdmap e342: 8 total, 8 up, 8 in 2024-09-11T07:23:24.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:24 smithi071 ceph-mon[37727]: pgmap v484: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:23:25.285 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 11 07:23:24 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-iscsi-iscsi-a[58197]: debug there is no tcmu-runner data available 2024-09-11T07:23:25.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:25 smithi196 ceph-mon[30773]: osdmap e343: 8 total, 8 up, 8 in 2024-09-11T07:23:25.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:25 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/2896692082' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-ec", "profile": ["k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T07:23:25.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:25 smithi196 ceph-mon[30773]: from='client.34442 v1:172.21.15.196:0/1952921461' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:23:25.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:25 smithi071 ceph-mon[27013]: osdmap e343: 8 total, 8 up, 8 in 2024-09-11T07:23:25.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:25 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/2896692082' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-ec", "profile": ["k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T07:23:25.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:25 smithi071 ceph-mon[27013]: from='client.34442 v1:172.21.15.196:0/1952921461' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:23:25.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:25 smithi071 ceph-mon[37727]: osdmap e343: 8 total, 8 up, 8 in 2024-09-11T07:23:25.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:25 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/2896692082' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-ec", "profile": ["k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-11T07:23:25.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:25 smithi071 ceph-mon[37727]: from='client.34442 v1:172.21.15.196:0/1952921461' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:23:26.488 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:23:26 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: ::ffff:172.21.15.196 - - [11/Sep/2024:07:23:26] "GET /metrics HTTP/1.1" 200 225220 "" "Prometheus/2.43.0" 2024-09-11T07:23:26.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:26 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/2896692082' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-ec", "profile": ["k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T07:23:26.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:26 smithi196 ceph-mon[30773]: osdmap e344: 8 total, 8 up, 8 in 2024-09-11T07:23:26.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:26 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/2896692082' entity='client.admin' cmd=[{"prefix": "osd pool create", "pg_num": 8, "pgp_num": 8, "pool": "test-ec", "pool_type": "erasure", "erasure_code_profile": "testprofile-test-ec"}]: dispatch 2024-09-11T07:23:26.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:26 smithi196 ceph-mon[30773]: pgmap v487: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:23:26.785 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 07:23:26 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-x[32609]: ::ffff:172.21.15.196 - - [11/Sep/2024:07:23:26] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:23:26.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:26 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/2896692082' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-ec", "profile": ["k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T07:23:26.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:26 smithi071 ceph-mon[27013]: osdmap e344: 8 total, 8 up, 8 in 2024-09-11T07:23:26.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:26 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/2896692082' entity='client.admin' cmd=[{"prefix": "osd pool create", "pg_num": 8, "pgp_num": 8, "pool": "test-ec", "pool_type": "erasure", "erasure_code_profile": "testprofile-test-ec"}]: dispatch 2024-09-11T07:23:26.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:26 smithi071 ceph-mon[27013]: pgmap v487: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:23:26.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:26 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/2896692082' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-ec", "profile": ["k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-11T07:23:26.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:26 smithi071 ceph-mon[37727]: osdmap e344: 8 total, 8 up, 8 in 2024-09-11T07:23:26.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:26 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/2896692082' entity='client.admin' cmd=[{"prefix": "osd pool create", "pg_num": 8, "pgp_num": 8, "pool": "test-ec", "pool_type": "erasure", "erasure_code_profile": "testprofile-test-ec"}]: dispatch 2024-09-11T07:23:26.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:26 smithi071 ceph-mon[37727]: pgmap v487: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:23:27.784 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:27 smithi196 ceph-mon[30773]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T07:23:27.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:27 smithi196 ceph-mon[30773]: osdmap e345: 8 total, 8 up, 8 in 2024-09-11T07:23:27.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:27 smithi071 ceph-mon[27013]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T07:23:27.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:27 smithi071 ceph-mon[27013]: osdmap e345: 8 total, 8 up, 8 in 2024-09-11T07:23:27.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:27 smithi071 ceph-mon[37727]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T07:23:27.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:27 smithi071 ceph-mon[37727]: osdmap e345: 8 total, 8 up, 8 in 2024-09-11T07:23:28.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:28 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/2896692082' entity='client.admin' cmd='[{"prefix": "osd pool create", "pg_num": 8, "pgp_num": 8, "pool": "test-ec", "pool_type": "erasure", "erasure_code_profile": "testprofile-test-ec"}]': finished 2024-09-11T07:23:28.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:28 smithi196 ceph-mon[30773]: osdmap e346: 8 total, 8 up, 8 in 2024-09-11T07:23:28.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:28 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/2896692082' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:23:28.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:28 smithi196 ceph-mon[30773]: pgmap v490: 172 pgs: 8 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:23:28.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:28 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/2896692082' entity='client.admin' cmd='[{"prefix": "osd pool create", "pg_num": 8, "pgp_num": 8, "pool": "test-ec", "pool_type": "erasure", "erasure_code_profile": "testprofile-test-ec"}]': finished 2024-09-11T07:23:28.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:28 smithi071 ceph-mon[27013]: osdmap e346: 8 total, 8 up, 8 in 2024-09-11T07:23:28.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:28 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/2896692082' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:23:28.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:28 smithi071 ceph-mon[27013]: pgmap v490: 172 pgs: 8 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:23:28.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:28 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/2896692082' entity='client.admin' cmd='[{"prefix": "osd pool create", "pg_num": 8, "pgp_num": 8, "pool": "test-ec", "pool_type": "erasure", "erasure_code_profile": "testprofile-test-ec"}]': finished 2024-09-11T07:23:28.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:28 smithi071 ceph-mon[37727]: osdmap e346: 8 total, 8 up, 8 in 2024-09-11T07:23:28.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:28 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/2896692082' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:23:28.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:28 smithi071 ceph-mon[37727]: pgmap v490: 172 pgs: 8 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:23:29.510 INFO:tasks.workunit.client.0.smithi071.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctxEc::test_alignment PASSED [ 87%] 2024-09-11T07:23:29.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:29 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/2896692082' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:23:29.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:29 smithi196 ceph-mon[30773]: osdmap e347: 8 total, 8 up, 8 in 2024-09-11T07:23:29.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:29 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:23:29.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:29 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/2896692082' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:23:29.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:29 smithi071 ceph-mon[27013]: osdmap e347: 8 total, 8 up, 8 in 2024-09-11T07:23:29.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:29 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:23:29.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:29 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/2896692082' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:23:29.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:29 smithi071 ceph-mon[37727]: osdmap e347: 8 total, 8 up, 8 in 2024-09-11T07:23:29.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:29 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:23:30.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:30 smithi196 ceph-mon[30773]: osdmap e348: 8 total, 8 up, 8 in 2024-09-11T07:23:30.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:30 smithi196 ceph-mon[30773]: pgmap v493: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:23:30.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:30 smithi071 ceph-mon[27013]: osdmap e348: 8 total, 8 up, 8 in 2024-09-11T07:23:30.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:30 smithi071 ceph-mon[27013]: pgmap v493: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:23:30.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:30 smithi071 ceph-mon[37727]: osdmap e348: 8 total, 8 up, 8 in 2024-09-11T07:23:30.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:30 smithi071 ceph-mon[37727]: pgmap v493: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:23:31.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:31 smithi196 ceph-mon[30773]: osdmap e349: 8 total, 8 up, 8 in 2024-09-11T07:23:31.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:31 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/2051393944' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:23:31.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:31 smithi196 ceph-mon[30773]: from='client.35123 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:23:31.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:31 smithi071 ceph-mon[27013]: osdmap e349: 8 total, 8 up, 8 in 2024-09-11T07:23:31.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:31 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/2051393944' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:23:31.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:31 smithi071 ceph-mon[27013]: from='client.35123 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:23:31.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:31 smithi071 ceph-mon[37727]: osdmap e349: 8 total, 8 up, 8 in 2024-09-11T07:23:31.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:31 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/2051393944' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:23:31.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:31 smithi071 ceph-mon[37727]: from='client.35123 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:23:32.529 INFO:tasks.workunit.client.0.smithi071.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx2::test_get_last_version PASSED [ 88%] 2024-09-11T07:23:32.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:32 smithi071 ceph-mon[27013]: from='client.35123 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:23:32.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:32 smithi071 ceph-mon[27013]: osdmap e350: 8 total, 8 up, 8 in 2024-09-11T07:23:32.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:32 smithi071 ceph-mon[27013]: pgmap v496: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:23:32.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:32 smithi071 ceph-mon[37727]: from='client.35123 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:23:32.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:32 smithi071 ceph-mon[37727]: osdmap e350: 8 total, 8 up, 8 in 2024-09-11T07:23:32.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:32 smithi071 ceph-mon[37727]: pgmap v496: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:23:33.034 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:32 smithi196 ceph-mon[30773]: from='client.35123 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:23:33.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:32 smithi196 ceph-mon[30773]: osdmap e350: 8 total, 8 up, 8 in 2024-09-11T07:23:33.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:32 smithi196 ceph-mon[30773]: pgmap v496: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:23:33.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:33 smithi071 ceph-mon[27013]: osdmap e351: 8 total, 8 up, 8 in 2024-09-11T07:23:33.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:33 smithi071 ceph-mon[27013]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T07:23:33.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:33 smithi071 ceph-mon[37727]: osdmap e351: 8 total, 8 up, 8 in 2024-09-11T07:23:33.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:33 smithi071 ceph-mon[37727]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T07:23:34.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:33 smithi196 ceph-mon[30773]: osdmap e351: 8 total, 8 up, 8 in 2024-09-11T07:23:34.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:33 smithi196 ceph-mon[30773]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T07:23:34.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:34 smithi071 ceph-mon[27013]: osdmap e352: 8 total, 8 up, 8 in 2024-09-11T07:23:34.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:34 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/1098292409' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:23:34.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:34 smithi071 ceph-mon[27013]: pgmap v499: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:23:34.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:34 smithi071 ceph-mon[37727]: osdmap e352: 8 total, 8 up, 8 in 2024-09-11T07:23:34.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:34 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/1098292409' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:23:34.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:34 smithi071 ceph-mon[37727]: pgmap v499: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:23:34.927 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:34 smithi196 ceph-mon[30773]: osdmap e352: 8 total, 8 up, 8 in 2024-09-11T07:23:34.927 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:34 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/1098292409' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:23:34.927 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:34 smithi196 ceph-mon[30773]: pgmap v499: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:23:35.285 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 11 07:23:34 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-iscsi-iscsi-a[58197]: debug there is no tcmu-runner data available 2024-09-11T07:23:35.555 INFO:tasks.workunit.client.0.smithi071.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx2::test_get_stats PASSED [ 89%] 2024-09-11T07:23:35.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:35 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/1098292409' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:23:35.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:35 smithi071 ceph-mon[27013]: osdmap e353: 8 total, 8 up, 8 in 2024-09-11T07:23:35.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:35 smithi071 ceph-mon[27013]: from='client.34442 v1:172.21.15.196:0/1952921461' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:23:35.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:35 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/1098292409' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:23:35.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:35 smithi071 ceph-mon[37727]: osdmap e353: 8 total, 8 up, 8 in 2024-09-11T07:23:35.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:35 smithi071 ceph-mon[37727]: from='client.34442 v1:172.21.15.196:0/1952921461' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:23:36.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:35 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/1098292409' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:23:36.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:35 smithi196 ceph-mon[30773]: osdmap e353: 8 total, 8 up, 8 in 2024-09-11T07:23:36.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:35 smithi196 ceph-mon[30773]: from='client.34442 v1:172.21.15.196:0/1952921461' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:23:36.563 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:23:36 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: ::ffff:172.21.15.196 - - [11/Sep/2024:07:23:36] "GET /metrics HTTP/1.1" 200 225218 "" "Prometheus/2.43.0" 2024-09-11T07:23:36.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:36 smithi071 ceph-mon[27013]: osdmap e354: 8 total, 8 up, 8 in 2024-09-11T07:23:36.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:36 smithi071 ceph-mon[27013]: pgmap v502: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:23:36.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:36 smithi071 ceph-mon[37727]: osdmap e354: 8 total, 8 up, 8 in 2024-09-11T07:23:36.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:36 smithi071 ceph-mon[37727]: pgmap v502: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:23:37.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:36 smithi196 ceph-mon[30773]: osdmap e354: 8 total, 8 up, 8 in 2024-09-11T07:23:37.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:36 smithi196 ceph-mon[30773]: pgmap v502: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:23:37.035 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 07:23:36 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-x[32609]: ::ffff:172.21.15.196 - - [11/Sep/2024:07:23:36] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:23:37.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:37 smithi071 ceph-mon[27013]: osdmap e355: 8 total, 8 up, 8 in 2024-09-11T07:23:37.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:37 smithi071 ceph-mon[37727]: osdmap e355: 8 total, 8 up, 8 in 2024-09-11T07:23:38.034 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:37 smithi196 ceph-mon[30773]: osdmap e355: 8 total, 8 up, 8 in 2024-09-11T07:23:38.592 INFO:tasks.workunit.client.0.smithi071.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestObject::test_read PASSED [ 91%] 2024-09-11T07:23:38.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:38 smithi071 ceph-mon[27013]: osdmap e356: 8 total, 8 up, 8 in 2024-09-11T07:23:38.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:38 smithi071 ceph-mon[27013]: pgmap v505: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:23:38.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:38 smithi071 ceph-mon[37727]: osdmap e356: 8 total, 8 up, 8 in 2024-09-11T07:23:38.840 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:38 smithi071 ceph-mon[37727]: pgmap v505: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:23:38.936 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:38 smithi196 ceph-mon[30773]: osdmap e356: 8 total, 8 up, 8 in 2024-09-11T07:23:38.936 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:38 smithi196 ceph-mon[30773]: pgmap v505: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:23:40.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:39 smithi196 ceph-mon[30773]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T07:23:40.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:39 smithi196 ceph-mon[30773]: osdmap e357: 8 total, 8 up, 8 in 2024-09-11T07:23:40.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:39 smithi071 ceph-mon[27013]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T07:23:40.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:39 smithi071 ceph-mon[27013]: osdmap e357: 8 total, 8 up, 8 in 2024-09-11T07:23:40.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:39 smithi071 ceph-mon[37727]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T07:23:40.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:39 smithi071 ceph-mon[37727]: osdmap e357: 8 total, 8 up, 8 in 2024-09-11T07:23:41.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:40 smithi196 ceph-mon[30773]: osdmap e358: 8 total, 8 up, 8 in 2024-09-11T07:23:41.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:40 smithi196 ceph-mon[30773]: pgmap v508: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:23:41.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:40 smithi196 ceph-mon[30773]: osdmap e359: 8 total, 8 up, 8 in 2024-09-11T07:23:41.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:40 smithi071 ceph-mon[27013]: osdmap e358: 8 total, 8 up, 8 in 2024-09-11T07:23:41.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:40 smithi071 ceph-mon[27013]: pgmap v508: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:23:41.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:40 smithi071 ceph-mon[27013]: osdmap e359: 8 total, 8 up, 8 in 2024-09-11T07:23:41.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:40 smithi071 ceph-mon[37727]: osdmap e358: 8 total, 8 up, 8 in 2024-09-11T07:23:41.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:40 smithi071 ceph-mon[37727]: pgmap v508: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:23:41.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:40 smithi071 ceph-mon[37727]: osdmap e359: 8 total, 8 up, 8 in 2024-09-11T07:23:41.607 INFO:tasks.workunit.client.0.smithi071.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestObject::test_seek PASSED [ 92%] 2024-09-11T07:23:43.034 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:42 smithi196 ceph-mon[30773]: osdmap e360: 8 total, 8 up, 8 in 2024-09-11T07:23:43.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:42 smithi196 ceph-mon[30773]: pgmap v511: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:23:43.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:42 smithi071 ceph-mon[27013]: osdmap e360: 8 total, 8 up, 8 in 2024-09-11T07:23:43.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:42 smithi071 ceph-mon[27013]: pgmap v511: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:23:43.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:42 smithi071 ceph-mon[37727]: osdmap e360: 8 total, 8 up, 8 in 2024-09-11T07:23:43.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:42 smithi071 ceph-mon[37727]: pgmap v511: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:23:44.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:43 smithi196 ceph-mon[30773]: osdmap e361: 8 total, 8 up, 8 in 2024-09-11T07:23:44.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:43 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:23:44.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:43 smithi071 ceph-mon[27013]: osdmap e361: 8 total, 8 up, 8 in 2024-09-11T07:23:44.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:43 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:23:44.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:43 smithi071 ceph-mon[37727]: osdmap e361: 8 total, 8 up, 8 in 2024-09-11T07:23:44.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:43 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:23:44.635 INFO:tasks.workunit.client.0.smithi071.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestObject::test_write PASSED [ 93%] 2024-09-11T07:23:44.938 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:44 smithi196 ceph-mon[30773]: osdmap e362: 8 total, 8 up, 8 in 2024-09-11T07:23:44.938 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:44 smithi196 ceph-mon[30773]: pgmap v514: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:23:44.938 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:44 smithi196 ceph-mon[30773]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T07:23:45.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:44 smithi071 ceph-mon[27013]: osdmap e362: 8 total, 8 up, 8 in 2024-09-11T07:23:45.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:44 smithi071 ceph-mon[27013]: pgmap v514: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:23:45.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:44 smithi071 ceph-mon[27013]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T07:23:45.089 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:44 smithi071 ceph-mon[37727]: osdmap e362: 8 total, 8 up, 8 in 2024-09-11T07:23:45.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:44 smithi071 ceph-mon[37727]: pgmap v514: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:23:45.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:44 smithi071 ceph-mon[37727]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T07:23:45.285 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 11 07:23:44 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-iscsi-iscsi-a[58197]: debug there is no tcmu-runner data available 2024-09-11T07:23:46.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:45 smithi196 ceph-mon[30773]: osdmap e363: 8 total, 8 up, 8 in 2024-09-11T07:23:46.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:45 smithi196 ceph-mon[30773]: from='client.34442 v1:172.21.15.196:0/1952921461' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:23:46.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:45 smithi071 ceph-mon[27013]: osdmap e363: 8 total, 8 up, 8 in 2024-09-11T07:23:46.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:45 smithi071 ceph-mon[27013]: from='client.34442 v1:172.21.15.196:0/1952921461' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:23:46.089 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:45 smithi071 ceph-mon[37727]: osdmap e363: 8 total, 8 up, 8 in 2024-09-11T07:23:46.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:45 smithi071 ceph-mon[37727]: from='client.34442 v1:172.21.15.196:0/1952921461' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:23:46.589 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:23:46 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: ::ffff:172.21.15.196 - - [11/Sep/2024:07:23:46] "GET /metrics HTTP/1.1" 200 225238 "" "Prometheus/2.43.0" 2024-09-11T07:23:47.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:46 smithi196 ceph-mon[30773]: osdmap e364: 8 total, 8 up, 8 in 2024-09-11T07:23:47.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:46 smithi196 ceph-mon[30773]: pgmap v517: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:23:47.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:46 smithi196 ceph-mon[30773]: osdmap e365: 8 total, 8 up, 8 in 2024-09-11T07:23:47.035 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 07:23:46 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-x[32609]: ::ffff:172.21.15.196 - - [11/Sep/2024:07:23:46] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:23:47.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:46 smithi071 ceph-mon[27013]: osdmap e364: 8 total, 8 up, 8 in 2024-09-11T07:23:47.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:46 smithi071 ceph-mon[27013]: pgmap v517: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:23:47.090 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:46 smithi071 ceph-mon[27013]: osdmap e365: 8 total, 8 up, 8 in 2024-09-11T07:23:47.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:46 smithi071 ceph-mon[37727]: osdmap e364: 8 total, 8 up, 8 in 2024-09-11T07:23:47.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:46 smithi071 ceph-mon[37727]: pgmap v517: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:23:47.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:46 smithi071 ceph-mon[37727]: osdmap e365: 8 total, 8 up, 8 in 2024-09-11T07:23:49.034 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:48 smithi196 ceph-mon[30773]: pgmap v519: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:23:49.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:48 smithi196 ceph-mon[30773]: osdmap e366: 8 total, 8 up, 8 in 2024-09-11T07:23:49.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:48 smithi071 ceph-mon[27013]: pgmap v519: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:23:49.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:48 smithi071 ceph-mon[27013]: osdmap e366: 8 total, 8 up, 8 in 2024-09-11T07:23:49.089 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:48 smithi071 ceph-mon[37727]: pgmap v519: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:23:49.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:48 smithi071 ceph-mon[37727]: osdmap e366: 8 total, 8 up, 8 in 2024-09-11T07:23:50.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:49 smithi196 ceph-mon[30773]: osdmap e367: 8 total, 8 up, 8 in 2024-09-11T07:23:50.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:49 smithi196 ceph-mon[30773]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T07:23:50.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:49 smithi071 ceph-mon[27013]: osdmap e367: 8 total, 8 up, 8 in 2024-09-11T07:23:50.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:49 smithi071 ceph-mon[27013]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T07:23:50.089 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:49 smithi071 ceph-mon[37727]: osdmap e367: 8 total, 8 up, 8 in 2024-09-11T07:23:50.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:49 smithi071 ceph-mon[37727]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T07:23:51.034 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:50 smithi196 ceph-mon[30773]: pgmap v522: 196 pgs: 196 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 765 B/s wr, 2 op/s 2024-09-11T07:23:51.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:50 smithi196 ceph-mon[30773]: osdmap e368: 8 total, 8 up, 8 in 2024-09-11T07:23:51.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:50 smithi071 ceph-mon[27013]: pgmap v522: 196 pgs: 196 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 765 B/s wr, 2 op/s 2024-09-11T07:23:51.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:50 smithi071 ceph-mon[27013]: osdmap e368: 8 total, 8 up, 8 in 2024-09-11T07:23:51.089 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:50 smithi071 ceph-mon[37727]: pgmap v522: 196 pgs: 196 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 765 B/s wr, 2 op/s 2024-09-11T07:23:51.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:50 smithi071 ceph-mon[37727]: osdmap e368: 8 total, 8 up, 8 in 2024-09-11T07:23:52.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:51 smithi196 ceph-mon[30773]: osdmap e369: 8 total, 8 up, 8 in 2024-09-11T07:23:52.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:51 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/972127607' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:23:52.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:51 smithi196 ceph-mon[30773]: from='client.35153 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:23:52.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:51 smithi071 ceph-mon[27013]: osdmap e369: 8 total, 8 up, 8 in 2024-09-11T07:23:52.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:51 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/972127607' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:23:52.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:51 smithi071 ceph-mon[27013]: from='client.35153 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:23:52.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:51 smithi071 ceph-mon[37727]: osdmap e369: 8 total, 8 up, 8 in 2024-09-11T07:23:52.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:51 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/972127607' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:23:52.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:51 smithi071 ceph-mon[37727]: from='client.35153 ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-09-11T07:23:52.680 INFO:tasks.workunit.client.0.smithi071.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoCtxSelfManagedSnaps::test PASSED [ 94%] 2024-09-11T07:23:52.700 INFO:tasks.workunit.client.0.smithi071.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestCommand::test_monmap_dump PASSED [ 95%] 2024-09-11T07:23:52.717 INFO:tasks.workunit.client.0.smithi071.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestCommand::test_osd_bench PASSED [ 96%] 2024-09-11T07:23:53.034 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:52 smithi196 ceph-mon[30773]: pgmap v525: 196 pgs: 196 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-11T07:23:53.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:52 smithi196 ceph-mon[30773]: from='client.35153 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:23:53.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:52 smithi196 ceph-mon[30773]: osdmap e370: 8 total, 8 up, 8 in 2024-09-11T07:23:53.089 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:52 smithi071 ceph-mon[37727]: pgmap v525: 196 pgs: 196 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-11T07:23:53.089 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:52 smithi071 ceph-mon[37727]: from='client.35153 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:23:53.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:52 smithi071 ceph-mon[37727]: osdmap e370: 8 total, 8 up, 8 in 2024-09-11T07:23:53.090 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:52 smithi071 ceph-mon[27013]: pgmap v525: 196 pgs: 196 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-11T07:23:53.090 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:52 smithi071 ceph-mon[27013]: from='client.35153 ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-09-11T07:23:53.090 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:52 smithi071 ceph-mon[27013]: osdmap e370: 8 total, 8 up, 8 in 2024-09-11T07:23:53.701 INFO:tasks.workunit.client.0.smithi071.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestCommand::test_ceph_osd_pool_create_utf8 PASSED [ 97%] 2024-09-11T07:23:54.034 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:53 smithi196 ceph-mon[30773]: osdmap e371: 8 total, 8 up, 8 in 2024-09-11T07:23:54.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:53 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/15175673' entity='client.admin' cmd=[{"prefix": "mon dump"}]: dispatch 2024-09-11T07:23:54.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:53 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/15175673' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T07:23:54.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:53 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/15175673' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json", "epoch": 1003}]: dispatch 2024-09-11T07:23:54.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:53 smithi196 ceph-mon[30773]: from='client.? v1:172.21.15.71:0/2335065938' entity='client.admin' cmd=[{"prefix": "osd pool create", "pg_num": 16, "pool": "\u9ec5"}]: dispatch 2024-09-11T07:23:54.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:53 smithi196 ceph-mon[30773]: from='client.35171 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pg_num": 16, "pool": "\u9ec5"}]: dispatch 2024-09-11T07:23:54.089 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:53 smithi071 ceph-mon[37727]: osdmap e371: 8 total, 8 up, 8 in 2024-09-11T07:23:54.089 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:53 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/15175673' entity='client.admin' cmd=[{"prefix": "mon dump"}]: dispatch 2024-09-11T07:23:54.089 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:53 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/15175673' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T07:23:54.089 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:53 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/15175673' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json", "epoch": 1003}]: dispatch 2024-09-11T07:23:54.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:53 smithi071 ceph-mon[37727]: from='client.? v1:172.21.15.71:0/2335065938' entity='client.admin' cmd=[{"prefix": "osd pool create", "pg_num": 16, "pool": "\u9ec5"}]: dispatch 2024-09-11T07:23:54.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:53 smithi071 ceph-mon[37727]: from='client.35171 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pg_num": 16, "pool": "\u9ec5"}]: dispatch 2024-09-11T07:23:54.090 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:53 smithi071 ceph-mon[27013]: osdmap e371: 8 total, 8 up, 8 in 2024-09-11T07:23:54.090 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:53 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/15175673' entity='client.admin' cmd=[{"prefix": "mon dump"}]: dispatch 2024-09-11T07:23:54.090 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:53 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/15175673' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-11T07:23:54.090 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:53 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/15175673' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json", "epoch": 1003}]: dispatch 2024-09-11T07:23:54.090 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:53 smithi071 ceph-mon[27013]: from='client.? v1:172.21.15.71:0/2335065938' entity='client.admin' cmd=[{"prefix": "osd pool create", "pg_num": 16, "pool": "\u9ec5"}]: dispatch 2024-09-11T07:23:54.090 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:53 smithi071 ceph-mon[27013]: from='client.35171 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pg_num": 16, "pool": "\u9ec5"}]: dispatch 2024-09-11T07:23:55.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:54 smithi196 ceph-mon[30773]: pgmap v528: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:23:55.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:54 smithi196 ceph-mon[30773]: from='client.35171 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pg_num": 16, "pool": "\u9ec5"}]': finished 2024-09-11T07:23:55.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:54 smithi196 ceph-mon[30773]: osdmap e372: 8 total, 8 up, 8 in 2024-09-11T07:23:55.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:54 smithi196 ceph-mon[30773]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T07:23:55.035 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 11 07:23:54 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-iscsi-iscsi-a[58197]: debug there is no tcmu-runner data available 2024-09-11T07:23:55.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:54 smithi071 ceph-mon[27013]: pgmap v528: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:23:55.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:54 smithi071 ceph-mon[27013]: from='client.35171 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pg_num": 16, "pool": "\u9ec5"}]': finished 2024-09-11T07:23:55.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:54 smithi071 ceph-mon[27013]: osdmap e372: 8 total, 8 up, 8 in 2024-09-11T07:23:55.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:54 smithi071 ceph-mon[27013]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T07:23:55.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:54 smithi071 ceph-mon[37727]: pgmap v528: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-11T07:23:55.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:54 smithi071 ceph-mon[37727]: from='client.35171 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pg_num": 16, "pool": "\u9ec5"}]': finished 2024-09-11T07:23:55.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:54 smithi071 ceph-mon[37727]: osdmap e372: 8 total, 8 up, 8 in 2024-09-11T07:23:55.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:54 smithi071 ceph-mon[37727]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T07:23:56.034 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:55 smithi196 ceph-mon[30773]: osdmap e373: 8 total, 8 up, 8 in 2024-09-11T07:23:56.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:55 smithi196 ceph-mon[30773]: from='client.34442 v1:172.21.15.196:0/1952921461' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:23:56.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:55 smithi196 ceph-mon[30773]: osdmap e374: 8 total, 8 up, 8 in 2024-09-11T07:23:56.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:55 smithi071 ceph-mon[27013]: osdmap e373: 8 total, 8 up, 8 in 2024-09-11T07:23:56.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:55 smithi071 ceph-mon[27013]: from='client.34442 v1:172.21.15.196:0/1952921461' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:23:56.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:55 smithi071 ceph-mon[27013]: osdmap e374: 8 total, 8 up, 8 in 2024-09-11T07:23:56.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:55 smithi071 ceph-mon[37727]: osdmap e373: 8 total, 8 up, 8 in 2024-09-11T07:23:56.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:55 smithi071 ceph-mon[37727]: from='client.34442 v1:172.21.15.196:0/1952921461' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-11T07:23:56.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:55 smithi071 ceph-mon[37727]: osdmap e374: 8 total, 8 up, 8 in 2024-09-11T07:23:56.589 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:23:56 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: ::ffff:172.21.15.196 - - [11/Sep/2024:07:23:56] "GET /metrics HTTP/1.1" 200 225238 "" "Prometheus/2.43.0" 2024-09-11T07:23:57.034 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:56 smithi196 ceph-mon[30773]: pgmap v531: 212 pgs: 32 unknown, 16 creating+peering, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-09-11T07:23:57.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:56 smithi196 ceph-mon[30773]: osdmap e375: 8 total, 8 up, 8 in 2024-09-11T07:23:57.035 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 07:23:56 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-x[32609]: ::ffff:172.21.15.196 - - [11/Sep/2024:07:23:56] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:23:57.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:56 smithi071 ceph-mon[27013]: pgmap v531: 212 pgs: 32 unknown, 16 creating+peering, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-09-11T07:23:57.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:56 smithi071 ceph-mon[27013]: osdmap e375: 8 total, 8 up, 8 in 2024-09-11T07:23:57.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:56 smithi071 ceph-mon[37727]: pgmap v531: 212 pgs: 32 unknown, 16 creating+peering, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-09-11T07:23:57.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:56 smithi071 ceph-mon[37727]: osdmap e375: 8 total, 8 up, 8 in 2024-09-11T07:23:57.734 INFO:tasks.workunit.client.0.smithi071.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestWatchNotify::test PASSED [ 98%] 2024-09-11T07:23:59.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:58 smithi196 ceph-mon[30773]: pgmap v534: 212 pgs: 32 unknown, 16 creating+peering, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:23:59.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:58 smithi196 ceph-mon[30773]: osdmap e376: 8 total, 8 up, 8 in 2024-09-11T07:23:59.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:58 smithi196 ceph-mon[30773]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:23:59.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:58 smithi071 ceph-mon[27013]: pgmap v534: 212 pgs: 32 unknown, 16 creating+peering, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:23:59.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:58 smithi071 ceph-mon[27013]: osdmap e376: 8 total, 8 up, 8 in 2024-09-11T07:23:59.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:58 smithi071 ceph-mon[27013]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:23:59.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:58 smithi071 ceph-mon[37727]: pgmap v534: 212 pgs: 32 unknown, 16 creating+peering, 164 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:23:59.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:58 smithi071 ceph-mon[37727]: osdmap e376: 8 total, 8 up, 8 in 2024-09-11T07:23:59.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:58 smithi071 ceph-mon[37727]: from='mgr.34493 v1:172.21.15.71:0/1965528780' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-11T07:24:00.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:59 smithi196 ceph-mon[30773]: osdmap e377: 8 total, 8 up, 8 in 2024-09-11T07:24:00.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:23:59 smithi196 ceph-mon[30773]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T07:24:00.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:59 smithi071 ceph-mon[27013]: osdmap e377: 8 total, 8 up, 8 in 2024-09-11T07:24:00.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:23:59 smithi071 ceph-mon[27013]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T07:24:00.089 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:59 smithi071 ceph-mon[37727]: osdmap e377: 8 total, 8 up, 8 in 2024-09-11T07:24:00.089 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:23:59 smithi071 ceph-mon[37727]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-11T07:24:01.034 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:24:00 smithi196 ceph-mon[30773]: pgmap v537: 212 pgs: 32 creating+peering, 180 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:24:01.035 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:24:00 smithi196 ceph-mon[30773]: osdmap e378: 8 total, 8 up, 8 in 2024-09-11T07:24:01.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:24:00 smithi071 ceph-mon[27013]: pgmap v537: 212 pgs: 32 creating+peering, 180 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:24:01.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:24:00 smithi071 ceph-mon[27013]: osdmap e378: 8 total, 8 up, 8 in 2024-09-11T07:24:01.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:24:00 smithi071 ceph-mon[37727]: pgmap v537: 212 pgs: 32 creating+peering, 180 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-11T07:24:01.090 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:24:00 smithi071 ceph-mon[37727]: osdmap e378: 8 total, 8 up, 8 in 2024-09-11T07:24:01.764 INFO:tasks.workunit.client.0.smithi071.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestWatchNotify::test_aio_notify PASSED [100%] 2024-09-11T07:24:01.764 INFO:tasks.workunit.client.0.smithi071.stdout: 2024-09-11T07:24:01.764 INFO:tasks.workunit.client.0.smithi071.stdout:=============================== warnings summary =============================== 2024-09-11T07:24:01.765 INFO:tasks.workunit.client.0.smithi071.stdout:../../../clone.client.0/src/test/pybind/test_rados.py:210 2024-09-11T07:24:01.765 INFO:tasks.workunit.client.0.smithi071.stdout: /home/ubuntu/cephtest/clone.client.0/src/test/pybind/test_rados.py:210: DeprecationWarning: invalid escape sequence \- 2024-09-11T07:24:01.765 INFO:tasks.workunit.client.0.smithi071.stdout: assert re.match('[0-9a-f\-]{36}', fsid, re.I) 2024-09-11T07:24:01.765 INFO:tasks.workunit.client.0.smithi071.stdout: 2024-09-11T07:24:01.765 INFO:tasks.workunit.client.0.smithi071.stdout::210 2024-09-11T07:24:01.765 INFO:tasks.workunit.client.0.smithi071.stdout::210 2024-09-11T07:24:01.765 INFO:tasks.workunit.client.0.smithi071.stdout::210 2024-09-11T07:24:01.765 INFO:tasks.workunit.client.0.smithi071.stdout::210 2024-09-11T07:24:01.765 INFO:tasks.workunit.client.0.smithi071.stdout::210 2024-09-11T07:24:01.765 INFO:tasks.workunit.client.0.smithi071.stdout::210 2024-09-11T07:24:01.765 INFO:tasks.workunit.client.0.smithi071.stdout::210 2024-09-11T07:24:01.765 INFO:tasks.workunit.client.0.smithi071.stdout::210 2024-09-11T07:24:01.765 INFO:tasks.workunit.client.0.smithi071.stdout::210 2024-09-11T07:24:01.766 INFO:tasks.workunit.client.0.smithi071.stdout: :210: DeprecationWarning: invalid escape sequence \- 2024-09-11T07:24:01.766 INFO:tasks.workunit.client.0.smithi071.stdout: 2024-09-11T07:24:01.766 INFO:tasks.workunit.client.0.smithi071.stdout:-- Docs: https://docs.pytest.org/en/stable/warnings.html 2024-09-11T07:24:01.766 INFO:tasks.workunit.client.0.smithi071.stdout:================= 89 passed, 10 warnings in 319.83s (0:05:19) ================== 2024-09-11T07:24:01.789 INFO:tasks.workunit.client.0.smithi071.stderr:+ exit 0 2024-09-11T07:24:01.789 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-11T07:24:01.790 DEBUG:teuthology.orchestra.run.smithi071:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-11T07:24:01.820 INFO:tasks.workunit:Stopping ['rados/test_python.sh'] on client.0... 2024-09-11T07:24:01.820 DEBUG:teuthology.orchestra.run.smithi071:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-09-11T07:24:02.034 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:24:01 smithi196 ceph-mon[30773]: osdmap e379: 8 total, 8 up, 8 in 2024-09-11T07:24:02.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:24:01 smithi071 ceph-mon[27013]: osdmap e379: 8 total, 8 up, 8 in 2024-09-11T07:24:02.089 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:24:01 smithi071 ceph-mon[37727]: osdmap e379: 8 total, 8 up, 8 in 2024-09-11T07:24:02.129 DEBUG:teuthology.parallel:result is None 2024-09-11T07:24:02.130 DEBUG:teuthology.orchestra.run.smithi071:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-09-11T07:24:02.154 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-09-11T07:24:02.155 DEBUG:teuthology.orchestra.run.smithi071:> rmdir -- /home/ubuntu/cephtest/mnt.0 2024-09-11T07:24:02.265 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2024-09-11T07:24:02.265 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-11T07:24:02.274 INFO:tasks.cephadm:Teardown begin 2024-09-11T07:24:02.275 DEBUG:teuthology.orchestra.run.smithi071:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-11T07:24:02.300 DEBUG:teuthology.orchestra.run.smithi196:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-11T07:24:02.333 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-11T07:24:02.334 DEBUG:teuthology.orchestra.run.smithi071:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-11T07:24:02.355 DEBUG:teuthology.orchestra.run.smithi196:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-11T07:24:02.393 INFO:tasks.cephadm:Stopping all daemons... 2024-09-11T07:24:02.393 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-09-11T07:24:02.393 DEBUG:teuthology.orchestra.run.smithi071:> sudo systemctl stop ceph-a81cffe6-700c-11ef-bcea-c7b262605968@mon.a 2024-09-11T07:24:02.692 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:24:02 smithi071 systemd[1]: Stopping Ceph mon.a for a81cffe6-700c-11ef-bcea-c7b262605968... 2024-09-11T07:24:03.001 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:24:02 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: [11/Sep/2024:07:24:02] ENGINE Bus STOPPING 2024-09-11T07:24:03.002 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:24:02 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mon-a[27009]: 2024-09-11T07:24:02.690+0000 7fea96a6a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T07:24:03.002 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:24:02 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mon-a[27009]: 2024-09-11T07:24:02.690+0000 7fea96a6a640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-09-11T07:24:03.002 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:24:02 smithi071 podman[75618]: 2024-09-11 07:24:02.721774475 +0000 UTC m=+0.141357455 container died 4d63348e9b889c37707e770b9469f1b00d91f29a1cd77a6e2d735c6dcb084047 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mon-a, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2) 2024-09-11T07:24:03.277 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:24:03 smithi071 podman[75618]: 2024-09-11 07:24:03.190706032 +0000 UTC m=+0.610289000 container cleanup 4d63348e9b889c37707e770b9469f1b00d91f29a1cd77a6e2d735c6dcb084047 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mon-a, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731) 2024-09-11T07:24:03.277 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:24:03 smithi071 bash[75618]: ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mon-a 2024-09-11T07:24:03.278 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:24:03 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: [11/Sep/2024:07:24:03] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-11T07:24:03.278 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:24:03 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: [11/Sep/2024:07:24:03] ENGINE Bus STOPPED 2024-09-11T07:24:03.278 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:24:03 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: [11/Sep/2024:07:24:03] ENGINE Bus STARTING 2024-09-11T07:24:03.589 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 11 07:24:03 smithi071 podman[75631]: 2024-09-11 07:24:03.475558229 +0000 UTC m=+0.750469647 container remove 4d63348e9b889c37707e770b9469f1b00d91f29a1cd77a6e2d735c6dcb084047 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mon-a, GIT_CLEAN=True, ceph=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-11T07:24:03.589 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:24:03 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: [11/Sep/2024:07:24:03] ENGINE Serving on http://:::9283 2024-09-11T07:24:03.589 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:24:03 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: [11/Sep/2024:07:24:03] ENGINE Bus STARTED 2024-09-11T07:24:03.637 DEBUG:teuthology.orchestra.run.smithi071:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a81cffe6-700c-11ef-bcea-c7b262605968@mon.a.service' 2024-09-11T07:24:03.685 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T07:24:03.685 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-09-11T07:24:03.685 INFO:tasks.cephadm.mon.b:Stopping mon.c... 2024-09-11T07:24:03.685 DEBUG:teuthology.orchestra.run.smithi071:> sudo systemctl stop ceph-a81cffe6-700c-11ef-bcea-c7b262605968@mon.c 2024-09-11T07:24:03.981 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:24:03 smithi071 systemd[1]: Stopping Ceph mon.c for a81cffe6-700c-11ef-bcea-c7b262605968... 2024-09-11T07:24:04.283 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:24:03 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: [11/Sep/2024:07:24:03] ENGINE Bus STOPPING 2024-09-11T07:24:04.283 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:24:03 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mon-c[37723]: 2024-09-11T07:24:03.979+0000 7f89f535c640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T07:24:04.284 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:24:03 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mon-c[37723]: 2024-09-11T07:24:03.979+0000 7f89f535c640 -1 mon.c@2(peon) e3 *** Got Signal Terminated *** 2024-09-11T07:24:04.284 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:24:04 smithi071 podman[75755]: 2024-09-11 07:24:04.194653434 +0000 UTC m=+0.319791338 container died 6951ab8da37f2c8a793edd9fa65bdb202b20b2c2363e387517c0d365866c1b58 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mon-c, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.37.2, ceph=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828) 2024-09-11T07:24:04.589 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:24:04 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: [11/Sep/2024:07:24:04] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-11T07:24:04.589 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:24:04 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: [11/Sep/2024:07:24:04] ENGINE Bus STOPPED 2024-09-11T07:24:04.589 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:24:04 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: [11/Sep/2024:07:24:04] ENGINE Bus STARTING 2024-09-11T07:24:04.589 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:24:04 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: [11/Sep/2024:07:24:04] ENGINE Serving on http://:::9283 2024-09-11T07:24:04.589 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:24:04 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: [11/Sep/2024:07:24:04] ENGINE Bus STARTED 2024-09-11T07:24:05.028 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:24:04 smithi071 podman[75755]: 2024-09-11 07:24:04.621926078 +0000 UTC m=+0.747063983 container cleanup 6951ab8da37f2c8a793edd9fa65bdb202b20b2c2363e387517c0d365866c1b58 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mon-c, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T07:24:05.028 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:24:04 smithi071 bash[75755]: ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mon-c 2024-09-11T07:24:05.028 INFO:journalctl@ceph.mon.c.smithi071.stdout:Sep 11 07:24:04 smithi071 podman[75767]: 2024-09-11 07:24:04.81491056 +0000 UTC m=+0.619903153 container remove 6951ab8da37f2c8a793edd9fa65bdb202b20b2c2363e387517c0d365866c1b58 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mon-c, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731) 2024-09-11T07:24:05.050 DEBUG:teuthology.orchestra.run.smithi071:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a81cffe6-700c-11ef-bcea-c7b262605968@mon.c.service' 2024-09-11T07:24:05.083 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T07:24:05.083 INFO:tasks.cephadm.mon.b:Stopped mon.c 2024-09-11T07:24:05.084 INFO:tasks.cephadm.mon.b:Stopping mon.b... 2024-09-11T07:24:05.084 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-a81cffe6-700c-11ef-bcea-c7b262605968@mon.b 2024-09-11T07:24:05.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:24:05 smithi196 systemd[1]: Stopping Ceph mon.b for a81cffe6-700c-11ef-bcea-c7b262605968... 2024-09-11T07:24:05.285 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 11 07:24:04 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-iscsi-iscsi-a[58197]: debug there is no tcmu-runner data available 2024-09-11T07:24:05.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:24:05 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mon-b[30769]: 2024-09-11T07:24:05.383+0000 7f903ab26640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T07:24:05.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:24:05 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mon-b[30769]: 2024-09-11T07:24:05.383+0000 7f903ab26640 -1 mon.b@1(peon) e3 *** Got Signal Terminated *** 2024-09-11T07:24:05.785 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:24:05 smithi196 podman[64961]: 2024-09-11 07:24:05.628819223 +0000 UTC m=+0.355755473 container died c6c02f1aa34eabd15759e0f3e7864a73ab6878a288113942d29b276a185f94e3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mon-b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-11T07:24:06.284 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:24:06 smithi196 podman[64961]: 2024-09-11 07:24:06.083337217 +0000 UTC m=+0.810273477 container cleanup c6c02f1aa34eabd15759e0f3e7864a73ab6878a288113942d29b276a185f94e3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mon-b, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-11T07:24:06.285 INFO:journalctl@ceph.mon.b.smithi196.stdout:Sep 11 07:24:06 smithi196 bash[64961]: ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mon-b 2024-09-11T07:24:06.592 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:24:06 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: ::ffff:172.21.15.196 - - [11/Sep/2024:07:24:06] "GET /metrics HTTP/1.1" 200 225291 "" "Prometheus/2.43.0" 2024-09-11T07:24:06.630 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a81cffe6-700c-11ef-bcea-c7b262605968@mon.b.service' 2024-09-11T07:24:06.678 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T07:24:06.678 INFO:tasks.cephadm.mon.b:Stopped mon.b 2024-09-11T07:24:06.678 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-09-11T07:24:06.678 DEBUG:teuthology.orchestra.run.smithi071:> sudo systemctl stop ceph-a81cffe6-700c-11ef-bcea-c7b262605968@mgr.y 2024-09-11T07:24:06.785 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 07:24:06 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-x[32609]: ::ffff:172.21.15.196 - - [11/Sep/2024:07:24:06] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-11T07:24:06.961 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:24:06 smithi071 systemd[1]: Stopping Ceph mgr.y for a81cffe6-700c-11ef-bcea-c7b262605968... 2024-09-11T07:24:07.286 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:24:06 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: 2024-09-11T07:24:06.958+0000 7efd853c6640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.y -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T07:24:07.286 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:24:06 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y[27239]: 2024-09-11T07:24:06.958+0000 7efd853c6640 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-09-11T07:24:07.286 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:24:06 smithi071 podman[75887]: 2024-09-11 07:24:06.990687989 +0000 UTC m=+0.123538974 container died 677f18b0b24aebb06dc2008e6a45a4ea7658d9ea333d2fd55527a4345eab00a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-11T07:24:07.589 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:24:07 smithi071 podman[75887]: 2024-09-11 07:24:07.367862079 +0000 UTC m=+0.500713063 container cleanup 677f18b0b24aebb06dc2008e6a45a4ea7658d9ea333d2fd55527a4345eab00a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d0e6828a2016d48cf25ad84064e50742bb1c39b9, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-11T07:24:07.589 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Sep 11 07:24:07 smithi071 bash[75887]: ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-y 2024-09-11T07:24:07.811 DEBUG:teuthology.orchestra.run.smithi071:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a81cffe6-700c-11ef-bcea-c7b262605968@mgr.y.service' 2024-09-11T07:24:07.845 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T07:24:07.845 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-09-11T07:24:07.846 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-09-11T07:24:07.846 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-a81cffe6-700c-11ef-bcea-c7b262605968@mgr.x 2024-09-11T07:24:08.177 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 07:24:07 smithi196 systemd[1]: Stopping Ceph mgr.x for a81cffe6-700c-11ef-bcea-c7b262605968... 2024-09-11T07:24:08.438 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 07:24:08 smithi196 podman[65085]: 2024-09-11 07:24:08.175354331 +0000 UTC m=+0.130408315 container died c748882f8ce1262fc2b136638be28edd90f0800a5428d82213bdffa8a08c4c40 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-x, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T07:24:08.785 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 07:24:08 smithi196 podman[65085]: 2024-09-11 07:24:08.596873944 +0000 UTC m=+0.551927934 container cleanup c748882f8ce1262fc2b136638be28edd90f0800a5428d82213bdffa8a08c4c40 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-x, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T07:24:08.785 INFO:journalctl@ceph.mgr.x.smithi196.stdout:Sep 11 07:24:08 smithi196 bash[65085]: ceph-a81cffe6-700c-11ef-bcea-c7b262605968-mgr-x 2024-09-11T07:24:08.976 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a81cffe6-700c-11ef-bcea-c7b262605968@mgr.x.service' 2024-09-11T07:24:09.014 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T07:24:09.014 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-09-11T07:24:09.014 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-11T07:24:09.014 DEBUG:teuthology.orchestra.run.smithi071:> sudo systemctl stop ceph-a81cffe6-700c-11ef-bcea-c7b262605968@osd.0 2024-09-11T07:24:09.339 INFO:journalctl@ceph.osd.0.smithi071.stdout:Sep 11 07:24:09 smithi071 systemd[1]: Stopping Ceph osd.0 for a81cffe6-700c-11ef-bcea-c7b262605968... 2024-09-11T07:24:09.839 INFO:journalctl@ceph.osd.0.smithi071.stdout:Sep 11 07:24:09 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-0[41593]: 2024-09-11T07:24:09.547+0000 7f299f0db640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T07:24:09.839 INFO:journalctl@ceph.osd.0.smithi071.stdout:Sep 11 07:24:09 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-0[41593]: 2024-09-11T07:24:09.547+0000 7f299f0db640 -1 osd.0 380 *** Got signal Terminated *** 2024-09-11T07:24:09.839 INFO:journalctl@ceph.osd.0.smithi071.stdout:Sep 11 07:24:09 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-0[41593]: 2024-09-11T07:24:09.547+0000 7f299f0db640 -1 osd.0 380 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T07:24:15.174 INFO:journalctl@ceph.osd.0.smithi071.stdout:Sep 11 07:24:14 smithi071 podman[76011]: 2024-09-11 07:24:14.905461993 +0000 UTC m=+5.704118793 container died f5f858cd0613791a30b1c1833492e074677154c2467fab4bd000567ae9db46bc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.build-date=20240731) 2024-09-11T07:24:15.558 INFO:journalctl@ceph.osd.0.smithi071.stdout:Sep 11 07:24:15 smithi071 podman[76011]: 2024-09-11 07:24:15.263900766 +0000 UTC m=+6.062557565 container cleanup f5f858cd0613791a30b1c1833492e074677154c2467fab4bd000567ae9db46bc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-0, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T07:24:15.558 INFO:journalctl@ceph.osd.0.smithi071.stdout:Sep 11 07:24:15 smithi071 bash[76011]: ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-0 2024-09-11T07:24:15.839 INFO:journalctl@ceph.osd.0.smithi071.stdout:Sep 11 07:24:15 smithi071 podman[76028]: 2024-09-11 07:24:15.556627342 +0000 UTC m=+0.639417601 container remove f5f858cd0613791a30b1c1833492e074677154c2467fab4bd000567ae9db46bc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-11T07:24:16.198 INFO:journalctl@ceph.osd.0.smithi071.stdout:Sep 11 07:24:15 smithi071 podman[76117]: 2024-09-11 07:24:15.776803612 +0000 UTC m=+0.015940488 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40 2024-09-11T07:24:16.198 INFO:journalctl@ceph.osd.0.smithi071.stdout:Sep 11 07:24:15 smithi071 podman[76117]: 2024-09-11 07:24:15.874739723 +0000 UTC m=+0.113876596 container create 848410eb18d6e0042790b7cfe0494711d901b090844715b37ef88fbb9b339871 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-0-deactivate, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-11T07:24:16.198 INFO:journalctl@ceph.osd.0.smithi071.stdout:Sep 11 07:24:16 smithi071 podman[76117]: 2024-09-11 07:24:16.120319363 +0000 UTC m=+0.359456238 container init 848410eb18d6e0042790b7cfe0494711d901b090844715b37ef88fbb9b339871 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-0-deactivate, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T07:24:16.198 INFO:journalctl@ceph.osd.0.smithi071.stdout:Sep 11 07:24:16 smithi071 podman[76117]: 2024-09-11 07:24:16.123734947 +0000 UTC m=+0.362871823 container start 848410eb18d6e0042790b7cfe0494711d901b090844715b37ef88fbb9b339871 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T07:24:16.198 INFO:journalctl@ceph.osd.0.smithi071.stdout:Sep 11 07:24:16 smithi071 podman[76117]: 2024-09-11 07:24:16.196314814 +0000 UTC m=+0.435451700 container attach 848410eb18d6e0042790b7cfe0494711d901b090844715b37ef88fbb9b339871 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-0-deactivate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.schema-version=1.0) 2024-09-11T07:24:16.589 INFO:journalctl@ceph.osd.0.smithi071.stdout:Sep 11 07:24:16 smithi071 podman[76117]: 2024-09-11 07:24:16.307128412 +0000 UTC m=+0.546265284 container died 848410eb18d6e0042790b7cfe0494711d901b090844715b37ef88fbb9b339871 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-0-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T07:24:16.882 DEBUG:teuthology.orchestra.run.smithi071:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a81cffe6-700c-11ef-bcea-c7b262605968@osd.0.service' 2024-09-11T07:24:16.921 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T07:24:16.921 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-11T07:24:16.922 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-11T07:24:16.922 DEBUG:teuthology.orchestra.run.smithi071:> sudo systemctl stop ceph-a81cffe6-700c-11ef-bcea-c7b262605968@osd.1 2024-09-11T07:24:17.304 INFO:journalctl@ceph.osd.1.smithi071.stdout:Sep 11 07:24:17 smithi071 systemd[1]: Stopping Ceph osd.1 for a81cffe6-700c-11ef-bcea-c7b262605968... 2024-09-11T07:24:17.589 INFO:journalctl@ceph.osd.1.smithi071.stdout:Sep 11 07:24:17 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-1[47504]: 2024-09-11T07:24:17.327+0000 7fb83bf71640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T07:24:17.589 INFO:journalctl@ceph.osd.1.smithi071.stdout:Sep 11 07:24:17 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-1[47504]: 2024-09-11T07:24:17.327+0000 7fb83bf71640 -1 osd.1 380 *** Got signal Terminated *** 2024-09-11T07:24:17.589 INFO:journalctl@ceph.osd.1.smithi071.stdout:Sep 11 07:24:17 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-1[47504]: 2024-09-11T07:24:17.327+0000 7fb83bf71640 -1 osd.1 380 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T07:24:22.971 INFO:journalctl@ceph.osd.1.smithi071.stdout:Sep 11 07:24:22 smithi071 podman[76224]: 2024-09-11 07:24:22.705090328 +0000 UTC m=+5.530244932 container died 2b43d6225ed3a6140996485fe3361be7f0a154442ef0cfd28331f30359f36fcc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-1, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-09-11T07:24:23.339 INFO:journalctl@ceph.osd.1.smithi071.stdout:Sep 11 07:24:23 smithi071 podman[76224]: 2024-09-11 07:24:23.060223618 +0000 UTC m=+5.885378219 container cleanup 2b43d6225ed3a6140996485fe3361be7f0a154442ef0cfd28331f30359f36fcc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-1, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-11T07:24:23.339 INFO:journalctl@ceph.osd.1.smithi071.stdout:Sep 11 07:24:23 smithi071 bash[76224]: ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-1 2024-09-11T07:24:23.663 INFO:journalctl@ceph.osd.1.smithi071.stdout:Sep 11 07:24:23 smithi071 podman[76251]: 2024-09-11 07:24:23.353060661 +0000 UTC m=+0.649141678 container remove 2b43d6225ed3a6140996485fe3361be7f0a154442ef0cfd28331f30359f36fcc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-1, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-09-11T07:24:23.983 INFO:journalctl@ceph.osd.1.smithi071.stdout:Sep 11 07:24:23 smithi071 podman[76341]: 2024-09-11 07:24:23.662191721 +0000 UTC m=+0.102968583 container create 2e6a3852ce78efb75ee3ff79774b421f49d3b53d19059ef9e01262c19daa510d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-1-deactivate, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-09-11T07:24:23.983 INFO:journalctl@ceph.osd.1.smithi071.stdout:Sep 11 07:24:23 smithi071 podman[76341]: 2024-09-11 07:24:23.574641521 +0000 UTC m=+0.015418393 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40 2024-09-11T07:24:23.983 INFO:journalctl@ceph.osd.1.smithi071.stdout:Sep 11 07:24:23 smithi071 podman[76341]: 2024-09-11 07:24:23.86501078 +0000 UTC m=+0.305787648 container init 2e6a3852ce78efb75ee3ff79774b421f49d3b53d19059ef9e01262c19daa510d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-1-deactivate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD) 2024-09-11T07:24:23.983 INFO:journalctl@ceph.osd.1.smithi071.stdout:Sep 11 07:24:23 smithi071 podman[76341]: 2024-09-11 07:24:23.868494423 +0000 UTC m=+0.309271287 container start 2e6a3852ce78efb75ee3ff79774b421f49d3b53d19059ef9e01262c19daa510d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-1-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.37.2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-09-11T07:24:23.983 INFO:journalctl@ceph.osd.1.smithi071.stdout:Sep 11 07:24:23 smithi071 podman[76341]: 2024-09-11 07:24:23.98156699 +0000 UTC m=+0.422343853 container attach 2e6a3852ce78efb75ee3ff79774b421f49d3b53d19059ef9e01262c19daa510d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-1-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.vendor=CentOS) 2024-09-11T07:24:24.339 INFO:journalctl@ceph.osd.1.smithi071.stdout:Sep 11 07:24:24 smithi071 podman[76341]: 2024-09-11 07:24:24.053973076 +0000 UTC m=+0.494749946 container died 2e6a3852ce78efb75ee3ff79774b421f49d3b53d19059ef9e01262c19daa510d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-1-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T07:24:24.634 DEBUG:teuthology.orchestra.run.smithi071:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a81cffe6-700c-11ef-bcea-c7b262605968@osd.1.service' 2024-09-11T07:24:24.675 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T07:24:24.675 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-11T07:24:24.676 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-11T07:24:24.676 DEBUG:teuthology.orchestra.run.smithi071:> sudo systemctl stop ceph-a81cffe6-700c-11ef-bcea-c7b262605968@osd.2 2024-09-11T07:24:25.048 INFO:journalctl@ceph.osd.2.smithi071.stdout:Sep 11 07:24:24 smithi071 systemd[1]: Stopping Ceph osd.2 for a81cffe6-700c-11ef-bcea-c7b262605968... 2024-09-11T07:24:25.339 INFO:journalctl@ceph.osd.2.smithi071.stdout:Sep 11 07:24:25 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-2[53217]: 2024-09-11T07:24:25.045+0000 7fdcb594e640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T07:24:25.339 INFO:journalctl@ceph.osd.2.smithi071.stdout:Sep 11 07:24:25 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-2[53217]: 2024-09-11T07:24:25.045+0000 7fdcb594e640 -1 osd.2 380 *** Got signal Terminated *** 2024-09-11T07:24:25.339 INFO:journalctl@ceph.osd.2.smithi071.stdout:Sep 11 07:24:25 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-2[53217]: 2024-09-11T07:24:25.045+0000 7fdcb594e640 -1 osd.2 380 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T07:24:30.763 INFO:journalctl@ceph.osd.2.smithi071.stdout:Sep 11 07:24:30 smithi071 podman[76445]: 2024-09-11 07:24:30.423625293 +0000 UTC m=+5.524292100 container died bd7583671f2caf04f7de33d1444cc451df8e4791d6b278ff92494b255cd75a81 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-2, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-09-11T07:24:30.763 INFO:journalctl@ceph.osd.2.smithi071.stdout:Sep 11 07:24:30 smithi071 podman[76445]: 2024-09-11 07:24:30.761867933 +0000 UTC m=+5.862534743 container cleanup bd7583671f2caf04f7de33d1444cc451df8e4791d6b278ff92494b255cd75a81 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux ) 2024-09-11T07:24:31.089 INFO:journalctl@ceph.osd.2.smithi071.stdout:Sep 11 07:24:30 smithi071 bash[76445]: ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-2 2024-09-11T07:24:31.089 INFO:journalctl@ceph.osd.2.smithi071.stdout:Sep 11 07:24:30 smithi071 podman[76461]: 2024-09-11 07:24:30.955439638 +0000 UTC m=+0.530118764 container remove bd7583671f2caf04f7de33d1444cc451df8e4791d6b278ff92494b255cd75a81 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T07:24:31.659 INFO:journalctl@ceph.osd.2.smithi071.stdout:Sep 11 07:24:31 smithi071 podman[76551]: 2024-09-11 07:24:31.278058738 +0000 UTC m=+0.015886794 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40 2024-09-11T07:24:31.660 INFO:journalctl@ceph.osd.2.smithi071.stdout:Sep 11 07:24:31 smithi071 podman[76551]: 2024-09-11 07:24:31.381036186 +0000 UTC m=+0.118864233 container create e7fa11f59d031da93ab4bd3b4989125c876bc6f5806ab804c92e8fad6cba45eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-2-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731) 2024-09-11T07:24:31.660 INFO:journalctl@ceph.osd.2.smithi071.stdout:Sep 11 07:24:31 smithi071 podman[76551]: 2024-09-11 07:24:31.57581132 +0000 UTC m=+0.313639367 container init e7fa11f59d031da93ab4bd3b4989125c876bc6f5806ab804c92e8fad6cba45eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-2-deactivate, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, ceph=True) 2024-09-11T07:24:31.660 INFO:journalctl@ceph.osd.2.smithi071.stdout:Sep 11 07:24:31 smithi071 podman[76551]: 2024-09-11 07:24:31.579320435 +0000 UTC m=+0.317148488 container start e7fa11f59d031da93ab4bd3b4989125c876bc6f5806ab804c92e8fad6cba45eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-2-deactivate, org.label-schema.build-date=20240731, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T07:24:31.660 INFO:journalctl@ceph.osd.2.smithi071.stdout:Sep 11 07:24:31 smithi071 podman[76551]: 2024-09-11 07:24:31.658469365 +0000 UTC m=+0.396297430 container attach e7fa11f59d031da93ab4bd3b4989125c876bc6f5806ab804c92e8fad6cba45eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-2-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-09-11T07:24:32.046 INFO:journalctl@ceph.osd.2.smithi071.stdout:Sep 11 07:24:31 smithi071 podman[76551]: 2024-09-11 07:24:31.755160096 +0000 UTC m=+0.492988152 container died e7fa11f59d031da93ab4bd3b4989125c876bc6f5806ab804c92e8fad6cba45eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-2-deactivate, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-11T07:24:32.345 DEBUG:teuthology.orchestra.run.smithi071:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a81cffe6-700c-11ef-bcea-c7b262605968@osd.2.service' 2024-09-11T07:24:32.385 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T07:24:32.385 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-11T07:24:32.386 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-11T07:24:32.386 DEBUG:teuthology.orchestra.run.smithi071:> sudo systemctl stop ceph-a81cffe6-700c-11ef-bcea-c7b262605968@osd.3 2024-09-11T07:24:32.589 INFO:journalctl@ceph.osd.3.smithi071.stdout:Sep 11 07:24:32 smithi071 systemd[1]: Stopping Ceph osd.3 for a81cffe6-700c-11ef-bcea-c7b262605968... 2024-09-11T07:24:33.089 INFO:journalctl@ceph.osd.3.smithi071.stdout:Sep 11 07:24:32 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-3[59319]: 2024-09-11T07:24:32.714+0000 7f978d93d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T07:24:33.089 INFO:journalctl@ceph.osd.3.smithi071.stdout:Sep 11 07:24:32 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-3[59319]: 2024-09-11T07:24:32.714+0000 7f978d93d640 -1 osd.3 380 *** Got signal Terminated *** 2024-09-11T07:24:33.089 INFO:journalctl@ceph.osd.3.smithi071.stdout:Sep 11 07:24:32 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-3[59319]: 2024-09-11T07:24:32.714+0000 7f978d93d640 -1 osd.3 380 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T07:24:35.839 INFO:journalctl@ceph.osd.3.smithi071.stdout:Sep 11 07:24:35 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-3[59319]: 2024-09-11T07:24:35.532+0000 7f9789f55640 -1 osd.3 380 heartbeat_check: no reply from 172.21.15.71:6803 osd.0 since back 2024-09-11T07:24:10.522416+0000 front 2024-09-11T07:24:10.522434+0000 (oldest deadline 2024-09-11T07:24:35.222414+0000) 2024-09-11T07:24:36.839 INFO:journalctl@ceph.osd.3.smithi071.stdout:Sep 11 07:24:36 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-3[59319]: 2024-09-11T07:24:36.559+0000 7f9789f55640 -1 osd.3 380 heartbeat_check: no reply from 172.21.15.71:6803 osd.0 since back 2024-09-11T07:24:10.522416+0000 front 2024-09-11T07:24:10.522434+0000 (oldest deadline 2024-09-11T07:24:35.222414+0000) 2024-09-11T07:24:37.285 INFO:journalctl@ceph.osd.4.smithi196.stdout:Sep 11 07:24:36 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-4[35524]: 2024-09-11T07:24:36.788+0000 7f8cfbd6c640 -1 osd.4 380 heartbeat_check: no reply from 172.21.15.71:6803 osd.0 since back 2024-09-11T07:24:14.322709+0000 front 2024-09-11T07:24:14.322667+0000 (oldest deadline 2024-09-11T07:24:36.022637+0000) 2024-09-11T07:24:37.839 INFO:journalctl@ceph.osd.3.smithi071.stdout:Sep 11 07:24:37 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-3[59319]: 2024-09-11T07:24:37.580+0000 7f9789f55640 -1 osd.3 380 heartbeat_check: no reply from 172.21.15.71:6803 osd.0 since back 2024-09-11T07:24:10.522416+0000 front 2024-09-11T07:24:10.522434+0000 (oldest deadline 2024-09-11T07:24:35.222414+0000) 2024-09-11T07:24:38.035 INFO:journalctl@ceph.osd.4.smithi196.stdout:Sep 11 07:24:37 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-4[35524]: 2024-09-11T07:24:37.764+0000 7f8cfbd6c640 -1 osd.4 380 heartbeat_check: no reply from 172.21.15.71:6803 osd.0 since back 2024-09-11T07:24:14.322709+0000 front 2024-09-11T07:24:14.322667+0000 (oldest deadline 2024-09-11T07:24:36.022637+0000) 2024-09-11T07:24:38.325 INFO:journalctl@ceph.osd.3.smithi071.stdout:Sep 11 07:24:38 smithi071 podman[76655]: 2024-09-11 07:24:38.07328208 +0000 UTC m=+5.456744478 container died 4ae95e71ae3999c8c34d16330bf523b56135d5d91ae030a587ee525f267b87e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-3, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-09-11T07:24:38.589 INFO:journalctl@ceph.osd.3.smithi071.stdout:Sep 11 07:24:38 smithi071 podman[76655]: 2024-09-11 07:24:38.41486288 +0000 UTC m=+5.798325266 container cleanup 4ae95e71ae3999c8c34d16330bf523b56135d5d91ae030a587ee525f267b87e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-3, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True) 2024-09-11T07:24:38.589 INFO:journalctl@ceph.osd.3.smithi071.stdout:Sep 11 07:24:38 smithi071 bash[76655]: ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-3 2024-09-11T07:24:39.026 INFO:journalctl@ceph.osd.3.smithi071.stdout:Sep 11 07:24:38 smithi071 podman[76680]: 2024-09-11 07:24:38.590901268 +0000 UTC m=+0.516738084 container remove 4ae95e71ae3999c8c34d16330bf523b56135d5d91ae030a587ee525f267b87e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-3, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.vendor=CentOS) 2024-09-11T07:24:39.034 INFO:journalctl@ceph.osd.4.smithi196.stdout:Sep 11 07:24:38 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-4[35524]: 2024-09-11T07:24:38.726+0000 7f8cfbd6c640 -1 osd.4 380 heartbeat_check: no reply from 172.21.15.71:6803 osd.0 since back 2024-09-11T07:24:14.322709+0000 front 2024-09-11T07:24:14.322667+0000 (oldest deadline 2024-09-11T07:24:36.022637+0000) 2024-09-11T07:24:39.303 INFO:journalctl@ceph.osd.3.smithi071.stdout:Sep 11 07:24:39 smithi071 podman[76773]: 2024-09-11 07:24:39.02509159 +0000 UTC m=+0.110750850 container create 40df7fd35e458f441dc063dd35d8ada801613a5e528b4e955a4b28757bb2aa42 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-3-deactivate, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-09-11T07:24:39.304 INFO:journalctl@ceph.osd.3.smithi071.stdout:Sep 11 07:24:39 smithi071 podman[76773]: 2024-09-11 07:24:38.930074318 +0000 UTC m=+0.015733578 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40 2024-09-11T07:24:39.304 INFO:journalctl@ceph.osd.3.smithi071.stdout:Sep 11 07:24:39 smithi071 podman[76773]: 2024-09-11 07:24:39.219163634 +0000 UTC m=+0.304822898 container init 40df7fd35e458f441dc063dd35d8ada801613a5e528b4e955a4b28757bb2aa42 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-3-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T07:24:39.304 INFO:journalctl@ceph.osd.3.smithi071.stdout:Sep 11 07:24:39 smithi071 podman[76773]: 2024-09-11 07:24:39.222519097 +0000 UTC m=+0.308178357 container start 40df7fd35e458f441dc063dd35d8ada801613a5e528b4e955a4b28757bb2aa42 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-11T07:24:39.304 INFO:journalctl@ceph.osd.3.smithi071.stdout:Sep 11 07:24:39 smithi071 podman[76773]: 2024-09-11 07:24:39.302139246 +0000 UTC m=+0.387798519 container attach 40df7fd35e458f441dc063dd35d8ada801613a5e528b4e955a4b28757bb2aa42 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-3-deactivate, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-09-11T07:24:39.306 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 07:24:39 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-6[47249]: 2024-09-11T07:24:39.135+0000 7f4952270640 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.71:6803 osd.0 since back 2024-09-11T07:24:12.317995+0000 front 2024-09-11T07:24:12.317972+0000 (oldest deadline 2024-09-11T07:24:38.218086+0000) 2024-09-11T07:24:39.589 INFO:journalctl@ceph.osd.3.smithi071.stdout:Sep 11 07:24:39 smithi071 podman[76773]: 2024-09-11 07:24:39.400573852 +0000 UTC m=+0.486233121 container died 40df7fd35e458f441dc063dd35d8ada801613a5e528b4e955a4b28757bb2aa42 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-3-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-11T07:24:39.980 DEBUG:teuthology.orchestra.run.smithi071:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a81cffe6-700c-11ef-bcea-c7b262605968@osd.3.service' 2024-09-11T07:24:40.020 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T07:24:40.020 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-11T07:24:40.020 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-11T07:24:40.020 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-a81cffe6-700c-11ef-bcea-c7b262605968@osd.4 2024-09-11T07:24:40.034 INFO:journalctl@ceph.osd.4.smithi196.stdout:Sep 11 07:24:39 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-4[35524]: 2024-09-11T07:24:39.712+0000 7f8cfbd6c640 -1 osd.4 380 heartbeat_check: no reply from 172.21.15.71:6803 osd.0 since back 2024-09-11T07:24:14.322709+0000 front 2024-09-11T07:24:14.322667+0000 (oldest deadline 2024-09-11T07:24:36.022637+0000) 2024-09-11T07:24:40.035 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 11 07:24:39 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-5[41502]: 2024-09-11T07:24:39.908+0000 7fc9039fb640 -1 osd.5 380 heartbeat_check: no reply from 172.21.15.71:6803 osd.0 since back 2024-09-11T07:24:13.584560+0000 front 2024-09-11T07:24:13.584531+0000 (oldest deadline 2024-09-11T07:24:39.384791+0000) 2024-09-11T07:24:40.333 INFO:journalctl@ceph.osd.4.smithi196.stdout:Sep 11 07:24:40 smithi196 systemd[1]: Stopping Ceph osd.4 for a81cffe6-700c-11ef-bcea-c7b262605968... 2024-09-11T07:24:40.333 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 07:24:40 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-6[47249]: 2024-09-11T07:24:40.172+0000 7f4952270640 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.71:6803 osd.0 since back 2024-09-11T07:24:12.317995+0000 front 2024-09-11T07:24:12.317972+0000 (oldest deadline 2024-09-11T07:24:38.218086+0000) 2024-09-11T07:24:40.730 INFO:journalctl@ceph.osd.4.smithi196.stdout:Sep 11 07:24:40 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-4[35524]: 2024-09-11T07:24:40.330+0000 7f8cff754640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T07:24:40.730 INFO:journalctl@ceph.osd.4.smithi196.stdout:Sep 11 07:24:40 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-4[35524]: 2024-09-11T07:24:40.330+0000 7f8cff754640 -1 osd.4 380 *** Got signal Terminated *** 2024-09-11T07:24:40.730 INFO:journalctl@ceph.osd.4.smithi196.stdout:Sep 11 07:24:40 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-4[35524]: 2024-09-11T07:24:40.330+0000 7f8cff754640 -1 osd.4 380 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T07:24:40.730 INFO:journalctl@ceph.osd.4.smithi196.stdout:Sep 11 07:24:40 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-4[35524]: 2024-09-11T07:24:40.728+0000 7f8cfbd6c640 -1 osd.4 380 heartbeat_check: no reply from 172.21.15.71:6803 osd.0 since back 2024-09-11T07:24:14.322709+0000 front 2024-09-11T07:24:14.322667+0000 (oldest deadline 2024-09-11T07:24:36.022637+0000) 2024-09-11T07:24:41.035 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 07:24:40 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-7[53063]: 2024-09-11T07:24:40.944+0000 7f292820b640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.71:6803 osd.0 since back 2024-09-11T07:24:09.268921+0000 front 2024-09-11T07:24:09.268930+0000 (oldest deadline 2024-09-11T07:24:40.469387+0000) 2024-09-11T07:24:41.035 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 11 07:24:40 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-5[41502]: 2024-09-11T07:24:40.937+0000 7fc9039fb640 -1 osd.5 380 heartbeat_check: no reply from 172.21.15.71:6803 osd.0 since back 2024-09-11T07:24:13.584560+0000 front 2024-09-11T07:24:13.584531+0000 (oldest deadline 2024-09-11T07:24:39.384791+0000) 2024-09-11T07:24:41.535 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 07:24:41 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-6[47249]: 2024-09-11T07:24:41.160+0000 7f4952270640 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.71:6803 osd.0 since back 2024-09-11T07:24:12.317995+0000 front 2024-09-11T07:24:12.317972+0000 (oldest deadline 2024-09-11T07:24:38.218086+0000) 2024-09-11T07:24:41.974 INFO:journalctl@ceph.osd.4.smithi196.stdout:Sep 11 07:24:41 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-4[35524]: 2024-09-11T07:24:41.721+0000 7f8cfbd6c640 -1 osd.4 380 heartbeat_check: no reply from 172.21.15.71:6803 osd.0 since back 2024-09-11T07:24:14.322709+0000 front 2024-09-11T07:24:14.322667+0000 (oldest deadline 2024-09-11T07:24:36.022637+0000) 2024-09-11T07:24:42.285 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 11 07:24:41 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-5[41502]: 2024-09-11T07:24:41.973+0000 7fc9039fb640 -1 osd.5 380 heartbeat_check: no reply from 172.21.15.71:6803 osd.0 since back 2024-09-11T07:24:13.584560+0000 front 2024-09-11T07:24:13.584531+0000 (oldest deadline 2024-09-11T07:24:39.384791+0000) 2024-09-11T07:24:42.285 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 07:24:42 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-6[47249]: 2024-09-11T07:24:42.155+0000 7f4952270640 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.71:6803 osd.0 since back 2024-09-11T07:24:12.317995+0000 front 2024-09-11T07:24:12.317972+0000 (oldest deadline 2024-09-11T07:24:38.218086+0000) 2024-09-11T07:24:42.285 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 07:24:41 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-7[53063]: 2024-09-11T07:24:41.973+0000 7f292820b640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.71:6803 osd.0 since back 2024-09-11T07:24:09.268921+0000 front 2024-09-11T07:24:09.268930+0000 (oldest deadline 2024-09-11T07:24:40.469387+0000) 2024-09-11T07:24:42.952 INFO:journalctl@ceph.osd.4.smithi196.stdout:Sep 11 07:24:42 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-4[35524]: 2024-09-11T07:24:42.695+0000 7f8cfbd6c640 -1 osd.4 380 heartbeat_check: no reply from 172.21.15.71:6803 osd.0 since back 2024-09-11T07:24:14.322709+0000 front 2024-09-11T07:24:14.322667+0000 (oldest deadline 2024-09-11T07:24:36.022637+0000) 2024-09-11T07:24:42.952 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 11 07:24:42 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-5[41502]: 2024-09-11T07:24:42.944+0000 7fc9039fb640 -1 osd.5 380 heartbeat_check: no reply from 172.21.15.71:6803 osd.0 since back 2024-09-11T07:24:13.584560+0000 front 2024-09-11T07:24:13.584531+0000 (oldest deadline 2024-09-11T07:24:39.384791+0000) 2024-09-11T07:24:43.204 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 07:24:42 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-7[53063]: 2024-09-11T07:24:42.950+0000 7f292820b640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.71:6803 osd.0 since back 2024-09-11T07:24:09.268921+0000 front 2024-09-11T07:24:09.268930+0000 (oldest deadline 2024-09-11T07:24:40.469387+0000) 2024-09-11T07:24:43.204 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 07:24:42 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-7[53063]: 2024-09-11T07:24:42.950+0000 7f292820b640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.71:6807 osd.1 since back 2024-09-11T07:24:20.469798+0000 front 2024-09-11T07:24:20.469892+0000 (oldest deadline 2024-09-11T07:24:42.769808+0000) 2024-09-11T07:24:43.535 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 07:24:43 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-6[47249]: 2024-09-11T07:24:43.203+0000 7f4952270640 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.71:6803 osd.0 since back 2024-09-11T07:24:12.317995+0000 front 2024-09-11T07:24:12.317972+0000 (oldest deadline 2024-09-11T07:24:38.218086+0000) 2024-09-11T07:24:43.839 INFO:journalctl@ceph.alertmanager.a.smithi071.stdout:Sep 11 07:24:43 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-alertmanager-a[71989]: ts=2024-09-11T07:24:43.526Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi071.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.71:8443: connect: connection refused" 2024-09-11T07:24:43.839 INFO:journalctl@ceph.alertmanager.a.smithi071.stdout:Sep 11 07:24:43 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-alertmanager-a[71989]: ts=2024-09-11T07:24:43.527Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi196.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.196:8443: connect: connection refused" 2024-09-11T07:24:44.035 INFO:journalctl@ceph.osd.4.smithi196.stdout:Sep 11 07:24:43 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-4[35524]: 2024-09-11T07:24:43.741+0000 7f8cfbd6c640 -1 osd.4 380 heartbeat_check: no reply from 172.21.15.71:6803 osd.0 since back 2024-09-11T07:24:14.322709+0000 front 2024-09-11T07:24:14.322667+0000 (oldest deadline 2024-09-11T07:24:36.022637+0000) 2024-09-11T07:24:44.035 INFO:journalctl@ceph.osd.4.smithi196.stdout:Sep 11 07:24:43 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-4[35524]: 2024-09-11T07:24:43.741+0000 7f8cfbd6c640 -1 osd.4 380 heartbeat_check: no reply from 172.21.15.71:6807 osd.1 since back 2024-09-11T07:24:18.824183+0000 front 2024-09-11T07:24:18.824210+0000 (oldest deadline 2024-09-11T07:24:43.524218+0000) 2024-09-11T07:24:44.035 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 11 07:24:43 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-5[41502]: 2024-09-11T07:24:43.917+0000 7fc9039fb640 -1 osd.5 380 heartbeat_check: no reply from 172.21.15.71:6803 osd.0 since back 2024-09-11T07:24:13.584560+0000 front 2024-09-11T07:24:13.584531+0000 (oldest deadline 2024-09-11T07:24:39.384791+0000) 2024-09-11T07:24:44.035 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 07:24:43 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-7[53063]: 2024-09-11T07:24:43.930+0000 7f292820b640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.71:6803 osd.0 since back 2024-09-11T07:24:09.268921+0000 front 2024-09-11T07:24:09.268930+0000 (oldest deadline 2024-09-11T07:24:40.469387+0000) 2024-09-11T07:24:44.035 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 07:24:43 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-7[53063]: 2024-09-11T07:24:43.930+0000 7f292820b640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.71:6807 osd.1 since back 2024-09-11T07:24:20.469798+0000 front 2024-09-11T07:24:20.469892+0000 (oldest deadline 2024-09-11T07:24:42.769808+0000) 2024-09-11T07:24:44.535 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 07:24:44 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-6[47249]: 2024-09-11T07:24:44.251+0000 7f4952270640 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.71:6803 osd.0 since back 2024-09-11T07:24:12.317995+0000 front 2024-09-11T07:24:12.317972+0000 (oldest deadline 2024-09-11T07:24:38.218086+0000) 2024-09-11T07:24:44.535 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 07:24:44 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-6[47249]: 2024-09-11T07:24:44.251+0000 7f4952270640 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.71:6807 osd.1 since back 2024-09-11T07:24:20.519202+0000 front 2024-09-11T07:24:20.519133+0000 (oldest deadline 2024-09-11T07:24:44.019040+0000) 2024-09-11T07:24:45.035 INFO:journalctl@ceph.osd.4.smithi196.stdout:Sep 11 07:24:44 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-4[35524]: 2024-09-11T07:24:44.748+0000 7f8cfbd6c640 -1 osd.4 380 heartbeat_check: no reply from 172.21.15.71:6803 osd.0 since back 2024-09-11T07:24:14.322709+0000 front 2024-09-11T07:24:14.322667+0000 (oldest deadline 2024-09-11T07:24:36.022637+0000) 2024-09-11T07:24:45.035 INFO:journalctl@ceph.osd.4.smithi196.stdout:Sep 11 07:24:44 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-4[35524]: 2024-09-11T07:24:44.748+0000 7f8cfbd6c640 -1 osd.4 380 heartbeat_check: no reply from 172.21.15.71:6807 osd.1 since back 2024-09-11T07:24:18.824183+0000 front 2024-09-11T07:24:18.824210+0000 (oldest deadline 2024-09-11T07:24:43.524218+0000) 2024-09-11T07:24:45.035 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 11 07:24:44 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-5[41502]: 2024-09-11T07:24:44.895+0000 7fc9039fb640 -1 osd.5 380 heartbeat_check: no reply from 172.21.15.71:6803 osd.0 since back 2024-09-11T07:24:13.584560+0000 front 2024-09-11T07:24:13.584531+0000 (oldest deadline 2024-09-11T07:24:39.384791+0000) 2024-09-11T07:24:45.035 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 07:24:44 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-7[53063]: 2024-09-11T07:24:44.970+0000 7f292820b640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.71:6803 osd.0 since back 2024-09-11T07:24:09.268921+0000 front 2024-09-11T07:24:09.268930+0000 (oldest deadline 2024-09-11T07:24:40.469387+0000) 2024-09-11T07:24:45.036 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 07:24:44 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-7[53063]: 2024-09-11T07:24:44.970+0000 7f292820b640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.71:6807 osd.1 since back 2024-09-11T07:24:20.469798+0000 front 2024-09-11T07:24:20.469892+0000 (oldest deadline 2024-09-11T07:24:42.769808+0000) 2024-09-11T07:24:45.535 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 07:24:45 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-6[47249]: 2024-09-11T07:24:45.268+0000 7f4952270640 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.71:6803 osd.0 since back 2024-09-11T07:24:12.317995+0000 front 2024-09-11T07:24:12.317972+0000 (oldest deadline 2024-09-11T07:24:38.218086+0000) 2024-09-11T07:24:45.535 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 07:24:45 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-6[47249]: 2024-09-11T07:24:45.268+0000 7f4952270640 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.71:6807 osd.1 since back 2024-09-11T07:24:20.519202+0000 front 2024-09-11T07:24:20.519133+0000 (oldest deadline 2024-09-11T07:24:44.019040+0000) 2024-09-11T07:24:46.000 INFO:journalctl@ceph.osd.4.smithi196.stdout:Sep 11 07:24:45 smithi196 podman[65233]: 2024-09-11 07:24:45.735524607 +0000 UTC m=+5.521333138 container died a720036c239264bc8ed80787ebfdc0ccbbf3cbd1a3226602b84fe958b2c3ef97 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-4, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828) 2024-09-11T07:24:46.001 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 11 07:24:45 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-5[41502]: 2024-09-11T07:24:45.907+0000 7fc9039fb640 -1 osd.5 380 heartbeat_check: no reply from 172.21.15.71:6803 osd.0 since back 2024-09-11T07:24:13.584560+0000 front 2024-09-11T07:24:13.584531+0000 (oldest deadline 2024-09-11T07:24:39.384791+0000) 2024-09-11T07:24:46.001 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 11 07:24:45 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-5[41502]: 2024-09-11T07:24:45.907+0000 7fc9039fb640 -1 osd.5 380 heartbeat_check: no reply from 172.21.15.71:6807 osd.1 since back 2024-09-11T07:24:22.285736+0000 front 2024-09-11T07:24:22.285604+0000 (oldest deadline 2024-09-11T07:24:45.185728+0000) 2024-09-11T07:24:46.001 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 07:24:45 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-7[53063]: 2024-09-11T07:24:45.962+0000 7f292820b640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.71:6803 osd.0 since back 2024-09-11T07:24:09.268921+0000 front 2024-09-11T07:24:09.268930+0000 (oldest deadline 2024-09-11T07:24:40.469387+0000) 2024-09-11T07:24:46.001 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 07:24:45 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-7[53063]: 2024-09-11T07:24:45.962+0000 7f292820b640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.71:6807 osd.1 since back 2024-09-11T07:24:20.469798+0000 front 2024-09-11T07:24:20.469892+0000 (oldest deadline 2024-09-11T07:24:42.769808+0000) 2024-09-11T07:24:46.285 INFO:journalctl@ceph.osd.4.smithi196.stdout:Sep 11 07:24:46 smithi196 podman[65233]: 2024-09-11 07:24:46.097577605 +0000 UTC m=+5.883386120 container cleanup a720036c239264bc8ed80787ebfdc0ccbbf3cbd1a3226602b84fe958b2c3ef97 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-4, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T07:24:46.285 INFO:journalctl@ceph.osd.4.smithi196.stdout:Sep 11 07:24:46 smithi196 bash[65233]: ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-4 2024-09-11T07:24:46.713 INFO:journalctl@ceph.osd.4.smithi196.stdout:Sep 11 07:24:46 smithi196 podman[65249]: 2024-09-11 07:24:46.382386491 +0000 UTC m=+0.645427313 container remove a720036c239264bc8ed80787ebfdc0ccbbf3cbd1a3226602b84fe958b2c3ef97 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-4, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-09-11T07:24:46.713 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 07:24:46 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-6[47249]: 2024-09-11T07:24:46.299+0000 7f4952270640 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.71:6803 osd.0 since back 2024-09-11T07:24:12.317995+0000 front 2024-09-11T07:24:12.317972+0000 (oldest deadline 2024-09-11T07:24:38.218086+0000) 2024-09-11T07:24:46.714 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 07:24:46 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-6[47249]: 2024-09-11T07:24:46.299+0000 7f4952270640 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.71:6807 osd.1 since back 2024-09-11T07:24:20.519202+0000 front 2024-09-11T07:24:20.519133+0000 (oldest deadline 2024-09-11T07:24:44.019040+0000) 2024-09-11T07:24:47.002 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 11 07:24:46 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-5[41502]: 2024-09-11T07:24:46.934+0000 7fc9039fb640 -1 osd.5 380 heartbeat_check: no reply from 172.21.15.71:6803 osd.0 since back 2024-09-11T07:24:13.584560+0000 front 2024-09-11T07:24:13.584531+0000 (oldest deadline 2024-09-11T07:24:39.384791+0000) 2024-09-11T07:24:47.002 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 11 07:24:46 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-5[41502]: 2024-09-11T07:24:46.934+0000 7fc9039fb640 -1 osd.5 380 heartbeat_check: no reply from 172.21.15.71:6807 osd.1 since back 2024-09-11T07:24:22.285736+0000 front 2024-09-11T07:24:22.285604+0000 (oldest deadline 2024-09-11T07:24:45.185728+0000) 2024-09-11T07:24:47.003 INFO:journalctl@ceph.osd.4.smithi196.stdout:Sep 11 07:24:46 smithi196 podman[65339]: 2024-09-11 07:24:46.614868678 +0000 UTC m=+0.015680494 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40 2024-09-11T07:24:47.003 INFO:journalctl@ceph.osd.4.smithi196.stdout:Sep 11 07:24:46 smithi196 podman[65339]: 2024-09-11 07:24:46.741526741 +0000 UTC m=+0.142338551 container create 56062945dc5e7000f3cbf4db098533eeea421ba2827695729a285cd7e5a9a61a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-4-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-11T07:24:47.003 INFO:journalctl@ceph.osd.4.smithi196.stdout:Sep 11 07:24:46 smithi196 podman[65339]: 2024-09-11 07:24:46.935510037 +0000 UTC m=+0.336321846 container init 56062945dc5e7000f3cbf4db098533eeea421ba2827695729a285cd7e5a9a61a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-4-deactivate, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, ceph=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-11T07:24:47.003 INFO:journalctl@ceph.osd.4.smithi196.stdout:Sep 11 07:24:46 smithi196 podman[65339]: 2024-09-11 07:24:46.93883663 +0000 UTC m=+0.339648444 container start 56062945dc5e7000f3cbf4db098533eeea421ba2827695729a285cd7e5a9a61a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-4-deactivate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2) 2024-09-11T07:24:47.285 INFO:journalctl@ceph.osd.4.smithi196.stdout:Sep 11 07:24:47 smithi196 podman[65339]: 2024-09-11 07:24:47.018844884 +0000 UTC m=+0.419656718 container attach 56062945dc5e7000f3cbf4db098533eeea421ba2827695729a285cd7e5a9a61a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-4-deactivate, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731) 2024-09-11T07:24:47.285 INFO:journalctl@ceph.osd.4.smithi196.stdout:Sep 11 07:24:47 smithi196 podman[65339]: 2024-09-11 07:24:47.114967316 +0000 UTC m=+0.515779129 container died 56062945dc5e7000f3cbf4db098533eeea421ba2827695729a285cd7e5a9a61a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, ceph=True) 2024-09-11T07:24:47.285 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 07:24:47 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-7[53063]: 2024-09-11T07:24:47.001+0000 7f292820b640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.71:6803 osd.0 since back 2024-09-11T07:24:09.268921+0000 front 2024-09-11T07:24:09.268930+0000 (oldest deadline 2024-09-11T07:24:40.469387+0000) 2024-09-11T07:24:47.286 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 07:24:47 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-7[53063]: 2024-09-11T07:24:47.001+0000 7f292820b640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.71:6807 osd.1 since back 2024-09-11T07:24:20.469798+0000 front 2024-09-11T07:24:20.469892+0000 (oldest deadline 2024-09-11T07:24:42.769808+0000) 2024-09-11T07:24:47.665 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 07:24:47 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-6[47249]: 2024-09-11T07:24:47.292+0000 7f4952270640 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.71:6803 osd.0 since back 2024-09-11T07:24:12.317995+0000 front 2024-09-11T07:24:12.317972+0000 (oldest deadline 2024-09-11T07:24:38.218086+0000) 2024-09-11T07:24:47.665 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 07:24:47 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-6[47249]: 2024-09-11T07:24:47.292+0000 7f4952270640 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.71:6807 osd.1 since back 2024-09-11T07:24:20.519202+0000 front 2024-09-11T07:24:20.519133+0000 (oldest deadline 2024-09-11T07:24:44.019040+0000) 2024-09-11T07:24:47.685 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a81cffe6-700c-11ef-bcea-c7b262605968@osd.4.service' 2024-09-11T07:24:47.766 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T07:24:47.766 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-11T07:24:47.767 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-11T07:24:47.767 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-a81cffe6-700c-11ef-bcea-c7b262605968@osd.5 2024-09-11T07:24:47.989 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 11 07:24:47 smithi196 systemd[1]: Stopping Ceph osd.5 for a81cffe6-700c-11ef-bcea-c7b262605968... 2024-09-11T07:24:47.989 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 11 07:24:47 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-5[41502]: 2024-09-11T07:24:47.890+0000 7fc9039fb640 -1 osd.5 380 heartbeat_check: no reply from 172.21.15.71:6803 osd.0 since back 2024-09-11T07:24:13.584560+0000 front 2024-09-11T07:24:13.584531+0000 (oldest deadline 2024-09-11T07:24:39.384791+0000) 2024-09-11T07:24:47.989 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 11 07:24:47 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-5[41502]: 2024-09-11T07:24:47.890+0000 7fc9039fb640 -1 osd.5 380 heartbeat_check: no reply from 172.21.15.71:6807 osd.1 since back 2024-09-11T07:24:22.285736+0000 front 2024-09-11T07:24:22.285604+0000 (oldest deadline 2024-09-11T07:24:45.185728+0000) 2024-09-11T07:24:48.285 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 11 07:24:48 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-5[41502]: 2024-09-11T07:24:48.084+0000 7fc9083e5640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T07:24:48.285 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 11 07:24:48 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-5[41502]: 2024-09-11T07:24:48.084+0000 7fc9083e5640 -1 osd.5 380 *** Got signal Terminated *** 2024-09-11T07:24:48.285 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 11 07:24:48 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-5[41502]: 2024-09-11T07:24:48.084+0000 7fc9083e5640 -1 osd.5 380 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T07:24:48.285 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 07:24:47 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-7[53063]: 2024-09-11T07:24:47.987+0000 7f292820b640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.71:6803 osd.0 since back 2024-09-11T07:24:09.268921+0000 front 2024-09-11T07:24:09.268930+0000 (oldest deadline 2024-09-11T07:24:40.469387+0000) 2024-09-11T07:24:48.285 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 07:24:47 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-7[53063]: 2024-09-11T07:24:47.987+0000 7f292820b640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.71:6807 osd.1 since back 2024-09-11T07:24:20.469798+0000 front 2024-09-11T07:24:20.469892+0000 (oldest deadline 2024-09-11T07:24:42.769808+0000) 2024-09-11T07:24:48.785 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 07:24:48 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-6[47249]: 2024-09-11T07:24:48.314+0000 7f4952270640 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.71:6803 osd.0 since back 2024-09-11T07:24:12.317995+0000 front 2024-09-11T07:24:12.317972+0000 (oldest deadline 2024-09-11T07:24:38.218086+0000) 2024-09-11T07:24:48.785 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 07:24:48 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-6[47249]: 2024-09-11T07:24:48.314+0000 7f4952270640 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.71:6807 osd.1 since back 2024-09-11T07:24:20.519202+0000 front 2024-09-11T07:24:20.519133+0000 (oldest deadline 2024-09-11T07:24:44.019040+0000) 2024-09-11T07:24:49.266 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 11 07:24:48 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-5[41502]: 2024-09-11T07:24:48.934+0000 7fc9039fb640 -1 osd.5 380 heartbeat_check: no reply from 172.21.15.71:6803 osd.0 since back 2024-09-11T07:24:13.584560+0000 front 2024-09-11T07:24:13.584531+0000 (oldest deadline 2024-09-11T07:24:39.384791+0000) 2024-09-11T07:24:49.267 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 11 07:24:48 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-5[41502]: 2024-09-11T07:24:48.934+0000 7fc9039fb640 -1 osd.5 380 heartbeat_check: no reply from 172.21.15.71:6807 osd.1 since back 2024-09-11T07:24:22.285736+0000 front 2024-09-11T07:24:22.285604+0000 (oldest deadline 2024-09-11T07:24:45.185728+0000) 2024-09-11T07:24:49.267 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 07:24:48 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-7[53063]: 2024-09-11T07:24:48.965+0000 7f292820b640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.71:6803 osd.0 since back 2024-09-11T07:24:09.268921+0000 front 2024-09-11T07:24:09.268930+0000 (oldest deadline 2024-09-11T07:24:40.469387+0000) 2024-09-11T07:24:49.267 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 07:24:48 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-7[53063]: 2024-09-11T07:24:48.965+0000 7f292820b640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.71:6807 osd.1 since back 2024-09-11T07:24:20.469798+0000 front 2024-09-11T07:24:20.469892+0000 (oldest deadline 2024-09-11T07:24:42.769808+0000) 2024-09-11T07:24:49.535 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 07:24:49 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-6[47249]: 2024-09-11T07:24:49.265+0000 7f4952270640 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.71:6803 osd.0 since back 2024-09-11T07:24:12.317995+0000 front 2024-09-11T07:24:12.317972+0000 (oldest deadline 2024-09-11T07:24:38.218086+0000) 2024-09-11T07:24:49.535 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 07:24:49 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-6[47249]: 2024-09-11T07:24:49.265+0000 7f4952270640 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.71:6807 osd.1 since back 2024-09-11T07:24:20.519202+0000 front 2024-09-11T07:24:20.519133+0000 (oldest deadline 2024-09-11T07:24:44.019040+0000) 2024-09-11T07:24:50.285 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 11 07:24:49 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-5[41502]: 2024-09-11T07:24:49.932+0000 7fc9039fb640 -1 osd.5 380 heartbeat_check: no reply from 172.21.15.71:6803 osd.0 since back 2024-09-11T07:24:13.584560+0000 front 2024-09-11T07:24:13.584531+0000 (oldest deadline 2024-09-11T07:24:39.384791+0000) 2024-09-11T07:24:50.285 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 11 07:24:49 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-5[41502]: 2024-09-11T07:24:49.932+0000 7fc9039fb640 -1 osd.5 380 heartbeat_check: no reply from 172.21.15.71:6807 osd.1 since back 2024-09-11T07:24:22.285736+0000 front 2024-09-11T07:24:22.285604+0000 (oldest deadline 2024-09-11T07:24:45.185728+0000) 2024-09-11T07:24:50.285 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 07:24:49 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-7[53063]: 2024-09-11T07:24:49.918+0000 7f292820b640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.71:6803 osd.0 since back 2024-09-11T07:24:09.268921+0000 front 2024-09-11T07:24:09.268930+0000 (oldest deadline 2024-09-11T07:24:40.469387+0000) 2024-09-11T07:24:50.285 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 07:24:49 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-7[53063]: 2024-09-11T07:24:49.918+0000 7f292820b640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.71:6807 osd.1 since back 2024-09-11T07:24:20.469798+0000 front 2024-09-11T07:24:20.469892+0000 (oldest deadline 2024-09-11T07:24:42.769808+0000) 2024-09-11T07:24:50.785 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 07:24:50 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-6[47249]: 2024-09-11T07:24:50.313+0000 7f4952270640 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.71:6803 osd.0 since back 2024-09-11T07:24:12.317995+0000 front 2024-09-11T07:24:12.317972+0000 (oldest deadline 2024-09-11T07:24:38.218086+0000) 2024-09-11T07:24:50.785 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 07:24:50 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-6[47249]: 2024-09-11T07:24:50.313+0000 7f4952270640 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.71:6807 osd.1 since back 2024-09-11T07:24:20.519202+0000 front 2024-09-11T07:24:20.519133+0000 (oldest deadline 2024-09-11T07:24:44.019040+0000) 2024-09-11T07:24:51.284 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 07:24:50 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-7[53063]: 2024-09-11T07:24:50.872+0000 7f292820b640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.71:6803 osd.0 since back 2024-09-11T07:24:09.268921+0000 front 2024-09-11T07:24:09.268930+0000 (oldest deadline 2024-09-11T07:24:40.469387+0000) 2024-09-11T07:24:51.285 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 07:24:50 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-7[53063]: 2024-09-11T07:24:50.872+0000 7f292820b640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.71:6807 osd.1 since back 2024-09-11T07:24:20.469798+0000 front 2024-09-11T07:24:20.469892+0000 (oldest deadline 2024-09-11T07:24:42.769808+0000) 2024-09-11T07:24:51.285 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 11 07:24:50 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-5[41502]: 2024-09-11T07:24:50.929+0000 7fc9039fb640 -1 osd.5 380 heartbeat_check: no reply from 172.21.15.71:6803 osd.0 since back 2024-09-11T07:24:13.584560+0000 front 2024-09-11T07:24:13.584531+0000 (oldest deadline 2024-09-11T07:24:39.384791+0000) 2024-09-11T07:24:51.285 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 11 07:24:50 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-5[41502]: 2024-09-11T07:24:50.929+0000 7fc9039fb640 -1 osd.5 380 heartbeat_check: no reply from 172.21.15.71:6807 osd.1 since back 2024-09-11T07:24:22.285736+0000 front 2024-09-11T07:24:22.285604+0000 (oldest deadline 2024-09-11T07:24:45.185728+0000) 2024-09-11T07:24:51.785 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 07:24:51 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-6[47249]: 2024-09-11T07:24:51.300+0000 7f4952270640 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.71:6803 osd.0 since back 2024-09-11T07:24:12.317995+0000 front 2024-09-11T07:24:12.317972+0000 (oldest deadline 2024-09-11T07:24:38.218086+0000) 2024-09-11T07:24:51.785 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 07:24:51 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-6[47249]: 2024-09-11T07:24:51.300+0000 7f4952270640 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.71:6807 osd.1 since back 2024-09-11T07:24:20.519202+0000 front 2024-09-11T07:24:20.519133+0000 (oldest deadline 2024-09-11T07:24:44.019040+0000) 2024-09-11T07:24:51.785 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 07:24:51 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-6[47249]: 2024-09-11T07:24:51.300+0000 7f4952270640 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.71:6811 osd.2 since back 2024-09-11T07:24:26.320238+0000 front 2024-09-11T07:24:26.320193+0000 (oldest deadline 2024-09-11T07:24:50.420068+0000) 2024-09-11T07:24:52.262 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 11 07:24:51 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-5[41502]: 2024-09-11T07:24:51.970+0000 7fc9039fb640 -1 osd.5 380 heartbeat_check: no reply from 172.21.15.71:6803 osd.0 since back 2024-09-11T07:24:13.584560+0000 front 2024-09-11T07:24:13.584531+0000 (oldest deadline 2024-09-11T07:24:39.384791+0000) 2024-09-11T07:24:52.262 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 11 07:24:51 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-5[41502]: 2024-09-11T07:24:51.970+0000 7fc9039fb640 -1 osd.5 380 heartbeat_check: no reply from 172.21.15.71:6807 osd.1 since back 2024-09-11T07:24:22.285736+0000 front 2024-09-11T07:24:22.285604+0000 (oldest deadline 2024-09-11T07:24:45.185728+0000) 2024-09-11T07:24:52.262 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 07:24:51 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-7[53063]: 2024-09-11T07:24:51.888+0000 7f292820b640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.71:6803 osd.0 since back 2024-09-11T07:24:09.268921+0000 front 2024-09-11T07:24:09.268930+0000 (oldest deadline 2024-09-11T07:24:40.469387+0000) 2024-09-11T07:24:52.262 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 07:24:51 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-7[53063]: 2024-09-11T07:24:51.888+0000 7f292820b640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.71:6807 osd.1 since back 2024-09-11T07:24:20.469798+0000 front 2024-09-11T07:24:20.469892+0000 (oldest deadline 2024-09-11T07:24:42.769808+0000) 2024-09-11T07:24:52.534 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 07:24:52 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-6[47249]: 2024-09-11T07:24:52.260+0000 7f4952270640 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.71:6803 osd.0 since back 2024-09-11T07:24:12.317995+0000 front 2024-09-11T07:24:12.317972+0000 (oldest deadline 2024-09-11T07:24:38.218086+0000) 2024-09-11T07:24:52.535 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 07:24:52 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-6[47249]: 2024-09-11T07:24:52.260+0000 7f4952270640 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.71:6807 osd.1 since back 2024-09-11T07:24:20.519202+0000 front 2024-09-11T07:24:20.519133+0000 (oldest deadline 2024-09-11T07:24:44.019040+0000) 2024-09-11T07:24:52.535 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 07:24:52 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-6[47249]: 2024-09-11T07:24:52.260+0000 7f4952270640 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.71:6811 osd.2 since back 2024-09-11T07:24:26.320238+0000 front 2024-09-11T07:24:26.320193+0000 (oldest deadline 2024-09-11T07:24:50.420068+0000) 2024-09-11T07:24:53.215 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 11 07:24:52 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-5[41502]: 2024-09-11T07:24:52.933+0000 7fc9039fb640 -1 osd.5 380 heartbeat_check: no reply from 172.21.15.71:6803 osd.0 since back 2024-09-11T07:24:13.584560+0000 front 2024-09-11T07:24:13.584531+0000 (oldest deadline 2024-09-11T07:24:39.384791+0000) 2024-09-11T07:24:53.215 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 11 07:24:52 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-5[41502]: 2024-09-11T07:24:52.933+0000 7fc9039fb640 -1 osd.5 380 heartbeat_check: no reply from 172.21.15.71:6807 osd.1 since back 2024-09-11T07:24:22.285736+0000 front 2024-09-11T07:24:22.285604+0000 (oldest deadline 2024-09-11T07:24:45.185728+0000) 2024-09-11T07:24:53.216 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 07:24:52 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-7[53063]: 2024-09-11T07:24:52.874+0000 7f292820b640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.71:6803 osd.0 since back 2024-09-11T07:24:09.268921+0000 front 2024-09-11T07:24:09.268930+0000 (oldest deadline 2024-09-11T07:24:40.469387+0000) 2024-09-11T07:24:53.216 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 07:24:52 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-7[53063]: 2024-09-11T07:24:52.874+0000 7f292820b640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.71:6807 osd.1 since back 2024-09-11T07:24:20.469798+0000 front 2024-09-11T07:24:20.469892+0000 (oldest deadline 2024-09-11T07:24:42.769808+0000) 2024-09-11T07:24:53.535 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 11 07:24:53 smithi196 podman[65441]: 2024-09-11 07:24:53.40253684 +0000 UTC m=+5.413088640 container died f4e21935ad8117a6e52ea0bf280cb19d15c57c622bd50327c463aca9ed549895 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-5, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828) 2024-09-11T07:24:53.535 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 07:24:53 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-6[47249]: 2024-09-11T07:24:53.213+0000 7f4952270640 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.71:6803 osd.0 since back 2024-09-11T07:24:12.317995+0000 front 2024-09-11T07:24:12.317972+0000 (oldest deadline 2024-09-11T07:24:38.218086+0000) 2024-09-11T07:24:53.535 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 07:24:53 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-6[47249]: 2024-09-11T07:24:53.213+0000 7f4952270640 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.71:6807 osd.1 since back 2024-09-11T07:24:20.519202+0000 front 2024-09-11T07:24:20.519133+0000 (oldest deadline 2024-09-11T07:24:44.019040+0000) 2024-09-11T07:24:53.535 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 07:24:53 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-6[47249]: 2024-09-11T07:24:53.213+0000 7f4952270640 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.71:6811 osd.2 since back 2024-09-11T07:24:26.320238+0000 front 2024-09-11T07:24:26.320193+0000 (oldest deadline 2024-09-11T07:24:50.420068+0000) 2024-09-11T07:24:53.839 INFO:journalctl@ceph.alertmanager.a.smithi071.stdout:Sep 11 07:24:53 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-alertmanager-a[71989]: ts=2024-09-11T07:24:53.526Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=2 err="ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://smithi196.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.196:8443: connect: connection refused; ceph-dashboard/webhook[0]: notify retry canceled after 8 attempts: Post \"https://smithi071.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.71:8443: connect: connection refused" 2024-09-11T07:24:53.839 INFO:journalctl@ceph.alertmanager.a.smithi071.stdout:Sep 11 07:24:53 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-alertmanager-a[71989]: ts=2024-09-11T07:24:53.526Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi071.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.71:8443: connect: connection refused" 2024-09-11T07:24:53.839 INFO:journalctl@ceph.alertmanager.a.smithi071.stdout:Sep 11 07:24:53 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-alertmanager-a[71989]: ts=2024-09-11T07:24:53.527Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi196.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.196:8443: connect: connection refused" 2024-09-11T07:24:53.918 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 11 07:24:53 smithi196 podman[65441]: 2024-09-11 07:24:53.733390347 +0000 UTC m=+5.743942134 container cleanup f4e21935ad8117a6e52ea0bf280cb19d15c57c622bd50327c463aca9ed549895 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-5, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-09-11T07:24:53.918 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 11 07:24:53 smithi196 bash[65441]: ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-5 2024-09-11T07:24:54.206 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 11 07:24:53 smithi196 podman[65458]: 2024-09-11 07:24:53.926317039 +0000 UTC m=+0.521733335 container remove f4e21935ad8117a6e52ea0bf280cb19d15c57c622bd50327c463aca9ed549895 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-5, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-11T07:24:54.206 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 07:24:53 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-7[53063]: 2024-09-11T07:24:53.916+0000 7f292820b640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.71:6803 osd.0 since back 2024-09-11T07:24:09.268921+0000 front 2024-09-11T07:24:09.268930+0000 (oldest deadline 2024-09-11T07:24:40.469387+0000) 2024-09-11T07:24:54.206 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 07:24:53 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-7[53063]: 2024-09-11T07:24:53.916+0000 7f292820b640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.71:6807 osd.1 since back 2024-09-11T07:24:20.469798+0000 front 2024-09-11T07:24:20.469892+0000 (oldest deadline 2024-09-11T07:24:42.769808+0000) 2024-09-11T07:24:54.206 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 07:24:53 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-7[53063]: 2024-09-11T07:24:53.916+0000 7f292820b640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.71:6811 osd.2 since back 2024-09-11T07:24:26.870973+0000 front 2024-09-11T07:24:26.870985+0000 (oldest deadline 2024-09-11T07:24:53.271373+0000) 2024-09-11T07:24:54.465 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 11 07:24:54 smithi196 podman[65546]: 2024-09-11 07:24:54.343826296 +0000 UTC m=+0.108993977 container create 2234469148ee8cb93b32a669f1f08f6a8d5daa96ed2c497ea43f469d9f9ef778 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-5-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-11T07:24:54.466 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 11 07:24:54 smithi196 podman[65546]: 2024-09-11 07:24:54.250905995 +0000 UTC m=+0.016073676 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40 2024-09-11T07:24:54.466 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 07:24:54 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-6[47249]: 2024-09-11T07:24:54.203+0000 7f4952270640 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.71:6803 osd.0 since back 2024-09-11T07:24:12.317995+0000 front 2024-09-11T07:24:12.317972+0000 (oldest deadline 2024-09-11T07:24:38.218086+0000) 2024-09-11T07:24:54.466 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 07:24:54 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-6[47249]: 2024-09-11T07:24:54.203+0000 7f4952270640 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.71:6807 osd.1 since back 2024-09-11T07:24:20.519202+0000 front 2024-09-11T07:24:20.519133+0000 (oldest deadline 2024-09-11T07:24:44.019040+0000) 2024-09-11T07:24:54.466 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 07:24:54 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-6[47249]: 2024-09-11T07:24:54.203+0000 7f4952270640 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.71:6811 osd.2 since back 2024-09-11T07:24:26.320238+0000 front 2024-09-11T07:24:26.320193+0000 (oldest deadline 2024-09-11T07:24:50.420068+0000) 2024-09-11T07:24:54.932 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 11 07:24:54 smithi196 podman[65546]: 2024-09-11 07:24:54.554757836 +0000 UTC m=+0.319925531 container init 2234469148ee8cb93b32a669f1f08f6a8d5daa96ed2c497ea43f469d9f9ef778 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-5-deactivate, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T07:24:54.932 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 11 07:24:54 smithi196 podman[65546]: 2024-09-11 07:24:54.558245589 +0000 UTC m=+0.323413270 container start 2234469148ee8cb93b32a669f1f08f6a8d5daa96ed2c497ea43f469d9f9ef778 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-5-deactivate, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True) 2024-09-11T07:24:54.932 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 11 07:24:54 smithi196 podman[65546]: 2024-09-11 07:24:54.637803643 +0000 UTC m=+0.402971350 container attach 2234469148ee8cb93b32a669f1f08f6a8d5daa96ed2c497ea43f469d9f9ef778 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-5-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2) 2024-09-11T07:24:54.932 INFO:journalctl@ceph.osd.5.smithi196.stdout:Sep 11 07:24:54 smithi196 podman[65546]: 2024-09-11 07:24:54.732569847 +0000 UTC m=+0.497737534 container died 2234469148ee8cb93b32a669f1f08f6a8d5daa96ed2c497ea43f469d9f9ef778 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-5-deactivate, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_CLEAN=True) 2024-09-11T07:24:55.234 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 07:24:54 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-7[53063]: 2024-09-11T07:24:54.929+0000 7f292820b640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.71:6803 osd.0 since back 2024-09-11T07:24:09.268921+0000 front 2024-09-11T07:24:09.268930+0000 (oldest deadline 2024-09-11T07:24:40.469387+0000) 2024-09-11T07:24:55.235 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 07:24:54 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-7[53063]: 2024-09-11T07:24:54.929+0000 7f292820b640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.71:6807 osd.1 since back 2024-09-11T07:24:20.469798+0000 front 2024-09-11T07:24:20.469892+0000 (oldest deadline 2024-09-11T07:24:42.769808+0000) 2024-09-11T07:24:55.235 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 07:24:54 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-7[53063]: 2024-09-11T07:24:54.929+0000 7f292820b640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.71:6811 osd.2 since back 2024-09-11T07:24:26.870973+0000 front 2024-09-11T07:24:26.870985+0000 (oldest deadline 2024-09-11T07:24:53.271373+0000) 2024-09-11T07:24:55.281 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a81cffe6-700c-11ef-bcea-c7b262605968@osd.5.service' 2024-09-11T07:24:55.321 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T07:24:55.321 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-11T07:24:55.322 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-09-11T07:24:55.322 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-a81cffe6-700c-11ef-bcea-c7b262605968@osd.6 2024-09-11T07:24:55.534 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 07:24:55 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-6[47249]: 2024-09-11T07:24:55.232+0000 7f4952270640 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.71:6803 osd.0 since back 2024-09-11T07:24:12.317995+0000 front 2024-09-11T07:24:12.317972+0000 (oldest deadline 2024-09-11T07:24:38.218086+0000) 2024-09-11T07:24:55.535 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 07:24:55 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-6[47249]: 2024-09-11T07:24:55.232+0000 7f4952270640 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.71:6807 osd.1 since back 2024-09-11T07:24:20.519202+0000 front 2024-09-11T07:24:20.519133+0000 (oldest deadline 2024-09-11T07:24:44.019040+0000) 2024-09-11T07:24:55.535 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 07:24:55 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-6[47249]: 2024-09-11T07:24:55.232+0000 7f4952270640 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.71:6811 osd.2 since back 2024-09-11T07:24:26.320238+0000 front 2024-09-11T07:24:26.320193+0000 (oldest deadline 2024-09-11T07:24:50.420068+0000) 2024-09-11T07:24:55.535 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 07:24:55 smithi196 systemd[1]: Stopping Ceph osd.6 for a81cffe6-700c-11ef-bcea-c7b262605968... 2024-09-11T07:24:56.035 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 07:24:55 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-6[47249]: 2024-09-11T07:24:55.643+0000 7f4954455640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T07:24:56.035 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 07:24:55 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-6[47249]: 2024-09-11T07:24:55.643+0000 7f4954455640 -1 osd.6 380 *** Got signal Terminated *** 2024-09-11T07:24:56.035 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 07:24:55 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-6[47249]: 2024-09-11T07:24:55.643+0000 7f4954455640 -1 osd.6 380 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T07:24:56.035 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 07:24:55 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-7[53063]: 2024-09-11T07:24:55.883+0000 7f292820b640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.71:6803 osd.0 since back 2024-09-11T07:24:09.268921+0000 front 2024-09-11T07:24:09.268930+0000 (oldest deadline 2024-09-11T07:24:40.469387+0000) 2024-09-11T07:24:56.035 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 07:24:55 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-7[53063]: 2024-09-11T07:24:55.883+0000 7f292820b640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.71:6807 osd.1 since back 2024-09-11T07:24:20.469798+0000 front 2024-09-11T07:24:20.469892+0000 (oldest deadline 2024-09-11T07:24:42.769808+0000) 2024-09-11T07:24:56.035 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 07:24:55 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-7[53063]: 2024-09-11T07:24:55.883+0000 7f292820b640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.71:6811 osd.2 since back 2024-09-11T07:24:26.870973+0000 front 2024-09-11T07:24:26.870985+0000 (oldest deadline 2024-09-11T07:24:53.271373+0000) 2024-09-11T07:24:56.535 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 07:24:56 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-6[47249]: 2024-09-11T07:24:56.236+0000 7f4952270640 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.71:6803 osd.0 since back 2024-09-11T07:24:12.317995+0000 front 2024-09-11T07:24:12.317972+0000 (oldest deadline 2024-09-11T07:24:38.218086+0000) 2024-09-11T07:24:56.535 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 07:24:56 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-6[47249]: 2024-09-11T07:24:56.236+0000 7f4952270640 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.71:6807 osd.1 since back 2024-09-11T07:24:20.519202+0000 front 2024-09-11T07:24:20.519133+0000 (oldest deadline 2024-09-11T07:24:44.019040+0000) 2024-09-11T07:24:56.535 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 07:24:56 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-6[47249]: 2024-09-11T07:24:56.236+0000 7f4952270640 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.71:6811 osd.2 since back 2024-09-11T07:24:26.320238+0000 front 2024-09-11T07:24:26.320193+0000 (oldest deadline 2024-09-11T07:24:50.420068+0000) 2024-09-11T07:24:57.260 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 07:24:56 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-7[53063]: 2024-09-11T07:24:56.924+0000 7f292820b640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.71:6803 osd.0 since back 2024-09-11T07:24:09.268921+0000 front 2024-09-11T07:24:09.268930+0000 (oldest deadline 2024-09-11T07:24:40.469387+0000) 2024-09-11T07:24:57.260 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 07:24:56 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-7[53063]: 2024-09-11T07:24:56.924+0000 7f292820b640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.71:6807 osd.1 since back 2024-09-11T07:24:20.469798+0000 front 2024-09-11T07:24:20.469892+0000 (oldest deadline 2024-09-11T07:24:42.769808+0000) 2024-09-11T07:24:57.260 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 07:24:56 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-7[53063]: 2024-09-11T07:24:56.924+0000 7f292820b640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.71:6811 osd.2 since back 2024-09-11T07:24:26.870973+0000 front 2024-09-11T07:24:26.870985+0000 (oldest deadline 2024-09-11T07:24:53.271373+0000) 2024-09-11T07:24:57.534 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 07:24:57 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-6[47249]: 2024-09-11T07:24:57.258+0000 7f4952270640 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.71:6803 osd.0 since back 2024-09-11T07:24:12.317995+0000 front 2024-09-11T07:24:12.317972+0000 (oldest deadline 2024-09-11T07:24:38.218086+0000) 2024-09-11T07:24:57.535 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 07:24:57 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-6[47249]: 2024-09-11T07:24:57.258+0000 7f4952270640 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.71:6807 osd.1 since back 2024-09-11T07:24:20.519202+0000 front 2024-09-11T07:24:20.519133+0000 (oldest deadline 2024-09-11T07:24:44.019040+0000) 2024-09-11T07:24:57.535 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 07:24:57 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-6[47249]: 2024-09-11T07:24:57.258+0000 7f4952270640 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.71:6811 osd.2 since back 2024-09-11T07:24:26.320238+0000 front 2024-09-11T07:24:26.320193+0000 (oldest deadline 2024-09-11T07:24:50.420068+0000) 2024-09-11T07:24:58.246 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 07:24:57 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-7[53063]: 2024-09-11T07:24:57.945+0000 7f292820b640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.71:6803 osd.0 since back 2024-09-11T07:24:09.268921+0000 front 2024-09-11T07:24:09.268930+0000 (oldest deadline 2024-09-11T07:24:40.469387+0000) 2024-09-11T07:24:58.246 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 07:24:57 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-7[53063]: 2024-09-11T07:24:57.945+0000 7f292820b640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.71:6807 osd.1 since back 2024-09-11T07:24:20.469798+0000 front 2024-09-11T07:24:20.469892+0000 (oldest deadline 2024-09-11T07:24:42.769808+0000) 2024-09-11T07:24:58.247 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 07:24:57 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-7[53063]: 2024-09-11T07:24:57.945+0000 7f292820b640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.71:6811 osd.2 since back 2024-09-11T07:24:26.870973+0000 front 2024-09-11T07:24:26.870985+0000 (oldest deadline 2024-09-11T07:24:53.271373+0000) 2024-09-11T07:24:58.535 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 07:24:58 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-6[47249]: 2024-09-11T07:24:58.244+0000 7f4952270640 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.71:6803 osd.0 since back 2024-09-11T07:24:12.317995+0000 front 2024-09-11T07:24:12.317972+0000 (oldest deadline 2024-09-11T07:24:38.218086+0000) 2024-09-11T07:24:58.535 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 07:24:58 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-6[47249]: 2024-09-11T07:24:58.244+0000 7f4952270640 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.71:6807 osd.1 since back 2024-09-11T07:24:20.519202+0000 front 2024-09-11T07:24:20.519133+0000 (oldest deadline 2024-09-11T07:24:44.019040+0000) 2024-09-11T07:24:58.535 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 07:24:58 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-6[47249]: 2024-09-11T07:24:58.244+0000 7f4952270640 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.71:6811 osd.2 since back 2024-09-11T07:24:26.320238+0000 front 2024-09-11T07:24:26.320193+0000 (oldest deadline 2024-09-11T07:24:50.420068+0000) 2024-09-11T07:24:59.285 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 07:24:58 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-7[53063]: 2024-09-11T07:24:58.994+0000 7f292820b640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.71:6803 osd.0 since back 2024-09-11T07:24:09.268921+0000 front 2024-09-11T07:24:09.268930+0000 (oldest deadline 2024-09-11T07:24:40.469387+0000) 2024-09-11T07:24:59.285 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 07:24:58 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-7[53063]: 2024-09-11T07:24:58.994+0000 7f292820b640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.71:6807 osd.1 since back 2024-09-11T07:24:20.469798+0000 front 2024-09-11T07:24:20.469892+0000 (oldest deadline 2024-09-11T07:24:42.769808+0000) 2024-09-11T07:24:59.285 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 07:24:58 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-7[53063]: 2024-09-11T07:24:58.994+0000 7f292820b640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.71:6811 osd.2 since back 2024-09-11T07:24:26.870973+0000 front 2024-09-11T07:24:26.870985+0000 (oldest deadline 2024-09-11T07:24:53.271373+0000) 2024-09-11T07:24:59.285 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 07:24:59 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-6[47249]: 2024-09-11T07:24:59.206+0000 7f4952270640 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.71:6803 osd.0 since back 2024-09-11T07:24:12.317995+0000 front 2024-09-11T07:24:12.317972+0000 (oldest deadline 2024-09-11T07:24:38.218086+0000) 2024-09-11T07:24:59.285 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 07:24:59 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-6[47249]: 2024-09-11T07:24:59.206+0000 7f4952270640 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.71:6807 osd.1 since back 2024-09-11T07:24:20.519202+0000 front 2024-09-11T07:24:20.519133+0000 (oldest deadline 2024-09-11T07:24:44.019040+0000) 2024-09-11T07:24:59.285 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 07:24:59 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-6[47249]: 2024-09-11T07:24:59.206+0000 7f4952270640 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.71:6811 osd.2 since back 2024-09-11T07:24:26.320238+0000 front 2024-09-11T07:24:26.320193+0000 (oldest deadline 2024-09-11T07:24:50.420068+0000) 2024-09-11T07:24:59.286 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 07:24:59 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-6[47249]: 2024-09-11T07:24:59.206+0000 7f4952270640 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.71:6815 osd.3 since back 2024-09-11T07:24:32.720844+0000 front 2024-09-11T07:24:32.720871+0000 (oldest deadline 2024-09-11T07:24:58.620876+0000) 2024-09-11T07:25:00.258 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 07:24:59 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-7[53063]: 2024-09-11T07:24:59.953+0000 7f292820b640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.71:6803 osd.0 since back 2024-09-11T07:24:09.268921+0000 front 2024-09-11T07:24:09.268930+0000 (oldest deadline 2024-09-11T07:24:40.469387+0000) 2024-09-11T07:25:00.258 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 07:24:59 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-7[53063]: 2024-09-11T07:24:59.953+0000 7f292820b640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.71:6807 osd.1 since back 2024-09-11T07:24:20.469798+0000 front 2024-09-11T07:24:20.469892+0000 (oldest deadline 2024-09-11T07:24:42.769808+0000) 2024-09-11T07:25:00.258 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 07:24:59 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-7[53063]: 2024-09-11T07:24:59.953+0000 7f292820b640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.71:6811 osd.2 since back 2024-09-11T07:24:26.870973+0000 front 2024-09-11T07:24:26.870985+0000 (oldest deadline 2024-09-11T07:24:53.271373+0000) 2024-09-11T07:25:00.258 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 07:24:59 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-7[53063]: 2024-09-11T07:24:59.953+0000 7f292820b640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.71:6815 osd.3 since back 2024-09-11T07:24:34.372291+0000 front 2024-09-11T07:24:34.372314+0000 (oldest deadline 2024-09-11T07:24:59.072207+0000) 2024-09-11T07:25:00.535 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 07:25:00 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-6[47249]: 2024-09-11T07:25:00.256+0000 7f4952270640 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.71:6803 osd.0 since back 2024-09-11T07:24:12.317995+0000 front 2024-09-11T07:24:12.317972+0000 (oldest deadline 2024-09-11T07:24:38.218086+0000) 2024-09-11T07:25:00.535 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 07:25:00 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-6[47249]: 2024-09-11T07:25:00.256+0000 7f4952270640 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.71:6807 osd.1 since back 2024-09-11T07:24:20.519202+0000 front 2024-09-11T07:24:20.519133+0000 (oldest deadline 2024-09-11T07:24:44.019040+0000) 2024-09-11T07:25:00.535 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 07:25:00 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-6[47249]: 2024-09-11T07:25:00.256+0000 7f4952270640 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.71:6811 osd.2 since back 2024-09-11T07:24:26.320238+0000 front 2024-09-11T07:24:26.320193+0000 (oldest deadline 2024-09-11T07:24:50.420068+0000) 2024-09-11T07:25:00.535 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 07:25:00 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-6[47249]: 2024-09-11T07:25:00.256+0000 7f4952270640 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.71:6815 osd.3 since back 2024-09-11T07:24:32.720844+0000 front 2024-09-11T07:24:32.720871+0000 (oldest deadline 2024-09-11T07:24:58.620876+0000) 2024-09-11T07:25:01.191 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 07:25:00 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-7[53063]: 2024-09-11T07:25:00.932+0000 7f292820b640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.71:6803 osd.0 since back 2024-09-11T07:24:09.268921+0000 front 2024-09-11T07:24:09.268930+0000 (oldest deadline 2024-09-11T07:24:40.469387+0000) 2024-09-11T07:25:01.191 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 07:25:00 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-7[53063]: 2024-09-11T07:25:00.932+0000 7f292820b640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.71:6807 osd.1 since back 2024-09-11T07:24:20.469798+0000 front 2024-09-11T07:24:20.469892+0000 (oldest deadline 2024-09-11T07:24:42.769808+0000) 2024-09-11T07:25:01.191 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 07:25:00 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-7[53063]: 2024-09-11T07:25:00.932+0000 7f292820b640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.71:6811 osd.2 since back 2024-09-11T07:24:26.870973+0000 front 2024-09-11T07:24:26.870985+0000 (oldest deadline 2024-09-11T07:24:53.271373+0000) 2024-09-11T07:25:01.191 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 07:25:00 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-7[53063]: 2024-09-11T07:25:00.932+0000 7f292820b640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.71:6815 osd.3 since back 2024-09-11T07:24:34.372291+0000 front 2024-09-11T07:24:34.372314+0000 (oldest deadline 2024-09-11T07:24:59.072207+0000) 2024-09-11T07:25:01.479 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 07:25:01 smithi196 podman[65648]: 2024-09-11 07:25:01.189930339 +0000 UTC m=+5.639466316 container died fd2dca6d3a0f60c5771f8367abbddb17b2d36ff4dc627bb9a6037c8f74676684 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-6, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-09-11T07:25:01.785 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 07:25:01 smithi196 podman[65648]: 2024-09-11 07:25:01.576789074 +0000 UTC m=+6.026325056 container cleanup fd2dca6d3a0f60c5771f8367abbddb17b2d36ff4dc627bb9a6037c8f74676684 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-6, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-09-11T07:25:01.785 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 07:25:01 smithi196 bash[65648]: ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-6 2024-09-11T07:25:02.196 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 07:25:01 smithi196 podman[65664]: 2024-09-11 07:25:01.836222033 +0000 UTC m=+0.641811715 container remove fd2dca6d3a0f60c5771f8367abbddb17b2d36ff4dc627bb9a6037c8f74676684 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-6, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True) 2024-09-11T07:25:02.197 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 07:25:01 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-7[53063]: 2024-09-11T07:25:01.918+0000 7f292820b640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.71:6803 osd.0 since back 2024-09-11T07:24:09.268921+0000 front 2024-09-11T07:24:09.268930+0000 (oldest deadline 2024-09-11T07:24:40.469387+0000) 2024-09-11T07:25:02.197 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 07:25:01 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-7[53063]: 2024-09-11T07:25:01.918+0000 7f292820b640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.71:6807 osd.1 since back 2024-09-11T07:24:20.469798+0000 front 2024-09-11T07:24:20.469892+0000 (oldest deadline 2024-09-11T07:24:42.769808+0000) 2024-09-11T07:25:02.197 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 07:25:01 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-7[53063]: 2024-09-11T07:25:01.918+0000 7f292820b640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.71:6811 osd.2 since back 2024-09-11T07:24:26.870973+0000 front 2024-09-11T07:24:26.870985+0000 (oldest deadline 2024-09-11T07:24:53.271373+0000) 2024-09-11T07:25:02.197 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 07:25:01 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-7[53063]: 2024-09-11T07:25:01.918+0000 7f292820b640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.71:6815 osd.3 since back 2024-09-11T07:24:34.372291+0000 front 2024-09-11T07:24:34.372314+0000 (oldest deadline 2024-09-11T07:24:59.072207+0000) 2024-09-11T07:25:02.474 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 07:25:02 smithi196 podman[65752]: 2024-09-11 07:25:02.195405476 +0000 UTC m=+0.111982785 container create e1a1152ef78a1940666a1f442873dad1a15ebf70c9c002e5e0959e58f5200529 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-6-deactivate, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.build-date=20240731, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS) 2024-09-11T07:25:02.475 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 07:25:02 smithi196 podman[65752]: 2024-09-11 07:25:02.099340418 +0000 UTC m=+0.015917739 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40 2024-09-11T07:25:02.475 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 07:25:02 smithi196 podman[65752]: 2024-09-11 07:25:02.398332682 +0000 UTC m=+0.314909994 container init e1a1152ef78a1940666a1f442873dad1a15ebf70c9c002e5e0959e58f5200529 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-6-deactivate, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-11T07:25:02.475 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 07:25:02 smithi196 podman[65752]: 2024-09-11 07:25:02.40159921 +0000 UTC m=+0.318176522 container start e1a1152ef78a1940666a1f442873dad1a15ebf70c9c002e5e0959e58f5200529 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-6-deactivate, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240731, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T07:25:02.475 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 07:25:02 smithi196 podman[65752]: 2024-09-11 07:25:02.473150625 +0000 UTC m=+0.389727940 container attach e1a1152ef78a1940666a1f442873dad1a15ebf70c9c002e5e0959e58f5200529 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-6-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2) 2024-09-11T07:25:02.784 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 07:25:02 smithi196 podman[65752]: 2024-09-11 07:25:02.583963725 +0000 UTC m=+0.500541045 container died e1a1152ef78a1940666a1f442873dad1a15ebf70c9c002e5e0959e58f5200529 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828) 2024-09-11T07:25:03.130 INFO:journalctl@ceph.osd.6.smithi196.stdout:Sep 11 07:25:03 smithi196 podman[65775]: 2024-09-11 07:25:03.109564411 +0000 UTC m=+0.518826220 container remove e1a1152ef78a1940666a1f442873dad1a15ebf70c9c002e5e0959e58f5200529 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-6-deactivate, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-11T07:25:03.131 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 07:25:02 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-7[53063]: 2024-09-11T07:25:02.921+0000 7f292820b640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.71:6803 osd.0 since back 2024-09-11T07:24:09.268921+0000 front 2024-09-11T07:24:09.268930+0000 (oldest deadline 2024-09-11T07:24:40.469387+0000) 2024-09-11T07:25:03.131 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 07:25:02 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-7[53063]: 2024-09-11T07:25:02.921+0000 7f292820b640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.71:6807 osd.1 since back 2024-09-11T07:24:20.469798+0000 front 2024-09-11T07:24:20.469892+0000 (oldest deadline 2024-09-11T07:24:42.769808+0000) 2024-09-11T07:25:03.131 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 07:25:02 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-7[53063]: 2024-09-11T07:25:02.921+0000 7f292820b640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.71:6811 osd.2 since back 2024-09-11T07:24:26.870973+0000 front 2024-09-11T07:24:26.870985+0000 (oldest deadline 2024-09-11T07:24:53.271373+0000) 2024-09-11T07:25:03.131 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 07:25:02 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-7[53063]: 2024-09-11T07:25:02.921+0000 7f292820b640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.71:6815 osd.3 since back 2024-09-11T07:24:34.372291+0000 front 2024-09-11T07:24:34.372314+0000 (oldest deadline 2024-09-11T07:24:59.072207+0000) 2024-09-11T07:25:03.135 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a81cffe6-700c-11ef-bcea-c7b262605968@osd.6.service' 2024-09-11T07:25:03.173 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T07:25:03.174 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-09-11T07:25:03.174 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-09-11T07:25:03.174 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-a81cffe6-700c-11ef-bcea-c7b262605968@osd.7 2024-09-11T07:25:03.497 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 07:25:03 smithi196 systemd[1]: Stopping Ceph osd.7 for a81cffe6-700c-11ef-bcea-c7b262605968... 2024-09-11T07:25:03.785 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 07:25:03 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-7[53063]: 2024-09-11T07:25:03.495+0000 7f292abf1640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T07:25:03.785 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 07:25:03 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-7[53063]: 2024-09-11T07:25:03.495+0000 7f292abf1640 -1 osd.7 380 *** Got signal Terminated *** 2024-09-11T07:25:03.785 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 07:25:03 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-7[53063]: 2024-09-11T07:25:03.495+0000 7f292abf1640 -1 osd.7 380 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-11T07:25:03.839 INFO:journalctl@ceph.alertmanager.a.smithi071.stdout:Sep 11 07:25:03 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-alertmanager-a[71989]: ts=2024-09-11T07:25:03.527Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=2 err="ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://smithi071.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.71:8443: connect: connection refused; ceph-dashboard/webhook[1]: notify retry canceled after 8 attempts: Post \"https://smithi196.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.196:8443: connect: connection refused" 2024-09-11T07:25:03.839 INFO:journalctl@ceph.alertmanager.a.smithi071.stdout:Sep 11 07:25:03 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-alertmanager-a[71989]: ts=2024-09-11T07:25:03.528Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi071.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.71:8443: connect: connection refused" 2024-09-11T07:25:03.839 INFO:journalctl@ceph.alertmanager.a.smithi071.stdout:Sep 11 07:25:03 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-alertmanager-a[71989]: ts=2024-09-11T07:25:03.528Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi196.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.196:8443: connect: connection refused" 2024-09-11T07:25:04.285 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 07:25:03 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-7[53063]: 2024-09-11T07:25:03.948+0000 7f292820b640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.71:6803 osd.0 since back 2024-09-11T07:24:09.268921+0000 front 2024-09-11T07:24:09.268930+0000 (oldest deadline 2024-09-11T07:24:40.469387+0000) 2024-09-11T07:25:04.285 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 07:25:03 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-7[53063]: 2024-09-11T07:25:03.948+0000 7f292820b640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.71:6807 osd.1 since back 2024-09-11T07:24:20.469798+0000 front 2024-09-11T07:24:20.469892+0000 (oldest deadline 2024-09-11T07:24:42.769808+0000) 2024-09-11T07:25:04.285 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 07:25:03 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-7[53063]: 2024-09-11T07:25:03.948+0000 7f292820b640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.71:6811 osd.2 since back 2024-09-11T07:24:26.870973+0000 front 2024-09-11T07:24:26.870985+0000 (oldest deadline 2024-09-11T07:24:53.271373+0000) 2024-09-11T07:25:04.285 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 07:25:03 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-7[53063]: 2024-09-11T07:25:03.948+0000 7f292820b640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.71:6815 osd.3 since back 2024-09-11T07:24:34.372291+0000 front 2024-09-11T07:24:34.372314+0000 (oldest deadline 2024-09-11T07:24:59.072207+0000) 2024-09-11T07:25:05.285 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 07:25:04 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-7[53063]: 2024-09-11T07:25:04.926+0000 7f292820b640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.71:6803 osd.0 since back 2024-09-11T07:24:09.268921+0000 front 2024-09-11T07:24:09.268930+0000 (oldest deadline 2024-09-11T07:24:40.469387+0000) 2024-09-11T07:25:05.285 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 07:25:04 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-7[53063]: 2024-09-11T07:25:04.926+0000 7f292820b640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.71:6807 osd.1 since back 2024-09-11T07:24:20.469798+0000 front 2024-09-11T07:24:20.469892+0000 (oldest deadline 2024-09-11T07:24:42.769808+0000) 2024-09-11T07:25:05.285 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 07:25:04 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-7[53063]: 2024-09-11T07:25:04.926+0000 7f292820b640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.71:6811 osd.2 since back 2024-09-11T07:24:26.870973+0000 front 2024-09-11T07:24:26.870985+0000 (oldest deadline 2024-09-11T07:24:53.271373+0000) 2024-09-11T07:25:05.285 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 07:25:04 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-7[53063]: 2024-09-11T07:25:04.926+0000 7f292820b640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.71:6815 osd.3 since back 2024-09-11T07:24:34.372291+0000 front 2024-09-11T07:24:34.372314+0000 (oldest deadline 2024-09-11T07:24:59.072207+0000) 2024-09-11T07:25:06.285 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 07:25:05 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-7[53063]: 2024-09-11T07:25:05.878+0000 7f292820b640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.71:6803 osd.0 since back 2024-09-11T07:24:09.268921+0000 front 2024-09-11T07:24:09.268930+0000 (oldest deadline 2024-09-11T07:24:40.469387+0000) 2024-09-11T07:25:06.285 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 07:25:05 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-7[53063]: 2024-09-11T07:25:05.878+0000 7f292820b640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.71:6807 osd.1 since back 2024-09-11T07:24:20.469798+0000 front 2024-09-11T07:24:20.469892+0000 (oldest deadline 2024-09-11T07:24:42.769808+0000) 2024-09-11T07:25:06.285 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 07:25:05 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-7[53063]: 2024-09-11T07:25:05.878+0000 7f292820b640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.71:6811 osd.2 since back 2024-09-11T07:24:26.870973+0000 front 2024-09-11T07:24:26.870985+0000 (oldest deadline 2024-09-11T07:24:53.271373+0000) 2024-09-11T07:25:06.285 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 07:25:05 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-7[53063]: 2024-09-11T07:25:05.878+0000 7f292820b640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.71:6815 osd.3 since back 2024-09-11T07:24:34.372291+0000 front 2024-09-11T07:24:34.372314+0000 (oldest deadline 2024-09-11T07:24:59.072207+0000) 2024-09-11T07:25:07.285 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 07:25:06 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-7[53063]: 2024-09-11T07:25:06.872+0000 7f292820b640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.71:6803 osd.0 since back 2024-09-11T07:24:09.268921+0000 front 2024-09-11T07:24:09.268930+0000 (oldest deadline 2024-09-11T07:24:40.469387+0000) 2024-09-11T07:25:07.285 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 07:25:06 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-7[53063]: 2024-09-11T07:25:06.872+0000 7f292820b640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.71:6807 osd.1 since back 2024-09-11T07:24:20.469798+0000 front 2024-09-11T07:24:20.469892+0000 (oldest deadline 2024-09-11T07:24:42.769808+0000) 2024-09-11T07:25:07.285 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 07:25:06 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-7[53063]: 2024-09-11T07:25:06.872+0000 7f292820b640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.71:6811 osd.2 since back 2024-09-11T07:24:26.870973+0000 front 2024-09-11T07:24:26.870985+0000 (oldest deadline 2024-09-11T07:24:53.271373+0000) 2024-09-11T07:25:07.285 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 07:25:06 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-7[53063]: 2024-09-11T07:25:06.872+0000 7f292820b640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.71:6815 osd.3 since back 2024-09-11T07:24:34.372291+0000 front 2024-09-11T07:24:34.372314+0000 (oldest deadline 2024-09-11T07:24:59.072207+0000) 2024-09-11T07:25:08.186 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 07:25:07 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-7[53063]: 2024-09-11T07:25:07.833+0000 7f292820b640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.71:6803 osd.0 since back 2024-09-11T07:24:09.268921+0000 front 2024-09-11T07:24:09.268930+0000 (oldest deadline 2024-09-11T07:24:40.469387+0000) 2024-09-11T07:25:08.186 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 07:25:07 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-7[53063]: 2024-09-11T07:25:07.833+0000 7f292820b640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.71:6807 osd.1 since back 2024-09-11T07:24:20.469798+0000 front 2024-09-11T07:24:20.469892+0000 (oldest deadline 2024-09-11T07:24:42.769808+0000) 2024-09-11T07:25:08.186 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 07:25:07 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-7[53063]: 2024-09-11T07:25:07.833+0000 7f292820b640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.71:6811 osd.2 since back 2024-09-11T07:24:26.870973+0000 front 2024-09-11T07:24:26.870985+0000 (oldest deadline 2024-09-11T07:24:53.271373+0000) 2024-09-11T07:25:08.186 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 07:25:07 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-7[53063]: 2024-09-11T07:25:07.833+0000 7f292820b640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.71:6815 osd.3 since back 2024-09-11T07:24:34.372291+0000 front 2024-09-11T07:24:34.372314+0000 (oldest deadline 2024-09-11T07:24:59.072207+0000) 2024-09-11T07:25:08.187 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 07:25:07 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-7[53063]: 2024-09-11T07:25:07.833+0000 7f292820b640 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.196:6802 osd.4 since back 2024-09-11T07:24:43.673507+0000 front 2024-09-11T07:24:43.673398+0000 (oldest deadline 2024-09-11T07:25:07.173547+0000) 2024-09-11T07:25:09.089 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 07:25:08 smithi196 podman[65855]: 2024-09-11 07:25:08.813192511 +0000 UTC m=+5.422537363 container died 66d243d9003e5df17fcd435451976bff78f435abfbf4b2161bd212189133e631 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-7, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-11T07:25:09.431 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 07:25:09 smithi196 podman[65855]: 2024-09-11 07:25:09.178664248 +0000 UTC m=+5.788009106 container cleanup 66d243d9003e5df17fcd435451976bff78f435abfbf4b2161bd212189133e631 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-7, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-11T07:25:09.431 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 07:25:09 smithi196 bash[65855]: ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-7 2024-09-11T07:25:09.786 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 07:25:09 smithi196 podman[65881]: 2024-09-11 07:25:09.42976318 +0000 UTC m=+0.607280785 container remove 66d243d9003e5df17fcd435451976bff78f435abfbf4b2161bd212189133e631 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-7, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, ceph=True) 2024-09-11T07:25:10.068 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 07:25:09 smithi196 podman[65975]: 2024-09-11 07:25:09.69335013 +0000 UTC m=+0.015785013 image pull 4a918172a49766385959227c53e75e2c302e923539324792cb5bee6cad0b2f2b quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40 2024-09-11T07:25:10.068 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 07:25:09 smithi196 podman[65975]: 2024-09-11 07:25:09.805601922 +0000 UTC m=+0.128036805 container create e0c477aa80c66eb6261407ba466d832e7e59fbb3a0964313b6dab172922f5f89 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-7-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-11T07:25:10.068 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 07:25:09 smithi196 podman[65975]: 2024-09-11 07:25:09.999623626 +0000 UTC m=+0.322058508 container init e0c477aa80c66eb6261407ba466d832e7e59fbb3a0964313b6dab172922f5f89 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-7-deactivate, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.license=GPLv2) 2024-09-11T07:25:10.068 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 07:25:10 smithi196 podman[65975]: 2024-09-11 07:25:10.002873865 +0000 UTC m=+0.325308757 container start e0c477aa80c66eb6261407ba466d832e7e59fbb3a0964313b6dab172922f5f89 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-7-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-11T07:25:10.068 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 07:25:10 smithi196 podman[65975]: 2024-09-11 07:25:10.066210126 +0000 UTC m=+0.388645025 container attach e0c477aa80c66eb6261407ba466d832e7e59fbb3a0964313b6dab172922f5f89 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-7-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True) 2024-09-11T07:25:10.462 INFO:journalctl@ceph.osd.7.smithi196.stdout:Sep 11 07:25:10 smithi196 podman[65975]: 2024-09-11 07:25:10.178148681 +0000 UTC m=+0.500583568 container died e0c477aa80c66eb6261407ba466d832e7e59fbb3a0964313b6dab172922f5f89 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-osd-7-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-09-11T07:25:10.754 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a81cffe6-700c-11ef-bcea-c7b262605968@osd.7.service' 2024-09-11T07:25:10.794 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T07:25:10.795 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-09-11T07:25:10.795 INFO:tasks.cephadm.ceph.rgw.foo.a:Stopping rgw.foo.a... 2024-09-11T07:25:10.795 DEBUG:teuthology.orchestra.run.smithi071:> sudo systemctl stop ceph-a81cffe6-700c-11ef-bcea-c7b262605968@rgw.foo.a 2024-09-11T07:25:11.088 INFO:journalctl@ceph.rgw.foo.a.smithi071.stdout:Sep 11 07:25:10 smithi071 systemd[1]: Stopping Ceph rgw.foo.a for a81cffe6-700c-11ef-bcea-c7b262605968... 2024-09-11T07:25:11.339 INFO:journalctl@ceph.rgw.foo.a.smithi071.stdout:Sep 11 07:25:11 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-rgw-foo-a[63944]: 2024-09-11T07:25:11.086+0000 7f1c7aaac640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/radosgw -n client.rgw.foo.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-11T07:25:11.339 INFO:journalctl@ceph.rgw.foo.a.smithi071.stdout:Sep 11 07:25:11 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-rgw-foo-a[63944]: 2024-09-11T07:25:11.086+0000 7f1c7efb47c0 -1 shutting down 2024-09-11T07:25:13.839 INFO:journalctl@ceph.alertmanager.a.smithi071.stdout:Sep 11 07:25:13 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-alertmanager-a[71989]: ts=2024-09-11T07:25:13.528Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=2 err="ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://smithi071.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.71:8443: connect: connection refused; ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://smithi196.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.196:8443: connect: connection refused" 2024-09-11T07:25:13.839 INFO:journalctl@ceph.alertmanager.a.smithi071.stdout:Sep 11 07:25:13 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-alertmanager-a[71989]: ts=2024-09-11T07:25:13.528Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi071.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.71:8443: connect: connection refused" 2024-09-11T07:25:13.839 INFO:journalctl@ceph.alertmanager.a.smithi071.stdout:Sep 11 07:25:13 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-alertmanager-a[71989]: ts=2024-09-11T07:25:13.529Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi196.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.196:8443: connect: connection refused" 2024-09-11T07:25:21.589 INFO:journalctl@ceph.rgw.foo.a.smithi071.stdout:Sep 11 07:25:21 smithi071 bash[76887]: time="2024-09-11T07:25:21Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-a81cffe6-700c-11ef-bcea-c7b262605968-rgw-foo-a in 10 seconds, resorting to SIGKILL" 2024-09-11T07:25:21.589 INFO:journalctl@ceph.rgw.foo.a.smithi071.stdout:Sep 11 07:25:21 smithi071 podman[76887]: 2024-09-11 07:25:21.107617725 +0000 UTC m=+10.113500768 container died 1b92fd942af38b2ed3c429aea193881422c090d4a0ce02cc1c1699759e10ea6d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-rgw-foo-a, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-09-11T07:25:21.911 INFO:journalctl@ceph.rgw.foo.a.smithi071.stdout:Sep 11 07:25:21 smithi071 podman[76887]: 2024-09-11 07:25:21.725859092 +0000 UTC m=+10.731742133 container cleanup 1b92fd942af38b2ed3c429aea193881422c090d4a0ce02cc1c1699759e10ea6d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-rgw-foo-a, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-09-11T07:25:21.911 INFO:journalctl@ceph.rgw.foo.a.smithi071.stdout:Sep 11 07:25:21 smithi071 bash[76887]: ceph-a81cffe6-700c-11ef-bcea-c7b262605968-rgw-foo-a 2024-09-11T07:25:22.148 DEBUG:teuthology.orchestra.run.smithi071:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a81cffe6-700c-11ef-bcea-c7b262605968@rgw.foo.a.service' 2024-09-11T07:25:22.169 INFO:journalctl@ceph.rgw.foo.a.smithi071.stdout:Sep 11 07:25:21 smithi071 podman[76901]: 2024-09-11 07:25:21.910183875 +0000 UTC m=+0.800564127 container remove 1b92fd942af38b2ed3c429aea193881422c090d4a0ce02cc1c1699759e10ea6d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-rgw-foo-a, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240731, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_BRANCH=HEAD) 2024-09-11T07:25:22.169 INFO:journalctl@ceph.rgw.foo.a.smithi071.stdout:Sep 11 07:25:21 smithi071 systemd[1]: ceph-a81cffe6-700c-11ef-bcea-c7b262605968@rgw.foo.a.service: Main process exited, code=exited, status=137/n/a 2024-09-11T07:25:22.169 INFO:journalctl@ceph.rgw.foo.a.smithi071.stdout:Sep 11 07:25:22 smithi071 systemd[1]: ceph-a81cffe6-700c-11ef-bcea-c7b262605968@rgw.foo.a.service: Failed with result 'exit-code'. 2024-09-11T07:25:22.169 INFO:journalctl@ceph.rgw.foo.a.smithi071.stdout:Sep 11 07:25:22 smithi071 systemd[1]: Stopped Ceph rgw.foo.a for a81cffe6-700c-11ef-bcea-c7b262605968. 2024-09-11T07:25:22.169 INFO:journalctl@ceph.rgw.foo.a.smithi071.stdout:Sep 11 07:25:22 smithi071 systemd[1]: ceph-a81cffe6-700c-11ef-bcea-c7b262605968@rgw.foo.a.service: Consumed 2.089s CPU time. 2024-09-11T07:25:22.182 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T07:25:22.182 INFO:tasks.cephadm.ceph.rgw.foo.a:Stopped rgw.foo.a 2024-09-11T07:25:22.182 INFO:tasks.cephadm.prometheus.a:Stopping prometheus.a... 2024-09-11T07:25:22.182 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ceph-a81cffe6-700c-11ef-bcea-c7b262605968@prometheus.a 2024-09-11T07:25:22.464 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 07:25:22 smithi196 systemd[1]: Stopping Ceph prometheus.a for a81cffe6-700c-11ef-bcea-c7b262605968... 2024-09-11T07:25:22.715 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 07:25:22 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-prometheus-a[62896]: ts=2024-09-11T07:25:22.463Z caller=main.go:840 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-09-11T07:25:22.715 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 07:25:22 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-prometheus-a[62896]: ts=2024-09-11T07:25:22.463Z caller=main.go:864 level=info msg="Stopping scrape discovery manager..." 2024-09-11T07:25:22.715 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 07:25:22 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-prometheus-a[62896]: ts=2024-09-11T07:25:22.463Z caller=main.go:878 level=info msg="Stopping notify discovery manager..." 2024-09-11T07:25:22.715 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 07:25:22 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-prometheus-a[62896]: ts=2024-09-11T07:25:22.463Z caller=manager.go:988 level=info component="rule manager" msg="Stopping rule manager..." 2024-09-11T07:25:22.715 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 07:25:22 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-prometheus-a[62896]: ts=2024-09-11T07:25:22.463Z caller=main.go:860 level=info msg="Scrape discovery manager stopped" 2024-09-11T07:25:22.716 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 07:25:22 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-prometheus-a[62896]: ts=2024-09-11T07:25:22.463Z caller=manager.go:998 level=info component="rule manager" msg="Rule manager stopped" 2024-09-11T07:25:22.716 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 07:25:22 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-prometheus-a[62896]: ts=2024-09-11T07:25:22.463Z caller=main.go:915 level=info msg="Stopping scrape manager..." 2024-09-11T07:25:22.716 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 07:25:22 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-prometheus-a[62896]: ts=2024-09-11T07:25:22.463Z caller=main.go:874 level=info msg="Notify discovery manager stopped" 2024-09-11T07:25:22.716 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 07:25:22 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-prometheus-a[62896]: ts=2024-09-11T07:25:22.463Z caller=main.go:907 level=info msg="Scrape manager stopped" 2024-09-11T07:25:22.716 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 07:25:22 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-prometheus-a[62896]: ts=2024-09-11T07:25:22.463Z caller=notifier.go:606 level=info component=notifier msg="Stopping notification manager..." 2024-09-11T07:25:22.716 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 07:25:22 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-prometheus-a[62896]: ts=2024-09-11T07:25:22.463Z caller=main.go:1135 level=info msg="Notifier manager stopped" 2024-09-11T07:25:22.716 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 07:25:22 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-prometheus-a[62896]: ts=2024-09-11T07:25:22.463Z caller=main.go:1147 level=info msg="See you next time!" 2024-09-11T07:25:22.716 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 07:25:22 smithi196 podman[66077]: 2024-09-11 07:25:22.473323506 +0000 UTC m=+0.099139441 container died 026075a3cf15820e30f8392a39c1e1f8568c677ae9bb86c91063927cddab2367 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-11T07:25:23.006 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 07:25:22 smithi196 podman[66077]: 2024-09-11 07:25:22.803986056 +0000 UTC m=+0.429801989 container cleanup 026075a3cf15820e30f8392a39c1e1f8568c677ae9bb86c91063927cddab2367 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-11T07:25:23.006 INFO:journalctl@ceph.prometheus.a.smithi196.stdout:Sep 11 07:25:22 smithi196 bash[66077]: ceph-a81cffe6-700c-11ef-bcea-c7b262605968-prometheus-a 2024-09-11T07:25:23.213 DEBUG:teuthology.orchestra.run.smithi196:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a81cffe6-700c-11ef-bcea-c7b262605968@prometheus.a.service' 2024-09-11T07:25:23.246 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-11T07:25:23.246 INFO:tasks.cephadm.prometheus.a:Stopped prometheus.a 2024-09-11T07:25:23.246 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid a81cffe6-700c-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-11T07:25:23.839 INFO:journalctl@ceph.alertmanager.a.smithi071.stdout:Sep 11 07:25:23 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-alertmanager-a[71989]: ts=2024-09-11T07:25:23.529Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=3 err="ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://smithi196.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.196:8443: connect: connection refused; ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://smithi071.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.71:8443: connect: connection refused" 2024-09-11T07:25:23.840 INFO:journalctl@ceph.alertmanager.a.smithi071.stdout:Sep 11 07:25:23 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-alertmanager-a[71989]: ts=2024-09-11T07:25:23.530Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi071.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.71:8443: connect: connection refused" 2024-09-11T07:25:23.840 INFO:journalctl@ceph.alertmanager.a.smithi071.stdout:Sep 11 07:25:23 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-alertmanager-a[71989]: ts=2024-09-11T07:25:23.530Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi196.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.196:8443: connect: connection refused" 2024-09-11T07:25:25.217 INFO:journalctl@ceph.alertmanager.a.smithi071.stdout:Sep 11 07:25:24 smithi071 systemd[1]: Stopping Ceph alertmanager.a for a81cffe6-700c-11ef-bcea-c7b262605968... 2024-09-11T07:25:25.217 INFO:journalctl@ceph.alertmanager.a.smithi071.stdout:Sep 11 07:25:25 smithi071 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-alertmanager-a[71989]: ts=2024-09-11T07:25:25.164Z caller=main.go:583 level=info msg="Received SIGTERM, exiting gracefully..." 2024-09-11T07:25:25.467 INFO:journalctl@ceph.alertmanager.a.smithi071.stdout:Sep 11 07:25:25 smithi071 podman[77332]: 2024-09-11 07:25:25.215894045 +0000 UTC m=+0.149691953 container died 5a42050d984d5b52db889262f99078937223a0d065fdfae878e6d5e0c52a8944 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-11T07:25:25.834 INFO:journalctl@ceph.alertmanager.a.smithi071.stdout:Sep 11 07:25:25 smithi071 podman[77332]: 2024-09-11 07:25:25.565240987 +0000 UTC m=+0.499038942 container cleanup 5a42050d984d5b52db889262f99078937223a0d065fdfae878e6d5e0c52a8944 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-11T07:25:25.834 INFO:journalctl@ceph.alertmanager.a.smithi071.stdout:Sep 11 07:25:25 smithi071 bash[77332]: ceph-a81cffe6-700c-11ef-bcea-c7b262605968-alertmanager-a 2024-09-11T07:25:26.089 INFO:journalctl@ceph.alertmanager.a.smithi071.stdout:Sep 11 07:25:25 smithi071 podman[77346]: 2024-09-11 07:25:25.832966786 +0000 UTC m=+0.613317594 container remove 5a42050d984d5b52db889262f99078937223a0d065fdfae878e6d5e0c52a8944 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-11T07:25:26.089 INFO:journalctl@ceph.alertmanager.a.smithi071.stdout:Sep 11 07:25:25 smithi071 podman[77346]: 2024-09-11 07:25:25.90832348 +0000 UTC m=+0.688674239 volume remove 64389bfa7951215fe7a9e9d1ae2e9e2dbd350e799bacd571dff2101240e8147f 2024-09-11T07:25:26.089 INFO:journalctl@ceph.alertmanager.a.smithi071.stdout:Sep 11 07:25:26 smithi071 systemd[1]: ceph-a81cffe6-700c-11ef-bcea-c7b262605968@alertmanager.a.service: Deactivated successfully. 2024-09-11T07:25:26.089 INFO:journalctl@ceph.alertmanager.a.smithi071.stdout:Sep 11 07:25:26 smithi071 systemd[1]: Stopped Ceph alertmanager.a for a81cffe6-700c-11ef-bcea-c7b262605968. 2024-09-11T07:25:26.089 INFO:journalctl@ceph.alertmanager.a.smithi071.stdout:Sep 11 07:25:26 smithi071 systemd[1]: ceph-a81cffe6-700c-11ef-bcea-c7b262605968@alertmanager.a.service: Consumed 1.135s CPU time. 2024-09-11T07:25:26.589 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 07:25:26 smithi071 systemd[1]: Stopping Ceph node-exporter.a for a81cffe6-700c-11ef-bcea-c7b262605968... 2024-09-11T07:25:26.589 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 07:25:26 smithi071 podman[77451]: 2024-09-11 07:25:26.447227932 +0000 UTC m=+0.106551637 container died 78244d3c3309f3567517357428fdc1887942981001be75d2032ddad6d56ab7ea (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-11T07:25:27.024 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 07:25:26 smithi071 podman[77451]: 2024-09-11 07:25:26.752984192 +0000 UTC m=+0.412307893 container cleanup 78244d3c3309f3567517357428fdc1887942981001be75d2032ddad6d56ab7ea (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-11T07:25:27.024 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 07:25:26 smithi071 bash[77451]: ceph-a81cffe6-700c-11ef-bcea-c7b262605968-node-exporter-a 2024-09-11T07:25:27.301 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 07:25:27 smithi071 podman[77463]: 2024-09-11 07:25:27.020558184 +0000 UTC m=+0.577248061 container remove 78244d3c3309f3567517357428fdc1887942981001be75d2032ddad6d56ab7ea (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-11T07:25:27.301 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 07:25:27 smithi071 systemd[1]: ceph-a81cffe6-700c-11ef-bcea-c7b262605968@node-exporter.a.service: Main process exited, code=exited, status=143/n/a 2024-09-11T07:25:27.301 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 07:25:27 smithi071 systemd[1]: ceph-a81cffe6-700c-11ef-bcea-c7b262605968@node-exporter.a.service: Failed with result 'exit-code'. 2024-09-11T07:25:27.301 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 07:25:27 smithi071 systemd[1]: Stopped Ceph node-exporter.a for a81cffe6-700c-11ef-bcea-c7b262605968. 2024-09-11T07:25:27.301 INFO:journalctl@ceph.node-exporter.a.smithi071.stdout:Sep 11 07:25:27 smithi071 systemd[1]: ceph-a81cffe6-700c-11ef-bcea-c7b262605968@node-exporter.a.service: Consumed 2.364s CPU time. 2024-09-11T07:25:27.791 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid a81cffe6-700c-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-11T07:25:29.285 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 11 07:25:29 smithi196 systemd[1]: Stopping Ceph iscsi.iscsi.a for a81cffe6-700c-11ef-bcea-c7b262605968... 2024-09-11T07:25:29.784 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 11 07:25:29 smithi196 ceph-a81cffe6-700c-11ef-bcea-c7b262605968-iscsi-iscsi-a[58197]: debug Shutdown received 2024-09-11T07:25:39.650 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 11 07:25:39 smithi196 bash[66448]: time="2024-09-11T07:25:39Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-a81cffe6-700c-11ef-bcea-c7b262605968-iscsi-iscsi-a in 10 seconds, resorting to SIGKILL" 2024-09-11T07:25:39.650 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 11 07:25:39 smithi196 podman[66448]: 2024-09-11 07:25:39.388303423 +0000 UTC m=+10.147575337 container died c5945f2292e84cef88acbdf0adca973844c1e60c17e1dbd4276810f8c4226ff9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-iscsi-iscsi-a, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-09-11T07:25:39.909 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 11 07:25:39 smithi196 podman[66448]: 2024-09-11 07:25:39.731576536 +0000 UTC m=+10.490848449 container cleanup c5945f2292e84cef88acbdf0adca973844c1e60c17e1dbd4276810f8c4226ff9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-iscsi-iscsi-a, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828) 2024-09-11T07:25:39.909 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 11 07:25:39 smithi196 bash[66448]: ceph-a81cffe6-700c-11ef-bcea-c7b262605968-iscsi-iscsi-a 2024-09-11T07:25:40.239 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 11 07:25:39 smithi196 podman[66463]: 2024-09-11 07:25:39.907807796 +0000 UTC m=+0.515764204 container remove c5945f2292e84cef88acbdf0adca973844c1e60c17e1dbd4276810f8c4226ff9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-iscsi-iscsi-a, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731) 2024-09-11T07:25:40.239 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 11 07:25:39 smithi196 systemd[1]: ceph-a81cffe6-700c-11ef-bcea-c7b262605968@iscsi.iscsi.a.service: Main process exited, code=exited, status=137/n/a 2024-09-11T07:25:40.535 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 11 07:25:40 smithi196 podman[66506]: 2024-09-11 07:25:40.237820955 +0000 UTC m=+0.113322571 container died 24f941d91c9d001e7c5b3d9d88207dd417b8befd86bc1f1b498f78564e3d8160 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-iscsi-iscsi-a-tcmu, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-09-11T07:25:40.938 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 11 07:25:40 smithi196 podman[66506]: 2024-09-11 07:25:40.543408965 +0000 UTC m=+0.418910581 container cleanup 24f941d91c9d001e7c5b3d9d88207dd417b8befd86bc1f1b498f78564e3d8160 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-iscsi-iscsi-a-tcmu, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-11T07:25:40.939 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 11 07:25:40 smithi196 bash[66506]: ceph-a81cffe6-700c-11ef-bcea-c7b262605968-iscsi-iscsi-a-tcmu 2024-09-11T07:25:40.939 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 11 07:25:40 smithi196 systemd[1]: ceph-a81cffe6-700c-11ef-bcea-c7b262605968@iscsi.iscsi.a.service: Failed with result 'exit-code'. 2024-09-11T07:25:40.939 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 11 07:25:40 smithi196 systemd[1]: ceph-a81cffe6-700c-11ef-bcea-c7b262605968@iscsi.iscsi.a.service: Unit process 58076 (conmon) remains running after unit stopped. 2024-09-11T07:25:40.939 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 11 07:25:40 smithi196 systemd[1]: ceph-a81cffe6-700c-11ef-bcea-c7b262605968@iscsi.iscsi.a.service: Unit process 66519 (podman) remains running after unit stopped. 2024-09-11T07:25:40.939 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 11 07:25:40 smithi196 systemd[1]: Stopped Ceph iscsi.iscsi.a for a81cffe6-700c-11ef-bcea-c7b262605968. 2024-09-11T07:25:40.939 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 11 07:25:40 smithi196 systemd[1]: ceph-a81cffe6-700c-11ef-bcea-c7b262605968@iscsi.iscsi.a.service: Consumed 2.503s CPU time. 2024-09-11T07:25:40.939 INFO:journalctl@ceph.iscsi.iscsi.a.smithi196.stdout:Sep 11 07:25:40 smithi196 podman[66519]: 2024-09-11 07:25:40.778797962 +0000 UTC m=+0.544932970 container remove 24f941d91c9d001e7c5b3d9d88207dd417b8befd86bc1f1b498f78564e3d8160 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef9c65e2a6ea9377a732e309273ea235301c4d330f09e2972aeebc38173d3e40, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-iscsi-iscsi-a-tcmu, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-10-2008-quincy-d0e6828, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True) 2024-09-11T07:25:41.193 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:25:40 smithi196 systemd[1]: Stopping Ceph grafana.a for a81cffe6-700c-11ef-bcea-c7b262605968... 2024-09-11T07:25:41.458 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:25:41 smithi196 podman[66639]: 2024-09-11 07:25:41.191912215 +0000 UTC m=+0.117011137 container died 15a51a14b3cc3ab89dbd5c2a734cf136f8c198cc90a4f9ec764fca3f8e0e478b (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a, architecture=x86_64, io.openshift.tags=base rhel8, vcs-type=git, build-date=2022-04-29T11:48:11.055897, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, io.buildah.version=1.32.0, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., release=239.1651231664, io.k8s.display-name=Red Hat Universal Base Image 8, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, vendor=Red Hat, Inc., vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, distribution-scope=public, name=ubi8, io.openshift.expose-services=, summary=Grafana Container configured for Ceph mgr/dashboard integration, maintainer=Paul Cuzner , url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, description=Ceph Grafana Container, com.redhat.component=ubi8-container, version=8.5) 2024-09-11T07:25:41.784 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:25:41 smithi196 podman[66639]: 2024-09-11 07:25:41.539096974 +0000 UTC m=+0.464195885 container cleanup 15a51a14b3cc3ab89dbd5c2a734cf136f8c198cc90a4f9ec764fca3f8e0e478b (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a, io.k8s.display-name=Red Hat Universal Base Image 8, vcs-type=git, architecture=x86_64, io.openshift.expose-services=, vendor=Red Hat, Inc., io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., description=Ceph Grafana Container, maintainer=Paul Cuzner , release=239.1651231664, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, name=ubi8, summary=Grafana Container configured for Ceph mgr/dashboard integration, build-date=2022-04-29T11:48:11.055897, version=8.5, io.openshift.tags=base rhel8, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, io.buildah.version=1.32.0, com.redhat.component=ubi8-container, distribution-scope=public, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI) 2024-09-11T07:25:41.785 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:25:41 smithi196 bash[66639]: ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a 2024-09-11T07:25:42.155 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:25:41 smithi196 podman[66652]: 2024-09-11 07:25:41.832259359 +0000 UTC m=+0.642776739 container remove 15a51a14b3cc3ab89dbd5c2a734cf136f8c198cc90a4f9ec764fca3f8e0e478b (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-grafana-a, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, summary=Grafana Container configured for Ceph mgr/dashboard integration, vcs-type=git, architecture=x86_64, name=ubi8, maintainer=Paul Cuzner , vendor=Red Hat, Inc., io.openshift.expose-services=, description=Ceph Grafana Container, release=239.1651231664, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., io.k8s.display-name=Red Hat Universal Base Image 8, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, com.redhat.component=ubi8-container, io.buildah.version=1.32.0, io.openshift.tags=base rhel8, build-date=2022-04-29T11:48:11.055897, version=8.5, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, distribution-scope=public) 2024-09-11T07:25:42.155 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:25:41 smithi196 systemd[1]: ceph-a81cffe6-700c-11ef-bcea-c7b262605968@grafana.a.service: Main process exited, code=exited, status=143/n/a 2024-09-11T07:25:42.155 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:25:41 smithi196 systemd[1]: ceph-a81cffe6-700c-11ef-bcea-c7b262605968@grafana.a.service: Failed with result 'exit-code'. 2024-09-11T07:25:42.155 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:25:41 smithi196 systemd[1]: Stopped Ceph grafana.a for a81cffe6-700c-11ef-bcea-c7b262605968. 2024-09-11T07:25:42.156 INFO:journalctl@ceph.grafana.a.smithi196.stdout:Sep 11 07:25:41 smithi196 systemd[1]: ceph-a81cffe6-700c-11ef-bcea-c7b262605968@grafana.a.service: Consumed 6.295s CPU time. 2024-09-11T07:25:42.422 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 07:25:42 smithi196 systemd[1]: Stopping Ceph node-exporter.b for a81cffe6-700c-11ef-bcea-c7b262605968... 2024-09-11T07:25:42.678 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 07:25:42 smithi196 podman[66770]: 2024-09-11 07:25:42.421244253 +0000 UTC m=+0.096237001 container died 2d67aba24099ae86cabd97bffecf07402c64547541a91f89c29d8cff6c9181b3 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-11T07:25:42.937 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 07:25:42 smithi196 podman[66770]: 2024-09-11 07:25:42.760362143 +0000 UTC m=+0.435354889 container cleanup 2d67aba24099ae86cabd97bffecf07402c64547541a91f89c29d8cff6c9181b3 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-11T07:25:42.937 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 07:25:42 smithi196 bash[66770]: ceph-a81cffe6-700c-11ef-bcea-c7b262605968-node-exporter-b 2024-09-11T07:25:43.285 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 07:25:42 smithi196 podman[66785]: 2024-09-11 07:25:42.936183114 +0000 UTC m=+0.515709766 container remove 2d67aba24099ae86cabd97bffecf07402c64547541a91f89c29d8cff6c9181b3 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-a81cffe6-700c-11ef-bcea-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-11T07:25:43.285 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 07:25:42 smithi196 systemd[1]: ceph-a81cffe6-700c-11ef-bcea-c7b262605968@node-exporter.b.service: Main process exited, code=exited, status=143/n/a 2024-09-11T07:25:43.285 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 07:25:43 smithi196 systemd[1]: ceph-a81cffe6-700c-11ef-bcea-c7b262605968@node-exporter.b.service: Failed with result 'exit-code'. 2024-09-11T07:25:43.285 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 07:25:43 smithi196 systemd[1]: Stopped Ceph node-exporter.b for a81cffe6-700c-11ef-bcea-c7b262605968. 2024-09-11T07:25:43.285 INFO:journalctl@ceph.node-exporter.b.smithi196.stdout:Sep 11 07:25:43 smithi196 systemd[1]: ceph-a81cffe6-700c-11ef-bcea-c7b262605968@node-exporter.b.service: Consumed 2.425s CPU time. 2024-09-11T07:25:43.790 DEBUG:teuthology.orchestra.run.smithi071:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-11T07:25:43.824 DEBUG:teuthology.orchestra.run.smithi196:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-11T07:25:43.851 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-11T07:25:43.854 DEBUG:teuthology.misc:Transferring archived files from smithi071:/var/lib/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/gabrioux-2024-09-11_06:53:43-orch:cephadm-wip-guits-quincy-2024-09-10-2008-quincy-distro-default-smithi/7901202/remote/smithi071/crash 2024-09-11T07:25:43.855 DEBUG:teuthology.orchestra.run.smithi071:> sudo tar c -f - -C /var/lib/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/crash -- . 2024-09-11T07:25:43.905 INFO:teuthology.orchestra.run.smithi071.stderr:tar: /var/lib/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-11T07:25:43.905 INFO:teuthology.orchestra.run.smithi071.stderr:tar: Error is not recoverable: exiting now 2024-09-11T07:25:43.907 DEBUG:teuthology.misc:Transferring archived files from smithi196:/var/lib/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/gabrioux-2024-09-11_06:53:43-orch:cephadm-wip-guits-quincy-2024-09-10-2008-quincy-distro-default-smithi/7901202/remote/smithi196/crash 2024-09-11T07:25:43.908 DEBUG:teuthology.orchestra.run.smithi196:> sudo tar c -f - -C /var/lib/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/crash -- . 2024-09-11T07:25:43.950 INFO:teuthology.orchestra.run.smithi196.stderr:tar: /var/lib/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-11T07:25:43.950 INFO:teuthology.orchestra.run.smithi196.stderr:tar: Error is not recoverable: exiting now 2024-09-11T07:25:43.951 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-11T07:25:43.952 DEBUG:teuthology.orchestra.run.smithi071:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'but it is still running' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(PG_' | egrep -v '\(OSD_' | egrep -v '\(OBJECT_' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | head -n 1 2024-09-11T07:25:43.980 INFO:teuthology.orchestra.run.smithi071.stderr:grep: /var/log/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/ceph.log: No such file or directory 2024-09-11T07:25:43.982 INFO:tasks.cephadm:Compressing logs... 2024-09-11T07:25:43.982 DEBUG:teuthology.orchestra.run.smithi071:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-11T07:25:44.025 DEBUG:teuthology.orchestra.run.smithi196:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-11T07:25:44.048 INFO:teuthology.orchestra.run.smithi071.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/ceph-osd.0.log 2024-09-11T07:25:44.049 INFO:teuthology.orchestra.run.smithi071.stderr:: No such file or directory 2024-09-11T07:25:44.049 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/ceph-osd.1.log 2024-09-11T07:25:44.049 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-11T07:25:44.052 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/ceph-osd.0.log: /var/log/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/ceph-mon.c.log 2024-09-11T07:25:44.052 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/ceph-mon.a.log 2024-09-11T07:25:44.052 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/ceph-osd.2.log 2024-09-11T07:25:44.053 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/ceph-mon.c.log: /var/log/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/ceph-mgr.y.log 2024-09-11T07:25:44.053 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/ceph-client.rgw.foo.a.log 2024-09-11T07:25:44.054 INFO:teuthology.orchestra.run.smithi196.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/tcmu-runner.log 2024-09-11T07:25:44.054 INFO:teuthology.orchestra.run.smithi196.stderr:: No such file or directory 2024-09-11T07:25:44.054 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/ceph-osd.7.log 2024-09-11T07:25:44.054 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-11T07:25:44.055 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/tcmu-runner.log: /var/log/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/ceph-osd.7.log: 62.9%gzip -- replaced with /var/log/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/tcmu-runner.log.gz 2024-09-11T07:25:44.055 INFO:teuthology.orchestra.run.smithi196.stderr: -5 --verbose -- /var/log/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/ceph-osd.4.log 2024-09-11T07:25:44.055 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/ceph-osd.6.log 2024-09-11T07:25:44.055 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/ceph-osd.5.log 2024-09-11T07:25:44.055 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/ceph-mgr.x.log 2024-09-11T07:25:44.055 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/ceph-mon.b.log 2024-09-11T07:25:44.055 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-11T07:25:44.055 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/ceph-mon.b.log: /var/log/ceph/cephadm.log: 92.5% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-11T07:25:44.055 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/ceph-osd.3.log 2024-09-11T07:25:44.056 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/ceph-client.rgw.foo.a.log: 58.6% -- replaced with /var/log/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/ceph-client.rgw.foo.a.log.gz 2024-09-11T07:25:44.056 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-11T07:25:44.059 INFO:teuthology.orchestra.run.smithi196.stderr: 91.0% -- replaced with /var/log/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/ceph-mgr.x.log.gz 2024-09-11T07:25:44.061 INFO:teuthology.orchestra.run.smithi196.stderr: 91.1% -- replaced with /var/log/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-11T07:25:44.064 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/ceph-osd.3.log: 91.2% -- replaced with /var/log/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-11T07:25:44.068 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/cephadm.log: 91.8% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-11T07:25:44.240 INFO:teuthology.orchestra.run.smithi071.stderr: 88.4% -- replaced with /var/log/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/ceph-mgr.y.log.gz 2024-09-11T07:25:44.292 INFO:teuthology.orchestra.run.smithi196.stderr: 91.9% -- replaced with /var/log/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/ceph-mon.b.log.gz 2024-09-11T07:25:44.367 INFO:teuthology.orchestra.run.smithi071.stderr: 91.4% -- replaced with /var/log/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/ceph-mon.c.log.gz 2024-09-11T07:25:45.017 INFO:teuthology.orchestra.run.smithi071.stderr: 91.0% -- replaced with /var/log/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/ceph-mon.a.log.gz 2024-09-11T07:25:45.951 INFO:teuthology.orchestra.run.smithi071.stderr: 94.1% -- replaced with /var/log/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/ceph-osd.2.log.gz 2024-09-11T07:25:45.968 INFO:teuthology.orchestra.run.smithi196.stderr: 94.1% -- replaced with /var/log/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/ceph-osd.6.log.gz 2024-09-11T07:25:45.971 INFO:teuthology.orchestra.run.smithi196.stderr: 94.1% -- replaced with /var/log/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/ceph-osd.5.log.gz 2024-09-11T07:25:46.021 INFO:teuthology.orchestra.run.smithi196.stderr: 94.2% -- replaced with /var/log/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/ceph-osd.4.log.gz 2024-09-11T07:25:46.047 INFO:teuthology.orchestra.run.smithi196.stderr: 94.1% -- replaced with /var/log/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/ceph-osd.7.log.gz 2024-09-11T07:25:46.049 INFO:teuthology.orchestra.run.smithi196.stderr: 2024-09-11T07:25:46.049 INFO:teuthology.orchestra.run.smithi196.stderr:real 0m2.009s 2024-09-11T07:25:46.049 INFO:teuthology.orchestra.run.smithi196.stderr:user 0m7.785s 2024-09-11T07:25:46.049 INFO:teuthology.orchestra.run.smithi196.stderr:sys 0m0.291s 2024-09-11T07:25:46.334 INFO:teuthology.orchestra.run.smithi071.stderr: 94.1% -- replaced with /var/log/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/ceph-osd.0.log.gz 2024-09-11T07:25:46.351 INFO:teuthology.orchestra.run.smithi071.stderr: 94.2% -- replaced with /var/log/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/ceph-osd.3.log.gz 2024-09-11T07:25:46.426 INFO:teuthology.orchestra.run.smithi071.stderr: 94.1% -- replaced with /var/log/ceph/a81cffe6-700c-11ef-bcea-c7b262605968/ceph-osd.1.log.gz 2024-09-11T07:25:46.428 INFO:teuthology.orchestra.run.smithi071.stderr: 2024-09-11T07:25:46.428 INFO:teuthology.orchestra.run.smithi071.stderr:real 0m2.390s 2024-09-11T07:25:46.428 INFO:teuthology.orchestra.run.smithi071.stderr:user 0m9.981s 2024-09-11T07:25:46.429 INFO:teuthology.orchestra.run.smithi071.stderr:sys 0m0.374s 2024-09-11T07:25:46.429 INFO:tasks.cephadm:Archiving logs... 2024-09-11T07:25:46.429 DEBUG:teuthology.misc:Transferring archived files from smithi071:/var/log/ceph to /home/teuthworker/archive/gabrioux-2024-09-11_06:53:43-orch:cephadm-wip-guits-quincy-2024-09-10-2008-quincy-distro-default-smithi/7901202/remote/smithi071/log 2024-09-11T07:25:46.430 DEBUG:teuthology.orchestra.run.smithi071:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-11T07:25:47.740 DEBUG:teuthology.misc:Transferring archived files from smithi196:/var/log/ceph to /home/teuthworker/archive/gabrioux-2024-09-11_06:53:43-orch:cephadm-wip-guits-quincy-2024-09-10-2008-quincy-distro-default-smithi/7901202/remote/smithi196/log 2024-09-11T07:25:47.741 DEBUG:teuthology.orchestra.run.smithi196:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-11T07:25:48.811 INFO:tasks.cephadm:Removing cluster... 2024-09-11T07:25:48.811 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid a81cffe6-700c-11ef-bcea-c7b262605968 --force 2024-09-11T07:25:49.289 DEBUG:teuthology.orchestra.run.smithi196:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid a81cffe6-700c-11ef-bcea-c7b262605968 --force 2024-09-11T07:25:49.767 INFO:tasks.cephadm:Removing cephadm ... 2024-09-11T07:25:49.767 DEBUG:teuthology.orchestra.run.smithi071:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-11T07:25:49.784 DEBUG:teuthology.orchestra.run.smithi196:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-11T07:25:49.799 INFO:tasks.cephadm:Teardown complete 2024-09-11T07:25:49.799 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-11T07:25:49.808 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-11T07:25:49.808 DEBUG:teuthology.orchestra.run.smithi071:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-11T07:25:49.826 DEBUG:teuthology.orchestra.run.smithi196:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-11T07:25:49.917 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-11T07:25:49.926 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-11T07:25:49.926 DEBUG:teuthology.orchestra.run.smithi071:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-11T07:25:49.929 DEBUG:teuthology.orchestra.run.smithi196:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-11T07:25:49.941 INFO:teuthology.orchestra.run.smithi071.stderr:bash: line 1: ntpq: command not found 2024-09-11T07:25:49.944 INFO:teuthology.orchestra.run.smithi196.stderr:bash: line 1: ntpq: command not found 2024-09-11T07:25:49.945 INFO:teuthology.orchestra.run.smithi071.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-11T07:25:49.945 INFO:teuthology.orchestra.run.smithi071.stdout:=============================================================================== 2024-09-11T07:25:49.946 INFO:teuthology.orchestra.run.smithi071.stdout:^+ hv01.front.sepia.ceph.com 3 6 377 33 +874us[ +874us] +/- 50ms 2024-09-11T07:25:49.946 INFO:teuthology.orchestra.run.smithi071.stdout:^* hv02.front.sepia.ceph.com 3 6 377 34 -469us[ -386us] +/- 52ms 2024-09-11T07:25:49.946 INFO:teuthology.orchestra.run.smithi071.stdout:^+ hv03.front.sepia.ceph.com 3 7 377 28 -601us[ -601us] +/- 67ms 2024-09-11T07:25:49.946 INFO:teuthology.orchestra.run.smithi071.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T07:25:49.950 INFO:teuthology.orchestra.run.smithi196.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-11T07:25:49.950 INFO:teuthology.orchestra.run.smithi196.stdout:=============================================================================== 2024-09-11T07:25:49.950 INFO:teuthology.orchestra.run.smithi196.stdout:^+ hv01.front.sepia.ceph.com 3 6 377 31 +915us[ +915us] +/- 50ms 2024-09-11T07:25:49.950 INFO:teuthology.orchestra.run.smithi196.stdout:^* hv02.front.sepia.ceph.com 3 6 377 36 -467us[ -386us] +/- 52ms 2024-09-11T07:25:49.950 INFO:teuthology.orchestra.run.smithi196.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 35 -630us[ -630us] +/- 67ms 2024-09-11T07:25:49.950 INFO:teuthology.orchestra.run.smithi196.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T07:25:49.951 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-11T07:25:49.982 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-11T07:25:49.982 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-11T07:25:50.064 DEBUG:teuthology.orchestra.run.smithi071:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-11T07:25:50.134 DEBUG:teuthology.orchestra.run.smithi196:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-11T07:25:50.202 DEBUG:teuthology.orchestra.run.smithi071:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-11T07:25:50.230 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:25:50.230 DEBUG:teuthology.orchestra.run.smithi196:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-11T07:25:50.258 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:25:50.259 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-11T07:25:50.268 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-11T07:25:50.303 INFO:teuthology.task.internal:Duration was 1361.303411 seconds 2024-09-11T07:25:50.304 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-11T07:25:50.336 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-11T07:25:50.336 DEBUG:teuthology.orchestra.run.smithi071:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-11T07:25:50.338 DEBUG:teuthology.orchestra.run.smithi196:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-11T07:25:50.375 INFO:teuthology.orchestra.run.smithi071.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-11T07:25:50.376 INFO:teuthology.orchestra.run.smithi196.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-11T07:25:50.876 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-11T07:25:50.876 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi071.front.sepia.ceph.com 2024-09-11T07:25:50.876 DEBUG:teuthology.orchestra.run.smithi071:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-11T07:25:50.936 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi196.front.sepia.ceph.com 2024-09-11T07:25:50.936 DEBUG:teuthology.orchestra.run.smithi196:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-11T07:25:50.965 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-11T07:25:50.965 DEBUG:teuthology.orchestra.run.smithi071:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-11T07:25:50.978 DEBUG:teuthology.orchestra.run.smithi196:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-11T07:25:51.032 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-11T07:25:51.032 DEBUG:teuthology.orchestra.run.smithi071:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-11T07:25:51.045 DEBUG:teuthology.orchestra.run.smithi196:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-11T07:25:51.307 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-11T07:25:51.318 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-11T07:25:51.318 DEBUG:teuthology.orchestra.run.smithi071:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-11T07:25:51.342 DEBUG:teuthology.orchestra.run.smithi196:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-11T07:25:51.368 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-11T07:25:51.377 DEBUG:teuthology.orchestra.run.smithi071:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-11T07:25:51.385 DEBUG:teuthology.orchestra.run.smithi196:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-11T07:25:51.406 INFO:teuthology.orchestra.run.smithi071.stdout:kernel.core_pattern = core 2024-09-11T07:25:51.431 INFO:teuthology.orchestra.run.smithi196.stdout:kernel.core_pattern = core 2024-09-11T07:25:51.477 DEBUG:teuthology.orchestra.run.smithi071:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-11T07:25:51.492 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:25:51.492 DEBUG:teuthology.orchestra.run.smithi196:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-11T07:25:51.532 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T07:25:51.532 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-11T07:25:51.542 INFO:teuthology.task.internal:Transferring archived files... 2024-09-11T07:25:51.543 DEBUG:teuthology.misc:Transferring archived files from smithi071:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/gabrioux-2024-09-11_06:53:43-orch:cephadm-wip-guits-quincy-2024-09-10-2008-quincy-distro-default-smithi/7901202/remote/smithi071 2024-09-11T07:25:51.543 DEBUG:teuthology.orchestra.run.smithi071:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-11T07:25:51.583 DEBUG:teuthology.misc:Transferring archived files from smithi196:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/gabrioux-2024-09-11_06:53:43-orch:cephadm-wip-guits-quincy-2024-09-10-2008-quincy-distro-default-smithi/7901202/remote/smithi196 2024-09-11T07:25:51.583 DEBUG:teuthology.orchestra.run.smithi196:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-11T07:25:51.619 INFO:teuthology.task.internal:Removing archive directory... 2024-09-11T07:25:51.619 DEBUG:teuthology.orchestra.run.smithi071:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-11T07:25:51.622 DEBUG:teuthology.orchestra.run.smithi196:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-11T07:25:51.669 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-11T07:25:51.680 INFO:teuthology.task.internal:Not uploading archives. 2024-09-11T07:25:51.680 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-11T07:25:51.712 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-11T07:25:51.712 DEBUG:teuthology.orchestra.run.smithi071:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-11T07:25:51.714 DEBUG:teuthology.orchestra.run.smithi196:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-11T07:25:51.727 INFO:teuthology.orchestra.run.smithi071.stdout: 265963 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 11 07:25 /home/ubuntu/cephtest 2024-09-11T07:25:51.729 INFO:teuthology.orchestra.run.smithi196.stdout: 266054 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 11 07:25 /home/ubuntu/cephtest 2024-09-11T07:25:51.731 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-11T07:25:51.745 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-11T07:25:51.814 INFO:teuthology.run:Summary data: description: orch:cephadm/with-work/{0-distro/centos_9.stream fixed-2 mode/root mon_election/connectivity msgr/async-v1only start tasks/rados_python} duration: 1361.3034110069275 flavor: default owner: scheduled_gabrioux@teuthology success: true 2024-09-11T07:25:51.814 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-11T07:25:51.893 INFO:teuthology.run:pass