2024-02-13T13:59:15.390 INFO:root:teuthology version: 0.0.1.dev256+gd9fdb22 2024-02-13T13:59:15.390 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --archive /home/teuthworker/archive/phlogistonjohn-2024-02-13_11:41:43-orch:cephadm-wip-phlogistonjohn-testing-2024-02-12-1654-distro-default-smithi/7558012 --owner scheduled_phlogistonjohn@teuthology --name phlogistonjohn-2024-02-13_11:41:43-orch:cephadm-wip-phlogistonjohn-testing-2024-02-12-1654-distro-default-smithi --description orch:cephadm/with-work/{0-distro/centos_9.stream_runc fixed-2 mode/root mon_election/connectivity msgr/async start tasks/rados_python} -- /home/teuthworker/archive/phlogistonjohn-2024-02-13_11:41:43-orch:cephadm-wip-phlogistonjohn-testing-2024-02-12-1654-distro-default-smithi/7558012/orig.config.yaml 2024-02-13T13:59:15.419 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-13T13:59:15.496 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/phlogistonjohn-2024-02-13_11:41:43-orch:cephadm-wip-phlogistonjohn-testing-2024-02-12-1654-distro-default-smithi/7558012 branch: wip-phlogistonjohn-testing-2024-02-12-1654 description: orch:cephadm/with-work/{0-distro/centos_9.stream_runc fixed-2 mode/root mon_election/connectivity msgr/async start tasks/rados_python} email: jmulligan@redhat.com first_in_suite: false job_id: '7558012' kernel: kdb: true sha1: distro last_in_suite: false machine_type: smithi name: phlogistonjohn-2024-02-13_11:41:43-orch:cephadm-wip-phlogistonjohn-testing-2024-02-12-1654-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 10 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-phlogistonjohn-testing-2024-02-12-1654 ceph: conf: global: mon election default strategy: 3 ms type: async mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: debug ms: 1 debug osd: 20 osd shutdown pgref assert: true flavor: default log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - but it is still running - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(PG_ - \(OSD_ - \(OBJECT_ - \(POOL_APP_NOT_ENABLED\) sha1: e1b79d5a4193a484176863ab96c0ca28c00e1662 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} cephadm: cephadm_mode: root install: ceph: extra_system_packages: deb: - python3-pytest rpm: - python3-pytest flavor: default sha1: e1b79d5a4193a484176863ab96c0ca28c00e1662 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 workunit: branch: wip-phlogistonjohn-testing-2024-02-12-1654 sha1: e1b79d5a4193a484176863ab96c0ca28c00e1662 owner: scheduled_phlogistonjohn@teuthology priority: 99 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - ceph.rgw.foo.a - node-exporter.a - alertmanager.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - grafana.a - node-exporter.b - ceph.iscsi.iscsi.a seed: 1588 sha1: e1b79d5a4193a484176863ab96c0ca28c00e1662 sleep_before_teardown: 0 subset: 1/15 suite: orch:cephadm suite_branch: wip-phlogistonjohn-testing-2024-02-12-1654 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_e1b79d5a4193a484176863ab96c0ca28c00e1662/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: e1b79d5a4193a484176863ab96c0ca28c00e1662 targets: smithi087.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPYH3T79/fS8yIgsiTb6IbwUfljLcOEV5eBwlRcb8CdsxB6pcYb0BFy2gnaz4NGKpP6BOg2HxnJBSzhFcCBHbi4= smithi148.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFfUCk8jh0XR5KaJqXZ5E9hxJyiF/VMF7OqStLQjgWjeVtIYNFd4mR7buTxN16zH/61k1o//+oFYzJFF/LOL394= tasks: - pexec: all: - sudo dnf install runc -y - sudo sed -i 's/^#runtime = "crun"/runtime = "runc"/g' /usr/share/containers/containers.conf - sudo sed -i 's/runtime = "crun"/#runtime = "crun"/g' /usr/share/containers/containers.conf - install: null - cephadm: conf: mgr: debug mgr: 20 debug ms: 1 - workunit: clients: client.0: - rados/test_python.sh timeout: 1h teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: d9fdb2209e15b39d9f061fd85399f352ce0f0894 timestamp: 2024-02-13_11:41:43 tube: smithi user: phlogistonjohn verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-02-13T13:59:15.497 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_e1b79d5a4193a484176863ab96c0ca28c00e1662/qa; will attempt to use it 2024-02-13T13:59:15.497 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_e1b79d5a4193a484176863ab96c0ca28c00e1662/qa/tasks 2024-02-13T13:59:15.497 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-02-13T13:59:15.499 INFO:teuthology.task.internal:Checking packages... 2024-02-13T13:59:15.523 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'e1b79d5a4193a484176863ab96c0ca28c00e1662' 2024-02-13T13:59:15.523 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-02-13T13:59:15.523 INFO:teuthology.packaging:ref: None 2024-02-13T13:59:15.523 INFO:teuthology.packaging:tag: None 2024-02-13T13:59:15.524 INFO:teuthology.packaging:branch: wip-phlogistonjohn-testing-2024-02-12-1654 2024-02-13T13:59:15.524 INFO:teuthology.packaging:sha1: e1b79d5a4193a484176863ab96c0ca28c00e1662 2024-02-13T13:59:15.524 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-phlogistonjohn-testing-2024-02-12-1654 2024-02-13T13:59:15.661 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-1423.ge1b79d5a 2024-02-13T13:59:15.663 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-02-13T13:59:15.670 INFO:teuthology.task.internal:no buildpackages task found 2024-02-13T13:59:15.670 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-02-13T13:59:15.717 INFO:teuthology.task.internal:Saving configuration 2024-02-13T13:59:15.731 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-02-13T13:59:15.745 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-13T13:59:15.767 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi087.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/phlogistonjohn-2024-02-13_11:41:43-orch:cephadm-wip-phlogistonjohn-testing-2024-02-12-1654-distro-default-smithi/7558012', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-13 13:54:06.895163', 'locked_by': 'scheduled_phlogistonjohn@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPYH3T79/fS8yIgsiTb6IbwUfljLcOEV5eBwlRcb8CdsxB6pcYb0BFy2gnaz4NGKpP6BOg2HxnJBSzhFcCBHbi4='} 2024-02-13T13:59:15.788 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi148.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/phlogistonjohn-2024-02-13_11:41:43-orch:cephadm-wip-phlogistonjohn-testing-2024-02-12-1654-distro-default-smithi/7558012', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-13 13:54:06.893557', 'locked_by': 'scheduled_phlogistonjohn@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFfUCk8jh0XR5KaJqXZ5E9hxJyiF/VMF7OqStLQjgWjeVtIYNFd4mR7buTxN16zH/61k1o//+oFYzJFF/LOL394='} 2024-02-13T13:59:15.788 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-02-13T13:59:15.800 INFO:teuthology.task.internal:roles: ubuntu@smithi087.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'ceph.rgw.foo.a', 'node-exporter.a', 'alertmanager.a'] 2024-02-13T13:59:15.800 INFO:teuthology.task.internal:roles: ubuntu@smithi148.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'grafana.a', 'node-exporter.b', 'ceph.iscsi.iscsi.a'] 2024-02-13T13:59:15.800 INFO:teuthology.run_tasks:Running task console_log... 2024-02-13T13:59:15.884 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f4de94450d0>, signals=[15]) 2024-02-13T13:59:15.885 INFO:teuthology.run_tasks:Running task internal.connect... 2024-02-13T13:59:15.894 INFO:teuthology.task.internal:Opening connections... 2024-02-13T13:59:15.894 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi087.front.sepia.ceph.com 2024-02-13T13:59:15.896 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi087.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-13T13:59:15.988 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi148.front.sepia.ceph.com 2024-02-13T13:59:15.989 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-13T13:59:16.077 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-02-13T13:59:16.087 DEBUG:teuthology.orchestra.run.smithi087:> uname -m 2024-02-13T13:59:16.104 INFO:teuthology.orchestra.run.smithi087.stdout:x86_64 2024-02-13T13:59:16.104 DEBUG:teuthology.orchestra.run.smithi087:> cat /etc/os-release 2024-02-13T13:59:16.160 INFO:teuthology.orchestra.run.smithi087.stdout:NAME="CentOS Stream" 2024-02-13T13:59:16.160 INFO:teuthology.orchestra.run.smithi087.stdout:VERSION="9" 2024-02-13T13:59:16.160 INFO:teuthology.orchestra.run.smithi087.stdout:ID="centos" 2024-02-13T13:59:16.161 INFO:teuthology.orchestra.run.smithi087.stdout:ID_LIKE="rhel fedora" 2024-02-13T13:59:16.161 INFO:teuthology.orchestra.run.smithi087.stdout:VERSION_ID="9" 2024-02-13T13:59:16.161 INFO:teuthology.orchestra.run.smithi087.stdout:PLATFORM_ID="platform:el9" 2024-02-13T13:59:16.161 INFO:teuthology.orchestra.run.smithi087.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-13T13:59:16.161 INFO:teuthology.orchestra.run.smithi087.stdout:ANSI_COLOR="0;31" 2024-02-13T13:59:16.161 INFO:teuthology.orchestra.run.smithi087.stdout:LOGO="fedora-logo-icon" 2024-02-13T13:59:16.161 INFO:teuthology.orchestra.run.smithi087.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-13T13:59:16.161 INFO:teuthology.orchestra.run.smithi087.stdout:HOME_URL="https://centos.org/" 2024-02-13T13:59:16.161 INFO:teuthology.orchestra.run.smithi087.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-13T13:59:16.161 INFO:teuthology.orchestra.run.smithi087.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-13T13:59:16.161 INFO:teuthology.orchestra.run.smithi087.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-13T13:59:16.162 INFO:teuthology.lock.ops:Updating smithi087.front.sepia.ceph.com on lock server 2024-02-13T13:59:16.183 DEBUG:teuthology.orchestra.run.smithi148:> uname -m 2024-02-13T13:59:16.201 INFO:teuthology.orchestra.run.smithi148.stdout:x86_64 2024-02-13T13:59:16.201 DEBUG:teuthology.orchestra.run.smithi148:> cat /etc/os-release 2024-02-13T13:59:16.258 INFO:teuthology.orchestra.run.smithi148.stdout:NAME="CentOS Stream" 2024-02-13T13:59:16.258 INFO:teuthology.orchestra.run.smithi148.stdout:VERSION="9" 2024-02-13T13:59:16.258 INFO:teuthology.orchestra.run.smithi148.stdout:ID="centos" 2024-02-13T13:59:16.258 INFO:teuthology.orchestra.run.smithi148.stdout:ID_LIKE="rhel fedora" 2024-02-13T13:59:16.258 INFO:teuthology.orchestra.run.smithi148.stdout:VERSION_ID="9" 2024-02-13T13:59:16.258 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM_ID="platform:el9" 2024-02-13T13:59:16.258 INFO:teuthology.orchestra.run.smithi148.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-13T13:59:16.258 INFO:teuthology.orchestra.run.smithi148.stdout:ANSI_COLOR="0;31" 2024-02-13T13:59:16.258 INFO:teuthology.orchestra.run.smithi148.stdout:LOGO="fedora-logo-icon" 2024-02-13T13:59:16.258 INFO:teuthology.orchestra.run.smithi148.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-13T13:59:16.258 INFO:teuthology.orchestra.run.smithi148.stdout:HOME_URL="https://centos.org/" 2024-02-13T13:59:16.258 INFO:teuthology.orchestra.run.smithi148.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-13T13:59:16.259 INFO:teuthology.orchestra.run.smithi148.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-13T13:59:16.259 INFO:teuthology.orchestra.run.smithi148.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-13T13:59:16.259 INFO:teuthology.lock.ops:Updating smithi148.front.sepia.ceph.com on lock server 2024-02-13T13:59:16.275 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-02-13T13:59:16.287 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-02-13T13:59:16.309 INFO:teuthology.task.internal:Checking for old test directory... 2024-02-13T13:59:16.310 DEBUG:teuthology.orchestra.run.smithi087:> test '!' -e /home/ubuntu/cephtest 2024-02-13T13:59:16.312 DEBUG:teuthology.orchestra.run.smithi148:> test '!' -e /home/ubuntu/cephtest 2024-02-13T13:59:16.328 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-02-13T13:59:16.339 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-02-13T13:59:16.339 DEBUG:teuthology.orchestra.run.smithi087:> test -z $(ls -A /var/lib/ceph) 2024-02-13T13:59:16.370 DEBUG:teuthology.orchestra.run.smithi148:> test -z $(ls -A /var/lib/ceph) 2024-02-13T13:59:16.435 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-02-13T13:59:16.487 INFO:teuthology.run_tasks:Running task kernel... 2024-02-13T13:59:16.507 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2024-02-13T13:59:16.507 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2024-02-13T13:59:16.507 DEBUG:teuthology.orchestra.run.smithi087:> test -f /run/.containerenv -o -f /.dockerenv 2024-02-13T13:59:16.508 DEBUG:teuthology.orchestra.run.smithi148:> test -f /run/.containerenv -o -f /.dockerenv 2024-02-13T13:59:16.523 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-13T13:59:16.523 DEBUG:teuthology.orchestra.run.smithi148:> uname -r 2024-02-13T13:59:16.524 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-13T13:59:16.525 DEBUG:teuthology.orchestra.run.smithi087:> uname -r 2024-02-13T13:59:16.580 INFO:teuthology.orchestra.run.smithi148.stdout:5.14.0-361.el9.x86_64 2024-02-13T13:59:16.580 INFO:teuthology.task.kernel:Running kernel on smithi148: 5.14.0-361.el9.x86_64 2024-02-13T13:59:16.581 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum install -y kernel 2024-02-13T13:59:16.582 INFO:teuthology.orchestra.run.smithi087.stdout:5.14.0-361.el9.x86_64 2024-02-13T13:59:16.582 INFO:teuthology.task.kernel:Running kernel on smithi087: 5.14.0-361.el9.x86_64 2024-02-13T13:59:16.582 DEBUG:teuthology.orchestra.run.smithi087:> sudo yum install -y kernel 2024-02-13T13:59:18.718 INFO:teuthology.orchestra.run.smithi087.stdout:CentOS-9 - AppStream 13 MB/s | 6.7 MB 00:00 2024-02-13T13:59:19.296 INFO:teuthology.orchestra.run.smithi148.stdout:CentOS-9 - AppStream 6.5 MB/s | 6.7 MB 00:01 2024-02-13T13:59:20.176 INFO:teuthology.orchestra.run.smithi087.stdout:CentOS-9 - BaseOS 5.6 MB/s | 2.0 MB 00:00 2024-02-13T13:59:20.896 INFO:teuthology.orchestra.run.smithi148.stdout:CentOS-9 - BaseOS 4.1 MB/s | 2.0 MB 00:00 2024-02-13T13:59:22.659 INFO:teuthology.orchestra.run.smithi087.stdout:CentOS Stream 9 - BaseOS 3.7 MB/s | 8.0 MB 00:02 2024-02-13T13:59:23.128 INFO:teuthology.orchestra.run.smithi148.stdout:CentOS Stream 9 - BaseOS 4.2 MB/s | 8.0 MB 00:01 2024-02-13T13:59:26.296 INFO:teuthology.orchestra.run.smithi148.stdout:CentOS Stream 9 - AppStream 9.1 MB/s | 19 MB 00:02 2024-02-13T13:59:29.818 INFO:teuthology.orchestra.run.smithi087.stdout:CentOS Stream 9 - AppStream 3.1 MB/s | 19 MB 00:06 2024-02-13T13:59:30.280 INFO:teuthology.orchestra.run.smithi148.stdout:CentOS Stream 9 - CRB 12 MB/s | 5.9 MB 00:00 2024-02-13T13:59:31.864 INFO:teuthology.orchestra.run.smithi148.stdout:CentOS Stream 9 - Extras packages 31 kB/s | 15 kB 00:00 2024-02-13T13:59:33.632 INFO:teuthology.orchestra.run.smithi148.stdout:Extra Packages for Enterprise Linux 12 MB/s | 20 MB 00:01 2024-02-13T13:59:35.643 INFO:teuthology.orchestra.run.smithi087.stdout:CentOS Stream 9 - CRB 2.5 MB/s | 5.9 MB 00:02 2024-02-13T13:59:37.044 INFO:teuthology.orchestra.run.smithi087.stdout:CentOS Stream 9 - Extras packages 50 kB/s | 15 kB 00:00 2024-02-13T13:59:38.565 INFO:teuthology.orchestra.run.smithi148.stdout:lab-extras 12 kB/s | 1.7 kB 00:00 2024-02-13T13:59:39.245 INFO:teuthology.orchestra.run.smithi087.stdout:Extra Packages for Enterprise Linux 9.8 MB/s | 20 MB 00:02 2024-02-13T13:59:40.888 INFO:teuthology.orchestra.run.smithi148.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-02-13T13:59:40.889 INFO:teuthology.orchestra.run.smithi148.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-02-13T13:59:41.028 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-02-13T13:59:41.037 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-02-13T13:59:41.038 INFO:teuthology.orchestra.run.smithi148.stdout: Package Arch Version Repository Size 2024-02-13T13:59:41.038 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-02-13T13:59:41.038 INFO:teuthology.orchestra.run.smithi148.stdout:Installing: 2024-02-13T13:59:41.038 INFO:teuthology.orchestra.run.smithi148.stdout: kernel x86_64 5.14.0-419.el9 CentOS-BaseOS 6.0 M 2024-02-13T13:59:41.038 INFO:teuthology.orchestra.run.smithi148.stdout:Installing dependencies: 2024-02-13T13:59:41.038 INFO:teuthology.orchestra.run.smithi148.stdout: kernel-core x86_64 5.14.0-419.el9 CentOS-BaseOS 21 M 2024-02-13T13:59:41.038 INFO:teuthology.orchestra.run.smithi148.stdout: kernel-modules x86_64 5.14.0-419.el9 CentOS-BaseOS 39 M 2024-02-13T13:59:41.038 INFO:teuthology.orchestra.run.smithi148.stdout: kernel-modules-core x86_64 5.14.0-419.el9 CentOS-BaseOS 33 M 2024-02-13T13:59:41.038 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-02-13T13:59:41.038 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction Summary 2024-02-13T13:59:41.038 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-02-13T13:59:41.038 INFO:teuthology.orchestra.run.smithi148.stdout:Install 4 Packages 2024-02-13T13:59:41.038 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-02-13T13:59:41.039 INFO:teuthology.orchestra.run.smithi148.stdout:Total download size: 99 M 2024-02-13T13:59:41.039 INFO:teuthology.orchestra.run.smithi148.stdout:Installed size: 122 M 2024-02-13T13:59:41.039 INFO:teuthology.orchestra.run.smithi148.stdout:Downloading Packages: 2024-02-13T13:59:41.850 INFO:teuthology.orchestra.run.smithi148.stdout:(1/4): kernel-5.14.0-419.el9.x86_64.rpm 7.4 MB/s | 6.0 MB 00:00 2024-02-13T13:59:42.620 INFO:teuthology.orchestra.run.smithi148.stdout:(2/4): kernel-core-5.14.0-419.el9.x86_64.rpm 13 MB/s | 21 MB 00:01 2024-02-13T13:59:43.650 INFO:teuthology.orchestra.run.smithi148.stdout:(3/4): kernel-modules-5.14.0-419.el9.x86_64.rpm 15 MB/s | 39 MB 00:02 2024-02-13T13:59:44.125 INFO:teuthology.orchestra.run.smithi148.stdout:(4/4): kernel-modules-core-5.14.0-419.el9.x86_6 15 MB/s | 33 MB 00:02 2024-02-13T13:59:44.128 INFO:teuthology.orchestra.run.smithi148.stdout:-------------------------------------------------------------------------------- 2024-02-13T13:59:44.128 INFO:teuthology.orchestra.run.smithi148.stdout:Total 32 MB/s | 99 MB 00:03 2024-02-13T13:59:44.128 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction check 2024-02-13T13:59:44.178 INFO:teuthology.orchestra.run.smithi087.stdout:lab-extras 12 kB/s | 1.7 kB 00:00 2024-02-13T13:59:44.500 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction check succeeded. 2024-02-13T13:59:44.500 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction test 2024-02-13T13:59:45.515 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction test succeeded. 2024-02-13T13:59:45.516 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction 2024-02-13T13:59:46.493 INFO:teuthology.orchestra.run.smithi087.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-02-13T13:59:46.494 INFO:teuthology.orchestra.run.smithi087.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-02-13T13:59:46.612 INFO:teuthology.orchestra.run.smithi087.stdout:Dependencies resolved. 2024-02-13T13:59:46.616 INFO:teuthology.orchestra.run.smithi087.stdout:================================================================================ 2024-02-13T13:59:46.616 INFO:teuthology.orchestra.run.smithi087.stdout: Package Arch Version Repository Size 2024-02-13T13:59:46.616 INFO:teuthology.orchestra.run.smithi087.stdout:================================================================================ 2024-02-13T13:59:46.616 INFO:teuthology.orchestra.run.smithi087.stdout:Installing: 2024-02-13T13:59:46.616 INFO:teuthology.orchestra.run.smithi087.stdout: kernel x86_64 5.14.0-419.el9 CentOS-BaseOS 6.0 M 2024-02-13T13:59:46.616 INFO:teuthology.orchestra.run.smithi087.stdout:Installing dependencies: 2024-02-13T13:59:46.616 INFO:teuthology.orchestra.run.smithi087.stdout: kernel-core x86_64 5.14.0-419.el9 CentOS-BaseOS 21 M 2024-02-13T13:59:46.616 INFO:teuthology.orchestra.run.smithi087.stdout: kernel-modules x86_64 5.14.0-419.el9 CentOS-BaseOS 39 M 2024-02-13T13:59:46.616 INFO:teuthology.orchestra.run.smithi087.stdout: kernel-modules-core x86_64 5.14.0-419.el9 CentOS-BaseOS 33 M 2024-02-13T13:59:46.617 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-02-13T13:59:46.617 INFO:teuthology.orchestra.run.smithi087.stdout:Transaction Summary 2024-02-13T13:59:46.617 INFO:teuthology.orchestra.run.smithi087.stdout:================================================================================ 2024-02-13T13:59:46.617 INFO:teuthology.orchestra.run.smithi087.stdout:Install 4 Packages 2024-02-13T13:59:46.617 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-02-13T13:59:46.617 INFO:teuthology.orchestra.run.smithi087.stdout:Total download size: 99 M 2024-02-13T13:59:46.617 INFO:teuthology.orchestra.run.smithi087.stdout:Installed size: 122 M 2024-02-13T13:59:46.618 INFO:teuthology.orchestra.run.smithi087.stdout:Downloading Packages: 2024-02-13T13:59:47.047 INFO:teuthology.orchestra.run.smithi087.stdout:(1/4): kernel-5.14.0-419.el9.x86_64.rpm 14 MB/s | 6.0 MB 00:00 2024-02-13T13:59:47.705 INFO:teuthology.orchestra.run.smithi087.stdout:(2/4): kernel-core-5.14.0-419.el9.x86_64.rpm 19 MB/s | 21 MB 00:01 2024-02-13T13:59:47.936 INFO:teuthology.orchestra.run.smithi148.stdout: Preparing : 1/1 2024-02-13T13:59:48.587 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : kernel-modules-core-5.14.0-419.el9.x86_64 1/4 2024-02-13T13:59:48.725 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : kernel-core-5.14.0-419.el9.x86_64 2/4 2024-02-13T13:59:48.780 INFO:teuthology.orchestra.run.smithi087.stdout:(3/4): kernel-modules-core-5.14.0-419.el9.x86_6 19 MB/s | 33 MB 00:01 2024-02-13T13:59:49.130 INFO:teuthology.orchestra.run.smithi087.stdout:(4/4): kernel-modules-5.14.0-419.el9.x86_64.rpm 16 MB/s | 39 MB 00:02 2024-02-13T13:59:49.134 INFO:teuthology.orchestra.run.smithi087.stdout:-------------------------------------------------------------------------------- 2024-02-13T13:59:49.134 INFO:teuthology.orchestra.run.smithi087.stdout:Total 39 MB/s | 99 MB 00:02 2024-02-13T13:59:49.134 INFO:teuthology.orchestra.run.smithi087.stdout:Running transaction check 2024-02-13T13:59:49.513 INFO:teuthology.orchestra.run.smithi087.stdout:Transaction check succeeded. 2024-02-13T13:59:49.513 INFO:teuthology.orchestra.run.smithi087.stdout:Running transaction test 2024-02-13T13:59:50.544 INFO:teuthology.orchestra.run.smithi087.stdout:Transaction test succeeded. 2024-02-13T13:59:50.545 INFO:teuthology.orchestra.run.smithi087.stdout:Running transaction 2024-02-13T13:59:51.460 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: kernel-core-5.14.0-419.el9.x86_64 2/4 2024-02-13T13:59:51.634 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : kernel-modules-5.14.0-419.el9.x86_64 3/4 2024-02-13T13:59:52.970 INFO:teuthology.orchestra.run.smithi087.stdout: Preparing : 1/1 2024-02-13T13:59:53.611 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : kernel-modules-core-5.14.0-419.el9.x86_64 1/4 2024-02-13T13:59:53.747 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : kernel-core-5.14.0-419.el9.x86_64 2/4 2024-02-13T13:59:56.364 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: kernel-modules-5.14.0-419.el9.x86_64 3/4 2024-02-13T13:59:56.479 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : kernel-5.14.0-419.el9.x86_64 4/4 2024-02-13T13:59:56.531 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: kernel-core-5.14.0-419.el9.x86_64 2/4 2024-02-13T13:59:56.718 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : kernel-modules-5.14.0-419.el9.x86_64 3/4 2024-02-13T14:00:01.210 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: kernel-modules-core-5.14.0-419.el9.x86_64 4/4 2024-02-13T14:00:01.427 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: kernel-modules-5.14.0-419.el9.x86_64 3/4 2024-02-13T14:00:01.538 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : kernel-5.14.0-419.el9.x86_64 4/4 2024-02-13T14:00:06.278 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: kernel-modules-core-5.14.0-419.el9.x86_64 4/4 2024-02-13T14:00:30.478 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: kernel-core-5.14.0-419.el9.x86_64 4/4 2024-02-13T14:00:30.479 INFO:teuthology.orchestra.run.smithi148.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-02-13T14:00:30.479 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-02-13T14:00:30.548 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: kernel-modules-5.14.0-419.el9.x86_64 4/4 2024-02-13T14:00:30.991 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: kernel-5.14.0-419.el9.x86_64 4/4 2024-02-13T14:00:30.991 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : kernel-5.14.0-419.el9.x86_64 1/4 2024-02-13T14:00:30.991 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : kernel-core-5.14.0-419.el9.x86_64 2/4 2024-02-13T14:00:30.991 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : kernel-modules-5.14.0-419.el9.x86_64 3/4 2024-02-13T14:00:31.412 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : kernel-modules-core-5.14.0-419.el9.x86_64 4/4 2024-02-13T14:00:31.412 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-02-13T14:00:31.412 INFO:teuthology.orchestra.run.smithi148.stdout:Installed: 2024-02-13T14:00:31.412 INFO:teuthology.orchestra.run.smithi148.stdout: kernel-5.14.0-419.el9.x86_64 2024-02-13T14:00:31.412 INFO:teuthology.orchestra.run.smithi148.stdout: kernel-core-5.14.0-419.el9.x86_64 2024-02-13T14:00:31.412 INFO:teuthology.orchestra.run.smithi148.stdout: kernel-modules-5.14.0-419.el9.x86_64 2024-02-13T14:00:31.412 INFO:teuthology.orchestra.run.smithi148.stdout: kernel-modules-core-5.14.0-419.el9.x86_64 2024-02-13T14:00:31.413 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-02-13T14:00:31.413 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-02-13T14:00:31.902 DEBUG:teuthology.orchestra.run.smithi148:> rpm -q kernel | sort -rV | head -n 1 2024-02-13T14:00:32.034 INFO:teuthology.orchestra.run.smithi148.stdout:kernel-5.14.0-419.el9.x86_64 2024-02-13T14:00:32.034 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-419.el9.x86_64 2024-02-13T14:00:32.034 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-419.el9.x86_64 2024-02-13T14:00:32.035 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-02-13T14:00:32.035 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-02-13T14:00:32.035 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-02-13T14:00:32.035 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi148.front.sepia.ceph.com, path=None, version=distro) 2024-02-13T14:00:32.035 DEBUG:teuthology.orchestra.run.smithi148:> rpm -q kernel | sort -rV | head -n 1 2024-02-13T14:00:32.142 INFO:teuthology.orchestra.run.smithi148.stdout:kernel-5.14.0-419.el9.x86_64 2024-02-13T14:00:32.142 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-419.el9.x86_64 2024-02-13T14:00:32.142 DEBUG:teuthology.orchestra.run.smithi148:> sudo rpm -qi grub2-tools 2024-02-13T14:00:32.179 INFO:teuthology.orchestra.run.smithi148.stdout:Name : grub2-tools 2024-02-13T14:00:32.179 INFO:teuthology.orchestra.run.smithi148.stdout:Epoch : 1 2024-02-13T14:00:32.180 INFO:teuthology.orchestra.run.smithi148.stdout:Version : 2.06 2024-02-13T14:00:32.180 INFO:teuthology.orchestra.run.smithi148.stdout:Release : 68.el9 2024-02-13T14:00:32.180 INFO:teuthology.orchestra.run.smithi148.stdout:Architecture: x86_64 2024-02-13T14:00:32.180 INFO:teuthology.orchestra.run.smithi148.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-02-13T14:00:32.180 INFO:teuthology.orchestra.run.smithi148.stdout:Group : Unspecified 2024-02-13T14:00:32.180 INFO:teuthology.orchestra.run.smithi148.stdout:Size : 8264254 2024-02-13T14:00:32.180 INFO:teuthology.orchestra.run.smithi148.stdout:License : GPLv3+ 2024-02-13T14:00:32.180 INFO:teuthology.orchestra.run.smithi148.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-02-13T14:00:32.180 INFO:teuthology.orchestra.run.smithi148.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-02-13T14:00:32.180 INFO:teuthology.orchestra.run.smithi148.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-02-13T14:00:32.180 INFO:teuthology.orchestra.run.smithi148.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-02-13T14:00:32.180 INFO:teuthology.orchestra.run.smithi148.stdout:Packager : builder@centos.org 2024-02-13T14:00:32.180 INFO:teuthology.orchestra.run.smithi148.stdout:Vendor : CentOS 2024-02-13T14:00:32.180 INFO:teuthology.orchestra.run.smithi148.stdout:URL : http://www.gnu.org/software/grub/ 2024-02-13T14:00:32.181 INFO:teuthology.orchestra.run.smithi148.stdout:Summary : Support tools for GRUB. 2024-02-13T14:00:32.181 INFO:teuthology.orchestra.run.smithi148.stdout:Description : 2024-02-13T14:00:32.181 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-02-13T14:00:32.181 INFO:teuthology.orchestra.run.smithi148.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-02-13T14:00:32.181 INFO:teuthology.orchestra.run.smithi148.stdout:customizable bootloader with modular architecture. It supports a rich 2024-02-13T14:00:32.181 INFO:teuthology.orchestra.run.smithi148.stdout:variety of kernel formats, file systems, computer architectures and 2024-02-13T14:00:32.181 INFO:teuthology.orchestra.run.smithi148.stdout:hardware devices. 2024-02-13T14:00:32.181 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-02-13T14:00:32.181 INFO:teuthology.orchestra.run.smithi148.stdout:This subpackage provides tools for support of all platforms. 2024-02-13T14:00:32.182 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-02-13T14:00:32.182 INFO:teuthology.task.kernel:Updating grub on smithi148 to boot 5.14.0-419.el9.x86_64 2024-02-13T14:00:32.182 DEBUG:teuthology.orchestra.run.smithi148:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-02-13T14:00:32.796 INFO:teuthology.orchestra.run.smithi148.stderr:Generating grub configuration file ... 2024-02-13T14:00:34.129 INFO:teuthology.orchestra.run.smithi148.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-02-13T14:00:34.163 INFO:teuthology.orchestra.run.smithi148.stderr:done 2024-02-13T14:00:34.165 DEBUG:teuthology.orchestra.run.smithi148:> mktemp 2024-02-13T14:00:34.181 INFO:teuthology.orchestra.run.smithi148.stdout:/tmp/tmp.bWIoHIpPL3 2024-02-13T14:00:34.181 DEBUG:teuthology.orchestra.run.smithi148:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.bWIoHIpPL3 2024-02-13T14:00:34.249 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod 0666 /tmp/tmp.bWIoHIpPL3 2024-02-13T14:00:34.386 DEBUG:teuthology.orchestra.remote:smithi148:/tmp/tmp.bWIoHIpPL3 is 6KB 2024-02-13T14:00:34.437 DEBUG:teuthology.orchestra.run.smithi148:> rm -fr /tmp/tmp.bWIoHIpPL3 2024-02-13T14:00:34.451 DEBUG:teuthology.orchestra.run.smithi148:> sudo /bin/ls /boot/loader/entries || true 2024-02-13T14:00:34.516 INFO:teuthology.orchestra.run.smithi148.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-02-13T14:00:34.516 INFO:teuthology.orchestra.run.smithi148.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-02-13T14:00:34.516 INFO:teuthology.orchestra.run.smithi148.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-02-13T14:00:34.516 INFO:teuthology.orchestra.run.smithi148.stdout:15949e3c88704300882482eddfd7baae-5.14.0-419.el9.x86_64.conf 2024-02-13T14:00:34.517 DEBUG:teuthology.orchestra.run.smithi148:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-419.el9.x86_64 2024-02-13T14:00:34.668 DEBUG:teuthology.orchestra.run.smithi148:> sudo shutdown -r now 2024-02-13T14:00:35.681 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: kernel-core-5.14.0-419.el9.x86_64 4/4 2024-02-13T14:00:35.681 INFO:teuthology.orchestra.run.smithi087.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-02-13T14:00:35.681 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-02-13T14:00:35.763 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: kernel-modules-5.14.0-419.el9.x86_64 4/4 2024-02-13T14:00:36.255 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: kernel-5.14.0-419.el9.x86_64 4/4 2024-02-13T14:00:36.255 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : kernel-5.14.0-419.el9.x86_64 1/4 2024-02-13T14:00:36.255 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : kernel-core-5.14.0-419.el9.x86_64 2/4 2024-02-13T14:00:36.255 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : kernel-modules-5.14.0-419.el9.x86_64 3/4 2024-02-13T14:00:36.703 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : kernel-modules-core-5.14.0-419.el9.x86_64 4/4 2024-02-13T14:00:36.703 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-02-13T14:00:36.703 INFO:teuthology.orchestra.run.smithi087.stdout:Installed: 2024-02-13T14:00:36.703 INFO:teuthology.orchestra.run.smithi087.stdout: kernel-5.14.0-419.el9.x86_64 2024-02-13T14:00:36.703 INFO:teuthology.orchestra.run.smithi087.stdout: kernel-core-5.14.0-419.el9.x86_64 2024-02-13T14:00:36.704 INFO:teuthology.orchestra.run.smithi087.stdout: kernel-modules-5.14.0-419.el9.x86_64 2024-02-13T14:00:36.704 INFO:teuthology.orchestra.run.smithi087.stdout: kernel-modules-core-5.14.0-419.el9.x86_64 2024-02-13T14:00:36.704 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-02-13T14:00:36.704 INFO:teuthology.orchestra.run.smithi087.stdout:Complete! 2024-02-13T14:00:37.239 DEBUG:teuthology.orchestra.run.smithi087:> rpm -q kernel | sort -rV | head -n 1 2024-02-13T14:00:37.364 INFO:teuthology.orchestra.run.smithi087.stdout:kernel-5.14.0-419.el9.x86_64 2024-02-13T14:00:37.365 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-419.el9.x86_64 2024-02-13T14:00:37.365 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-419.el9.x86_64 2024-02-13T14:00:37.365 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-02-13T14:00:37.365 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-02-13T14:00:37.365 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-02-13T14:00:37.366 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi087.front.sepia.ceph.com, path=None, version=distro) 2024-02-13T14:00:37.366 DEBUG:teuthology.orchestra.run.smithi087:> rpm -q kernel | sort -rV | head -n 1 2024-02-13T14:00:37.473 INFO:teuthology.orchestra.run.smithi087.stdout:kernel-5.14.0-419.el9.x86_64 2024-02-13T14:00:37.473 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-419.el9.x86_64 2024-02-13T14:00:37.473 DEBUG:teuthology.orchestra.run.smithi087:> sudo rpm -qi grub2-tools 2024-02-13T14:00:37.511 INFO:teuthology.orchestra.run.smithi087.stdout:Name : grub2-tools 2024-02-13T14:00:37.512 INFO:teuthology.orchestra.run.smithi087.stdout:Epoch : 1 2024-02-13T14:00:37.512 INFO:teuthology.orchestra.run.smithi087.stdout:Version : 2.06 2024-02-13T14:00:37.512 INFO:teuthology.orchestra.run.smithi087.stdout:Release : 68.el9 2024-02-13T14:00:37.512 INFO:teuthology.orchestra.run.smithi087.stdout:Architecture: x86_64 2024-02-13T14:00:37.512 INFO:teuthology.orchestra.run.smithi087.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-02-13T14:00:37.512 INFO:teuthology.orchestra.run.smithi087.stdout:Group : Unspecified 2024-02-13T14:00:37.512 INFO:teuthology.orchestra.run.smithi087.stdout:Size : 8264254 2024-02-13T14:00:37.512 INFO:teuthology.orchestra.run.smithi087.stdout:License : GPLv3+ 2024-02-13T14:00:37.512 INFO:teuthology.orchestra.run.smithi087.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-02-13T14:00:37.512 INFO:teuthology.orchestra.run.smithi087.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-02-13T14:00:37.512 INFO:teuthology.orchestra.run.smithi087.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-02-13T14:00:37.512 INFO:teuthology.orchestra.run.smithi087.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-02-13T14:00:37.512 INFO:teuthology.orchestra.run.smithi087.stdout:Packager : builder@centos.org 2024-02-13T14:00:37.512 INFO:teuthology.orchestra.run.smithi087.stdout:Vendor : CentOS 2024-02-13T14:00:37.513 INFO:teuthology.orchestra.run.smithi087.stdout:URL : http://www.gnu.org/software/grub/ 2024-02-13T14:00:37.513 INFO:teuthology.orchestra.run.smithi087.stdout:Summary : Support tools for GRUB. 2024-02-13T14:00:37.513 INFO:teuthology.orchestra.run.smithi087.stdout:Description : 2024-02-13T14:00:37.513 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-02-13T14:00:37.513 INFO:teuthology.orchestra.run.smithi087.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-02-13T14:00:37.513 INFO:teuthology.orchestra.run.smithi087.stdout:customizable bootloader with modular architecture. It supports a rich 2024-02-13T14:00:37.513 INFO:teuthology.orchestra.run.smithi087.stdout:variety of kernel formats, file systems, computer architectures and 2024-02-13T14:00:37.513 INFO:teuthology.orchestra.run.smithi087.stdout:hardware devices. 2024-02-13T14:00:37.513 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-02-13T14:00:37.513 INFO:teuthology.orchestra.run.smithi087.stdout:This subpackage provides tools for support of all platforms. 2024-02-13T14:00:37.514 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-02-13T14:00:37.514 INFO:teuthology.task.kernel:Updating grub on smithi087 to boot 5.14.0-419.el9.x86_64 2024-02-13T14:00:37.514 DEBUG:teuthology.orchestra.run.smithi087:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-02-13T14:00:38.117 INFO:teuthology.orchestra.run.smithi087.stderr:Generating grub configuration file ... 2024-02-13T14:00:39.372 INFO:teuthology.orchestra.run.smithi087.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-02-13T14:00:39.407 INFO:teuthology.orchestra.run.smithi087.stderr:done 2024-02-13T14:00:39.409 DEBUG:teuthology.orchestra.run.smithi087:> mktemp 2024-02-13T14:00:39.424 INFO:teuthology.orchestra.run.smithi087.stdout:/tmp/tmp.CEn9VQou4v 2024-02-13T14:00:39.425 DEBUG:teuthology.orchestra.run.smithi087:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.CEn9VQou4v 2024-02-13T14:00:39.493 DEBUG:teuthology.orchestra.run.smithi087:> sudo chmod 0666 /tmp/tmp.CEn9VQou4v 2024-02-13T14:00:39.637 DEBUG:teuthology.orchestra.remote:smithi087:/tmp/tmp.CEn9VQou4v is 6KB 2024-02-13T14:00:39.686 DEBUG:teuthology.orchestra.run.smithi087:> rm -fr /tmp/tmp.CEn9VQou4v 2024-02-13T14:00:39.701 DEBUG:teuthology.orchestra.run.smithi087:> sudo /bin/ls /boot/loader/entries || true 2024-02-13T14:00:39.771 INFO:teuthology.orchestra.run.smithi087.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-02-13T14:00:39.772 INFO:teuthology.orchestra.run.smithi087.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-02-13T14:00:39.772 INFO:teuthology.orchestra.run.smithi087.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-02-13T14:00:39.772 INFO:teuthology.orchestra.run.smithi087.stdout:15949e3c88704300882482eddfd7baae-5.14.0-419.el9.x86_64.conf 2024-02-13T14:00:39.773 DEBUG:teuthology.orchestra.run.smithi087:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-419.el9.x86_64 2024-02-13T14:00:39.925 DEBUG:teuthology.orchestra.run.smithi087:> sudo shutdown -r now 2024-02-13T14:01:04.696 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-02-13T14:01:04.697 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi148.front.sepia.ceph.com' 2024-02-13T14:01:04.697 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-13T14:01:09.930 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-02-13T14:01:09.930 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi087.front.sepia.ceph.com' 2024-02-13T14:01:09.931 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi087.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-13T14:01:23.260 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.148 2024-02-13T14:01:28.384 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.87 2024-02-13T14:01:32.265 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi148.front.sepia.ceph.com' 2024-02-13T14:01:32.266 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-13T14:01:35.328 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.148 2024-02-13T14:01:37.387 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi087.front.sepia.ceph.com' 2024-02-13T14:01:37.388 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi087.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-13T14:01:40.444 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.87 2024-02-13T14:01:47.334 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi148.front.sepia.ceph.com' 2024-02-13T14:01:47.335 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-13T14:01:52.450 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi087.front.sepia.ceph.com' 2024-02-13T14:01:52.451 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi087.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-13T14:02:05.756 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.148 2024-02-13T14:02:10.876 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.87 2024-02-13T14:02:20.767 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi148.front.sepia.ceph.com' 2024-02-13T14:02:20.769 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-13T14:02:22.002 DEBUG:teuthology.orchestra.run.smithi148:> true 2024-02-13T14:02:22.497 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi148.front.sepia.ceph.com' 2024-02-13T14:02:22.498 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-419.el9.x86_64"... 2024-02-13T14:02:22.498 DEBUG:teuthology.orchestra.run.smithi148:> uname -r 2024-02-13T14:02:22.512 INFO:teuthology.orchestra.run.smithi148.stdout:5.14.0-419.el9.x86_64 2024-02-13T14:02:22.512 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-419.el9.x86_64 vs 5.14.0-419.el9.x86_64 2024-02-13T14:02:22.512 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-02-13T14:02:22.512 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-02-13T14:02:23.513 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-02-13T14:02:23.513 DEBUG:teuthology.orchestra.run.smithi148:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-02-13T14:02:23.588 INFO:teuthology.orchestra.run.smithi148.stdout:ttyS1 2024-02-13T14:02:23.616 DEBUG:teuthology.parallel:result is None 2024-02-13T14:02:25.876 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi087.front.sepia.ceph.com' 2024-02-13T14:02:25.877 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi087.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-13T14:02:26.268 DEBUG:teuthology.orchestra.run.smithi087:> true 2024-02-13T14:02:26.637 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi087.front.sepia.ceph.com' 2024-02-13T14:02:26.637 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-419.el9.x86_64"... 2024-02-13T14:02:26.638 DEBUG:teuthology.orchestra.run.smithi087:> uname -r 2024-02-13T14:02:26.651 INFO:teuthology.orchestra.run.smithi087.stdout:5.14.0-419.el9.x86_64 2024-02-13T14:02:26.652 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-419.el9.x86_64 vs 5.14.0-419.el9.x86_64 2024-02-13T14:02:26.652 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-02-13T14:02:26.652 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-02-13T14:02:27.652 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-02-13T14:02:27.653 DEBUG:teuthology.orchestra.run.smithi087:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-02-13T14:02:27.713 INFO:teuthology.orchestra.run.smithi087.stdout:ttyS1 2024-02-13T14:02:27.759 DEBUG:teuthology.parallel:result is None 2024-02-13T14:02:27.759 INFO:teuthology.run_tasks:Running task internal.base... 2024-02-13T14:02:27.767 INFO:teuthology.task.internal:Creating test directory... 2024-02-13T14:02:27.767 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-02-13T14:02:27.769 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-02-13T14:02:27.784 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-02-13T14:02:27.810 INFO:teuthology.run_tasks:Running task internal.archive... 2024-02-13T14:02:27.842 INFO:teuthology.task.internal:Creating archive directory... 2024-02-13T14:02:27.842 DEBUG:teuthology.orchestra.run.smithi087:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-02-13T14:02:27.844 DEBUG:teuthology.orchestra.run.smithi148:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-02-13T14:02:27.876 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-02-13T14:02:27.882 INFO:teuthology.task.internal:Enabling coredump saving... 2024-02-13T14:02:27.882 DEBUG:teuthology.orchestra.run.smithi087:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-02-13T14:02:27.917 DEBUG:teuthology.orchestra.run.smithi148:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-02-13T14:02:27.949 INFO:teuthology.orchestra.run.smithi087.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-13T14:02:27.950 INFO:teuthology.orchestra.run.smithi148.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-13T14:02:27.962 INFO:teuthology.orchestra.run.smithi087.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-13T14:02:27.969 INFO:teuthology.orchestra.run.smithi148.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-13T14:02:27.971 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-02-13T14:02:27.978 INFO:teuthology.task.internal:Configuring sudo... 2024-02-13T14:02:27.978 DEBUG:teuthology.orchestra.run.smithi087:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-02-13T14:02:28.010 DEBUG:teuthology.orchestra.run.smithi148:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-02-13T14:02:28.037 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-02-13T14:02:28.047 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-02-13T14:02:28.047 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-02-13T14:02:28.080 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-02-13T14:02:28.094 DEBUG:teuthology.orchestra.run.smithi087:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-13T14:02:28.178 DEBUG:teuthology.orchestra.run.smithi087:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-13T14:02:28.228 DEBUG:teuthology.orchestra.run.smithi087:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-13T14:02:28.313 DEBUG:teuthology.orchestra.run.smithi087:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-13T14:02:28.360 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-02-13T14:02:28.360 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-02-13T14:02:28.428 DEBUG:teuthology.orchestra.run.smithi148:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-13T14:02:28.464 DEBUG:teuthology.orchestra.run.smithi148:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-13T14:02:28.540 DEBUG:teuthology.orchestra.run.smithi148:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-13T14:02:28.577 DEBUG:teuthology.orchestra.run.smithi148:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-13T14:02:28.639 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-02-13T14:02:28.639 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-02-13T14:02:28.723 DEBUG:teuthology.orchestra.run.smithi087:> sudo service rsyslog restart 2024-02-13T14:02:28.728 DEBUG:teuthology.orchestra.run.smithi148:> sudo service rsyslog restart 2024-02-13T14:02:28.789 INFO:teuthology.orchestra.run.smithi087.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-13T14:02:28.952 INFO:teuthology.orchestra.run.smithi148.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-13T14:02:29.351 INFO:teuthology.run_tasks:Running task internal.timer... 2024-02-13T14:02:29.358 INFO:teuthology.task.internal:Starting timer... 2024-02-13T14:02:29.359 INFO:teuthology.run_tasks:Running task pcp... 2024-02-13T14:02:29.406 INFO:teuthology.run_tasks:Running task selinux... 2024-02-13T14:02:29.437 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-02-13T14:02:29.437 DEBUG:teuthology.orchestra.run.smithi087:> sudo service auditd rotate 2024-02-13T14:02:29.517 INFO:teuthology.orchestra.run.smithi087.stdout:Rotating logs: 2024-02-13T14:02:29.523 DEBUG:teuthology.orchestra.run.smithi148:> sudo service auditd rotate 2024-02-13T14:02:29.599 INFO:teuthology.orchestra.run.smithi148.stdout:Rotating logs: 2024-02-13T14:02:29.601 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-02-13T14:02:29.601 DEBUG:teuthology.orchestra.run.smithi087:> /usr/sbin/getenforce 2024-02-13T14:02:29.631 INFO:teuthology.orchestra.run.smithi087.stdout:Permissive 2024-02-13T14:02:29.632 DEBUG:teuthology.orchestra.run.smithi148:> /usr/sbin/getenforce 2024-02-13T14:02:29.657 INFO:teuthology.orchestra.run.smithi148.stdout:Permissive 2024-02-13T14:02:29.657 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi087.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi148.front.sepia.ceph.com': 'permissive'} 2024-02-13T14:02:29.657 DEBUG:teuthology.orchestra.run.smithi087:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-13T14:02:29.699 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-13T14:02:29.699 DEBUG:teuthology.orchestra.run.smithi148:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-13T14:02:29.727 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-13T14:02:29.727 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-02-13T14:02:29.727 DEBUG:teuthology.orchestra.run.smithi087:> sudo /usr/sbin/setenforce permissive 2024-02-13T14:02:29.775 DEBUG:teuthology.orchestra.run.smithi148:> sudo /usr/sbin/setenforce permissive 2024-02-13T14:02:29.809 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-02-13T14:02:29.818 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-02-13T14:02:29.838 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-02-13T14:02:29.840 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi087.front.sepia.ceph.com,smithi148.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-02-13T14:06:40.312 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi087.front.sepia.ceph.com'), Remote(name='ubuntu@smithi148.front.sepia.ceph.com')] 2024-02-13T14:06:40.314 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi087.front.sepia.ceph.com' 2024-02-13T14:06:40.315 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi087.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-13T14:06:40.406 DEBUG:teuthology.orchestra.run.smithi087:> true 2024-02-13T14:06:40.487 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi087.front.sepia.ceph.com' 2024-02-13T14:06:40.487 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi148.front.sepia.ceph.com' 2024-02-13T14:06:40.488 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-13T14:06:40.585 DEBUG:teuthology.orchestra.run.smithi148:> true 2024-02-13T14:06:40.659 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi148.front.sepia.ceph.com' 2024-02-13T14:06:40.659 INFO:teuthology.run_tasks:Running task clock... 2024-02-13T14:06:40.671 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-02-13T14:06:40.671 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-02-13T14:06:40.671 DEBUG:teuthology.orchestra.run.smithi087:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-13T14:06:40.674 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-02-13T14:06:40.674 DEBUG:teuthology.orchestra.run.smithi148:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-13T14:06:40.703 INFO:teuthology.orchestra.run.smithi087.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-02-13T14:06:40.720 INFO:teuthology.orchestra.run.smithi087.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-02-13T14:06:40.730 INFO:teuthology.orchestra.run.smithi148.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-02-13T14:06:40.747 INFO:teuthology.orchestra.run.smithi148.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-02-13T14:06:40.750 INFO:teuthology.orchestra.run.smithi087.stderr:sudo: ntpd: command not found 2024-02-13T14:06:40.765 INFO:teuthology.orchestra.run.smithi087.stdout:506 Cannot talk to daemon 2024-02-13T14:06:40.779 INFO:teuthology.orchestra.run.smithi148.stderr:sudo: ntpd: command not found 2024-02-13T14:06:40.780 INFO:teuthology.orchestra.run.smithi087.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-02-13T14:06:40.793 INFO:teuthology.orchestra.run.smithi148.stdout:506 Cannot talk to daemon 2024-02-13T14:06:40.796 INFO:teuthology.orchestra.run.smithi087.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-02-13T14:06:40.810 INFO:teuthology.orchestra.run.smithi148.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-02-13T14:06:40.826 INFO:teuthology.orchestra.run.smithi148.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-02-13T14:06:40.846 INFO:teuthology.orchestra.run.smithi087.stderr:bash: line 1: ntpq: command not found 2024-02-13T14:06:40.849 INFO:teuthology.orchestra.run.smithi087.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-13T14:06:40.849 INFO:teuthology.orchestra.run.smithi087.stdout:=============================================================================== 2024-02-13T14:06:40.849 INFO:teuthology.orchestra.run.smithi087.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-13T14:06:40.849 INFO:teuthology.orchestra.run.smithi087.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-13T14:06:40.849 INFO:teuthology.orchestra.run.smithi087.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-13T14:06:40.849 INFO:teuthology.orchestra.run.smithi087.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-13T14:06:40.882 INFO:teuthology.orchestra.run.smithi148.stderr:bash: line 1: ntpq: command not found 2024-02-13T14:06:40.885 INFO:teuthology.orchestra.run.smithi148.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-13T14:06:40.885 INFO:teuthology.orchestra.run.smithi148.stdout:=============================================================================== 2024-02-13T14:06:40.885 INFO:teuthology.orchestra.run.smithi148.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-13T14:06:40.885 INFO:teuthology.orchestra.run.smithi148.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-13T14:06:40.885 INFO:teuthology.orchestra.run.smithi148.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-13T14:06:40.885 INFO:teuthology.orchestra.run.smithi148.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-13T14:06:40.886 INFO:teuthology.run_tasks:Running task pexec... 2024-02-13T14:06:40.898 INFO:teuthology.task.pexec:Executing custom commands... 2024-02-13T14:06:40.898 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi087.front.sepia.ceph.com 2024-02-13T14:06:40.899 DEBUG:teuthology.orchestra.run.smithi087:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-02-13T14:06:40.899 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi148.front.sepia.ceph.com 2024-02-13T14:06:40.899 DEBUG:teuthology.orchestra.run.smithi148:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-02-13T14:06:41.514 INFO:teuthology.orchestra.run.smithi087.stdout:Last metadata expiration check: 0:00:48 ago on Tue 13 Feb 2024 02:05:53 PM UTC. 2024-02-13T14:06:41.538 INFO:teuthology.orchestra.run.smithi148.stdout:Last metadata expiration check: 0:00:49 ago on Tue 13 Feb 2024 02:05:52 PM UTC. 2024-02-13T14:06:41.669 INFO:teuthology.orchestra.run.smithi087.stdout:Dependencies resolved. 2024-02-13T14:06:41.678 INFO:teuthology.orchestra.run.smithi087.stdout:================================================================================ 2024-02-13T14:06:41.678 INFO:teuthology.orchestra.run.smithi087.stdout: Package Architecture Version Repository Size 2024-02-13T14:06:41.679 INFO:teuthology.orchestra.run.smithi087.stdout:================================================================================ 2024-02-13T14:06:41.679 INFO:teuthology.orchestra.run.smithi087.stdout:Installing: 2024-02-13T14:06:41.679 INFO:teuthology.orchestra.run.smithi087.stdout: runc x86_64 4:1.1.12-1.el9 CentOS-AppStream 3.1 M 2024-02-13T14:06:41.679 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-02-13T14:06:41.679 INFO:teuthology.orchestra.run.smithi087.stdout:Transaction Summary 2024-02-13T14:06:41.679 INFO:teuthology.orchestra.run.smithi087.stdout:================================================================================ 2024-02-13T14:06:41.679 INFO:teuthology.orchestra.run.smithi087.stdout:Install 1 Package 2024-02-13T14:06:41.679 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-02-13T14:06:41.679 INFO:teuthology.orchestra.run.smithi087.stdout:Total download size: 3.1 M 2024-02-13T14:06:41.679 INFO:teuthology.orchestra.run.smithi087.stdout:Installed size: 9.8 M 2024-02-13T14:06:41.679 INFO:teuthology.orchestra.run.smithi087.stdout:Downloading Packages: 2024-02-13T14:06:41.692 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-02-13T14:06:41.706 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-02-13T14:06:41.706 INFO:teuthology.orchestra.run.smithi148.stdout: Package Architecture Version Repository Size 2024-02-13T14:06:41.706 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-02-13T14:06:41.706 INFO:teuthology.orchestra.run.smithi148.stdout:Installing: 2024-02-13T14:06:41.707 INFO:teuthology.orchestra.run.smithi148.stdout: runc x86_64 4:1.1.12-1.el9 CentOS-AppStream 3.1 M 2024-02-13T14:06:41.707 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-02-13T14:06:41.707 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction Summary 2024-02-13T14:06:41.707 INFO:teuthology.orchestra.run.smithi148.stdout:================================================================================ 2024-02-13T14:06:41.707 INFO:teuthology.orchestra.run.smithi148.stdout:Install 1 Package 2024-02-13T14:06:41.707 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-02-13T14:06:41.707 INFO:teuthology.orchestra.run.smithi148.stdout:Total download size: 3.1 M 2024-02-13T14:06:41.707 INFO:teuthology.orchestra.run.smithi148.stdout:Installed size: 9.8 M 2024-02-13T14:06:41.707 INFO:teuthology.orchestra.run.smithi148.stdout:Downloading Packages: 2024-02-13T14:06:42.212 INFO:teuthology.orchestra.run.smithi087.stdout:runc-1.1.12-1.el9.x86_64.rpm 5.9 MB/s | 3.1 MB 00:00 2024-02-13T14:06:42.212 INFO:teuthology.orchestra.run.smithi087.stdout:-------------------------------------------------------------------------------- 2024-02-13T14:06:42.213 INFO:teuthology.orchestra.run.smithi087.stdout:Total 5.8 MB/s | 3.1 MB 00:00 2024-02-13T14:06:42.213 INFO:teuthology.orchestra.run.smithi087.stdout:Running transaction check 2024-02-13T14:06:42.218 INFO:teuthology.orchestra.run.smithi087.stdout:Transaction check succeeded. 2024-02-13T14:06:42.218 INFO:teuthology.orchestra.run.smithi087.stdout:Running transaction test 2024-02-13T14:06:42.254 INFO:teuthology.orchestra.run.smithi148.stdout:runc-1.1.12-1.el9.x86_64.rpm 5.7 MB/s | 3.1 MB 00:00 2024-02-13T14:06:42.254 INFO:teuthology.orchestra.run.smithi148.stdout:-------------------------------------------------------------------------------- 2024-02-13T14:06:42.255 INFO:teuthology.orchestra.run.smithi148.stdout:Total 5.7 MB/s | 3.1 MB 00:00 2024-02-13T14:06:42.255 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction check 2024-02-13T14:06:42.261 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction check succeeded. 2024-02-13T14:06:42.261 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction test 2024-02-13T14:06:42.289 INFO:teuthology.orchestra.run.smithi087.stdout:Transaction test succeeded. 2024-02-13T14:06:42.289 INFO:teuthology.orchestra.run.smithi087.stdout:Running transaction 2024-02-13T14:06:42.331 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction test succeeded. 2024-02-13T14:06:42.332 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction 2024-02-13T14:06:42.505 INFO:teuthology.orchestra.run.smithi087.stdout: Preparing : 1/1 2024-02-13T14:06:42.529 INFO:teuthology.orchestra.run.smithi148.stdout: Preparing : 1/1 2024-02-13T14:06:42.598 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : runc-4:1.1.12-1.el9.x86_64 1/1 2024-02-13T14:06:42.616 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : runc-4:1.1.12-1.el9.x86_64 1/1 2024-02-13T14:06:43.183 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: runc-4:1.1.12-1.el9.x86_64 1/1 2024-02-13T14:06:43.248 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: runc-4:1.1.12-1.el9.x86_64 1/1 2024-02-13T14:06:43.510 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : runc-4:1.1.12-1.el9.x86_64 1/1 2024-02-13T14:06:43.510 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-02-13T14:06:43.510 INFO:teuthology.orchestra.run.smithi087.stdout:Installed: 2024-02-13T14:06:43.510 INFO:teuthology.orchestra.run.smithi087.stdout: runc-4:1.1.12-1.el9.x86_64 2024-02-13T14:06:43.510 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-02-13T14:06:43.510 INFO:teuthology.orchestra.run.smithi087.stdout:Complete! 2024-02-13T14:06:43.587 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : runc-4:1.1.12-1.el9.x86_64 1/1 2024-02-13T14:06:43.587 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-02-13T14:06:43.587 INFO:teuthology.orchestra.run.smithi148.stdout:Installed: 2024-02-13T14:06:43.587 INFO:teuthology.orchestra.run.smithi148.stdout: runc-4:1.1.12-1.el9.x86_64 2024-02-13T14:06:43.587 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-02-13T14:06:43.587 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-02-13T14:06:43.761 DEBUG:teuthology.parallel:result is None 2024-02-13T14:06:43.851 DEBUG:teuthology.parallel:result is None 2024-02-13T14:06:43.851 INFO:teuthology.run_tasks:Running task install... 2024-02-13T14:06:43.861 DEBUG:teuthology.task.install:project ceph 2024-02-13T14:06:43.861 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'extra_system_packages': {'deb': ['python3-pytest'], 'rpm': ['python3-pytest']}, 'flavor': 'default', 'sha1': 'e1b79d5a4193a484176863ab96c0ca28c00e1662'}} 2024-02-13T14:06:43.861 DEBUG:teuthology.task.install:config {'extra_system_packages': {'deb': ['python3-pytest'], 'rpm': ['python3-pytest']}, 'flavor': 'default', 'sha1': 'e1b79d5a4193a484176863ab96c0ca28c00e1662'} 2024-02-13T14:06:43.861 INFO:teuthology.task.install:Using flavor: default 2024-02-13T14:06:43.867 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-02-13T14:06:43.867 INFO:teuthology.task.install:extra packages: [] 2024-02-13T14:06:43.868 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': {'deb': ['python3-pytest'], 'rpm': ['python3-pytest']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'e1b79d5a4193a484176863ab96c0ca28c00e1662', 'tag': None, 'wait_for_package': False} 2024-02-13T14:06:43.868 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=e1b79d5a4193a484176863ab96c0ca28c00e1662 2024-02-13T14:06:43.870 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': {'deb': ['python3-pytest'], 'rpm': ['python3-pytest']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'e1b79d5a4193a484176863ab96c0ca28c00e1662', 'tag': None, 'wait_for_package': False} 2024-02-13T14:06:43.871 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=e1b79d5a4193a484176863ab96c0ca28c00e1662 2024-02-13T14:06:44.095 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-phlogistonjohn-testing-2024-02-12-1654/e1b79d5a4193a484176863ab96c0ca28c00e1662/centos/9/flavors/default/ 2024-02-13T14:06:44.095 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1423.ge1b79d5a 2024-02-13T14:06:44.097 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-phlogistonjohn-testing-2024-02-12-1654/e1b79d5a4193a484176863ab96c0ca28c00e1662/centos/9/flavors/default/ 2024-02-13T14:06:44.097 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1423.ge1b79d5a 2024-02-13T14:06:44.207 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-phlogistonjohn-testing-2024-02-12-1654/e1b79d5a4193a484176863ab96c0ca28c00e1662/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-phlogistonjohn-testing-2024-02-12-1654/e1b79d5a4193a484176863ab96c0ca28c00e1662/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-phlogistonjohn-testing-2024-02-12-1654/e1b79d5a4193a484176863ab96c0ca28c00e1662/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-02-13T14:06:44.208 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-02-13T14:06:44.208 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-02-13T14:06:44.235 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-pytest, python3-pytest, python3-pytest on remote rpm x86_64 2024-02-13T14:06:44.236 DEBUG:teuthology.orchestra.run.smithi087:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/e1b79d5a4193a484176863ab96c0ca28c00e1662/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-02-13T14:06:44.267 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-phlogistonjohn-testing-2024-02-12-1654/e1b79d5a4193a484176863ab96c0ca28c00e1662/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-phlogistonjohn-testing-2024-02-12-1654/e1b79d5a4193a484176863ab96c0ca28c00e1662/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-phlogistonjohn-testing-2024-02-12-1654/e1b79d5a4193a484176863ab96c0ca28c00e1662/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-02-13T14:06:44.267 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-02-13T14:06:44.267 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-02-13T14:06:44.294 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-pytest, python3-pytest, python3-pytest, python3-pytest on remote rpm x86_64 2024-02-13T14:06:44.294 DEBUG:teuthology.orchestra.run.smithi148:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/e1b79d5a4193a484176863ab96c0ca28c00e1662/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-02-13T14:06:44.305 DEBUG:teuthology.orchestra.run.smithi087:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-02-13T14:06:44.362 DEBUG:teuthology.orchestra.run.smithi148:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-02-13T14:06:44.385 DEBUG:teuthology.orchestra.run.smithi087:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-02-13T14:06:44.451 INFO:teuthology.orchestra.run.smithi087.stdout:check_obsoletes = 1 2024-02-13T14:06:44.453 DEBUG:teuthology.orchestra.run.smithi087:> sudo yum clean all 2024-02-13T14:06:44.461 DEBUG:teuthology.orchestra.run.smithi148:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-02-13T14:06:44.487 INFO:teuthology.orchestra.run.smithi148.stdout:check_obsoletes = 1 2024-02-13T14:06:44.488 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum clean all 2024-02-13T14:06:44.691 INFO:teuthology.orchestra.run.smithi087.stdout:74 files removed 2024-02-13T14:06:44.718 DEBUG:teuthology.orchestra.run.smithi087:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-pytest python3-pytest python3-pytest python3-pytest 2024-02-13T14:06:44.727 INFO:teuthology.orchestra.run.smithi148.stdout:74 files removed 2024-02-13T14:06:44.755 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-pytest python3-pytest python3-pytest python3-pytest 2024-02-13T14:06:45.492 INFO:teuthology.orchestra.run.smithi148.stdout:ceph packages for x86_64 197 kB/s | 82 kB 00:00 2024-02-13T14:06:45.527 INFO:teuthology.orchestra.run.smithi087.stdout:ceph packages for x86_64 165 kB/s | 82 kB 00:00 2024-02-13T14:06:45.885 INFO:teuthology.orchestra.run.smithi148.stdout:ceph noarch packages 29 kB/s | 10 kB 00:00 2024-02-13T14:06:45.944 INFO:teuthology.orchestra.run.smithi087.stdout:ceph noarch packages 27 kB/s | 10 kB 00:00 2024-02-13T14:06:46.252 INFO:teuthology.orchestra.run.smithi148.stdout:ceph source packages 5.8 kB/s | 1.9 kB 00:00 2024-02-13T14:06:46.303 INFO:teuthology.orchestra.run.smithi087.stdout:ceph source packages 5.9 kB/s | 1.9 kB 00:00 2024-02-13T14:06:46.940 INFO:teuthology.orchestra.run.smithi148.stdout:CentOS-9 - AppStream 10 MB/s | 6.7 MB 00:00 2024-02-13T14:06:47.105 INFO:teuthology.orchestra.run.smithi087.stdout:CentOS-9 - AppStream 8.8 MB/s | 6.7 MB 00:00 2024-02-13T14:06:48.640 INFO:teuthology.orchestra.run.smithi148.stdout:CentOS-9 - BaseOS 3.5 MB/s | 2.0 MB 00:00 2024-02-13T14:06:48.689 INFO:teuthology.orchestra.run.smithi087.stdout:CentOS-9 - BaseOS 4.2 MB/s | 2.0 MB 00:00 2024-02-13T14:06:49.489 INFO:teuthology.orchestra.run.smithi148.stdout:CentOS Stream 9 - BaseOS 15 MB/s | 8.0 MB 00:00 2024-02-13T14:06:51.288 INFO:teuthology.orchestra.run.smithi087.stdout:CentOS Stream 9 - BaseOS 3.5 MB/s | 8.0 MB 00:02 2024-02-13T14:06:53.740 INFO:teuthology.orchestra.run.smithi148.stdout:CentOS Stream 9 - AppStream 5.9 MB/s | 19 MB 00:03 2024-02-13T14:06:54.364 INFO:teuthology.orchestra.run.smithi087.stdout:CentOS Stream 9 - AppStream 9.4 MB/s | 19 MB 00:01 2024-02-13T14:06:57.915 INFO:teuthology.orchestra.run.smithi148.stdout:CentOS Stream 9 - CRB 13 MB/s | 5.9 MB 00:00 2024-02-13T14:06:59.316 INFO:teuthology.orchestra.run.smithi148.stdout:CentOS Stream 9 - Extras packages 52 kB/s | 15 kB 00:00 2024-02-13T14:07:01.939 INFO:teuthology.orchestra.run.smithi087.stdout:CentOS Stream 9 - CRB 1.5 MB/s | 5.9 MB 00:04 2024-02-13T14:07:02.678 INFO:teuthology.orchestra.run.smithi148.stdout:Extra Packages for Enterprise Linux 6.3 MB/s | 20 MB 00:03 2024-02-13T14:07:04.357 INFO:teuthology.orchestra.run.smithi087.stdout:CentOS Stream 9 - Extras packages 12 kB/s | 15 kB 00:01 2024-02-13T14:07:07.667 INFO:teuthology.orchestra.run.smithi148.stdout:lab-extras 12 kB/s | 1.7 kB 00:00 2024-02-13T14:07:10.305 INFO:teuthology.orchestra.run.smithi148.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-02-13T14:07:10.306 INFO:teuthology.orchestra.run.smithi148.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-02-13T14:07:10.385 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-02-13T14:07:10.392 INFO:teuthology.orchestra.run.smithi148.stdout:============================================================================================ 2024-02-13T14:07:10.393 INFO:teuthology.orchestra.run.smithi148.stdout: Package Arch Version Repository Size 2024-02-13T14:07:10.393 INFO:teuthology.orchestra.run.smithi148.stdout:============================================================================================ 2024-02-13T14:07:10.393 INFO:teuthology.orchestra.run.smithi148.stdout:Installing: 2024-02-13T14:07:10.393 INFO:teuthology.orchestra.run.smithi148.stdout: ceph x86_64 2:19.0.0-1423.ge1b79d5a.el9 ceph 6.6 k 2024-02-13T14:07:10.393 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-base x86_64 2:19.0.0-1423.ge1b79d5a.el9 ceph 5.3 M 2024-02-13T14:07:10.393 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-fuse x86_64 2:19.0.0-1423.ge1b79d5a.el9 ceph 885 k 2024-02-13T14:07:10.393 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1423.ge1b79d5a.el9 ceph 145 k 2024-02-13T14:07:10.393 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr x86_64 2:19.0.0-1423.ge1b79d5a.el9 ceph 1.5 M 2024-02-13T14:07:10.393 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1423.ge1b79d5a.el9 ceph-noarch 139 k 2024-02-13T14:07:10.393 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1423.ge1b79d5a.el9 ceph-noarch 1.8 M 2024-02-13T14:07:10.393 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1423.ge1b79d5a.el9 ceph-noarch 7.4 M 2024-02-13T14:07:10.393 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-rook noarch 2:19.0.0-1423.ge1b79d5a.el9 ceph-noarch 52 k 2024-02-13T14:07:10.394 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-radosgw x86_64 2:19.0.0-1423.ge1b79d5a.el9 ceph 9.7 M 2024-02-13T14:07:10.394 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-test x86_64 2:19.0.0-1423.ge1b79d5a.el9 ceph 47 M 2024-02-13T14:07:10.394 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-volume noarch 2:19.0.0-1423.ge1b79d5a.el9 ceph-noarch 264 k 2024-02-13T14:07:10.394 INFO:teuthology.orchestra.run.smithi148.stdout: cephadm noarch 2:19.0.0-1423.ge1b79d5a.el9 ceph-noarch 760 k 2024-02-13T14:07:10.394 INFO:teuthology.orchestra.run.smithi148.stdout: libcephfs-devel x86_64 2:19.0.0-1423.ge1b79d5a.el9 ceph 31 k 2024-02-13T14:07:10.394 INFO:teuthology.orchestra.run.smithi148.stdout: libcephfs2 x86_64 2:19.0.0-1423.ge1b79d5a.el9 ceph 743 k 2024-02-13T14:07:10.394 INFO:teuthology.orchestra.run.smithi148.stdout: librados-devel x86_64 2:19.0.0-1423.ge1b79d5a.el9 ceph 127 k 2024-02-13T14:07:10.394 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cephfs x86_64 2:19.0.0-1423.ge1b79d5a.el9 ceph 162 k 2024-02-13T14:07:10.394 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pytest noarch 6.2.2-6.el9 CentOS-AppStream 474 k 2024-02-13T14:07:10.394 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rados x86_64 2:19.0.0-1423.ge1b79d5a.el9 ceph 321 k 2024-02-13T14:07:10.394 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rbd x86_64 2:19.0.0-1423.ge1b79d5a.el9 ceph 298 k 2024-02-13T14:07:10.394 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rgw x86_64 2:19.0.0-1423.ge1b79d5a.el9 ceph 99 k 2024-02-13T14:07:10.394 INFO:teuthology.orchestra.run.smithi148.stdout: rbd-fuse x86_64 2:19.0.0-1423.ge1b79d5a.el9 ceph 85 k 2024-02-13T14:07:10.395 INFO:teuthology.orchestra.run.smithi148.stdout: rbd-mirror x86_64 2:19.0.0-1423.ge1b79d5a.el9 ceph 3.1 M 2024-02-13T14:07:10.395 INFO:teuthology.orchestra.run.smithi148.stdout: rbd-nbd x86_64 2:19.0.0-1423.ge1b79d5a.el9 ceph 171 k 2024-02-13T14:07:10.395 INFO:teuthology.orchestra.run.smithi148.stdout:Upgrading: 2024-02-13T14:07:10.395 INFO:teuthology.orchestra.run.smithi148.stdout: librados2 x86_64 2:19.0.0-1423.ge1b79d5a.el9 ceph 3.4 M 2024-02-13T14:07:10.395 INFO:teuthology.orchestra.run.smithi148.stdout: librbd1 x86_64 2:19.0.0-1423.ge1b79d5a.el9 ceph 3.1 M 2024-02-13T14:07:10.395 INFO:teuthology.orchestra.run.smithi148.stdout:Installing dependencies: 2024-02-13T14:07:10.395 INFO:teuthology.orchestra.run.smithi148.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-02-13T14:07:10.395 INFO:teuthology.orchestra.run.smithi148.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-02-13T14:07:10.395 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-common x86_64 2:19.0.0-1423.ge1b79d5a.el9 ceph 20 M 2024-02-13T14:07:10.395 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1423.ge1b79d5a.el9 ceph-noarch 31 k 2024-02-13T14:07:10.395 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mds x86_64 2:19.0.0-1423.ge1b79d5a.el9 ceph 2.1 M 2024-02-13T14:07:10.395 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1423.ge1b79d5a.el9 ceph-noarch 248 k 2024-02-13T14:07:10.395 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mon x86_64 2:19.0.0-1423.ge1b79d5a.el9 ceph 4.7 M 2024-02-13T14:07:10.396 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-osd x86_64 2:19.0.0-1423.ge1b79d5a.el9 ceph 17 M 2024-02-13T14:07:10.396 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1423.ge1b79d5a.el9 ceph-noarch 15 k 2024-02-13T14:07:10.396 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-selinux x86_64 2:19.0.0-1423.ge1b79d5a.el9 ceph 25 k 2024-02-13T14:07:10.396 INFO:teuthology.orchestra.run.smithi148.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-02-13T14:07:10.396 INFO:teuthology.orchestra.run.smithi148.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-02-13T14:07:10.396 INFO:teuthology.orchestra.run.smithi148.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-02-13T14:07:10.396 INFO:teuthology.orchestra.run.smithi148.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-02-13T14:07:10.396 INFO:teuthology.orchestra.run.smithi148.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-02-13T14:07:10.396 INFO:teuthology.orchestra.run.smithi148.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-02-13T14:07:10.396 INFO:teuthology.orchestra.run.smithi148.stdout: jq x86_64 1.6-16.el9 CentOS-BaseOS 186 k 2024-02-13T14:07:10.396 INFO:teuthology.orchestra.run.smithi148.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-02-13T14:07:10.396 INFO:teuthology.orchestra.run.smithi148.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-02-13T14:07:10.397 INFO:teuthology.orchestra.run.smithi148.stdout: libcephsqlite x86_64 2:19.0.0-1423.ge1b79d5a.el9 ceph 163 k 2024-02-13T14:07:10.397 INFO:teuthology.orchestra.run.smithi148.stdout: libgfortran x86_64 11.4.1-3.el9 CentOS-BaseOS 802 k 2024-02-13T14:07:10.397 INFO:teuthology.orchestra.run.smithi148.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-02-13T14:07:10.397 INFO:teuthology.orchestra.run.smithi148.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-02-13T14:07:10.397 INFO:teuthology.orchestra.run.smithi148.stdout: libquadmath x86_64 11.4.1-3.el9 CentOS-BaseOS 194 k 2024-02-13T14:07:10.397 INFO:teuthology.orchestra.run.smithi148.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-02-13T14:07:10.397 INFO:teuthology.orchestra.run.smithi148.stdout: libradosstriper1 x86_64 2:19.0.0-1423.ge1b79d5a.el9 ceph 499 k 2024-02-13T14:07:10.397 INFO:teuthology.orchestra.run.smithi148.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-02-13T14:07:10.397 INFO:teuthology.orchestra.run.smithi148.stdout: librgw2 x86_64 2:19.0.0-1423.ge1b79d5a.el9 ceph 5.1 M 2024-02-13T14:07:10.397 INFO:teuthology.orchestra.run.smithi148.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-02-13T14:07:10.397 INFO:teuthology.orchestra.run.smithi148.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-02-13T14:07:10.397 INFO:teuthology.orchestra.run.smithi148.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-02-13T14:07:10.397 INFO:teuthology.orchestra.run.smithi148.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-02-13T14:07:10.397 INFO:teuthology.orchestra.run.smithi148.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-02-13T14:07:10.397 INFO:teuthology.orchestra.run.smithi148.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-BaseOS 217 k 2024-02-13T14:07:10.397 INFO:teuthology.orchestra.run.smithi148.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-02-13T14:07:10.398 INFO:teuthology.orchestra.run.smithi148.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-02-13T14:07:10.398 INFO:teuthology.orchestra.run.smithi148.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-02-13T14:07:10.398 INFO:teuthology.orchestra.run.smithi148.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-02-13T14:07:10.398 INFO:teuthology.orchestra.run.smithi148.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-02-13T14:07:10.398 INFO:teuthology.orchestra.run.smithi148.stdout: python3-asyncssh noarch 2.13.2-3.el9 epel 548 k 2024-02-13T14:07:10.398 INFO:teuthology.orchestra.run.smithi148.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-02-13T14:07:10.398 INFO:teuthology.orchestra.run.smithi148.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-02-13T14:07:10.398 INFO:teuthology.orchestra.run.smithi148.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-02-13T14:07:10.398 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-02-13T14:07:10.398 INFO:teuthology.orchestra.run.smithi148.stdout: python3-ceph-argparse x86_64 2:19.0.0-1423.ge1b79d5a.el9 ceph 45 k 2024-02-13T14:07:10.398 INFO:teuthology.orchestra.run.smithi148.stdout: python3-ceph-common x86_64 2:19.0.0-1423.ge1b79d5a.el9 ceph 132 k 2024-02-13T14:07:10.398 INFO:teuthology.orchestra.run.smithi148.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-02-13T14:07:10.398 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-02-13T14:07:10.398 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-02-13T14:07:10.398 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-02-13T14:07:10.398 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-02-13T14:07:10.399 INFO:teuthology.orchestra.run.smithi148.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-02-13T14:07:10.399 INFO:teuthology.orchestra.run.smithi148.stdout: python3-google-auth noarch 1:2.27.0-1.el9 epel 203 k 2024-02-13T14:07:10.399 INFO:teuthology.orchestra.run.smithi148.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-02-13T14:07:10.399 INFO:teuthology.orchestra.run.smithi148.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-02-13T14:07:10.399 INFO:teuthology.orchestra.run.smithi148.stdout: python3-iniconfig noarch 1.1.1-7.el9 CentOS-AppStream 16 k 2024-02-13T14:07:10.399 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-02-13T14:07:10.399 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-02-13T14:07:10.399 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-02-13T14:07:10.399 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-02-13T14:07:10.399 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-02-13T14:07:10.399 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jinja2 noarch 2.11.3-5.el9 CentOS-AppStream 227 k 2024-02-13T14:07:10.399 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-02-13T14:07:10.399 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-02-13T14:07:10.400 INFO:teuthology.orchestra.run.smithi148.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-02-13T14:07:10.400 INFO:teuthology.orchestra.run.smithi148.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-02-13T14:07:10.400 INFO:teuthology.orchestra.run.smithi148.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-02-13T14:07:10.400 INFO:teuthology.orchestra.run.smithi148.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-02-13T14:07:10.400 INFO:teuthology.orchestra.run.smithi148.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-02-13T14:07:10.400 INFO:teuthology.orchestra.run.smithi148.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-02-13T14:07:10.400 INFO:teuthology.orchestra.run.smithi148.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-02-13T14:07:10.400 INFO:teuthology.orchestra.run.smithi148.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-02-13T14:07:10.400 INFO:teuthology.orchestra.run.smithi148.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-02-13T14:07:10.400 INFO:teuthology.orchestra.run.smithi148.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-02-13T14:07:10.400 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-02-13T14:07:10.400 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pluggy noarch 0.13.1-7.el9 CentOS-AppStream 38 k 2024-02-13T14:07:10.400 INFO:teuthology.orchestra.run.smithi148.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-02-13T14:07:10.400 INFO:teuthology.orchestra.run.smithi148.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-02-13T14:07:10.401 INFO:teuthology.orchestra.run.smithi148.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-02-13T14:07:10.401 INFO:teuthology.orchestra.run.smithi148.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-02-13T14:07:10.401 INFO:teuthology.orchestra.run.smithi148.stdout: python3-py noarch 1.10.0-6.el9 CentOS-AppStream 427 k 2024-02-13T14:07:10.401 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-02-13T14:07:10.401 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-02-13T14:07:10.401 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-02-13T14:07:10.401 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-02-13T14:07:10.401 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-02-13T14:07:10.401 INFO:teuthology.orchestra.run.smithi148.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-02-13T14:07:10.401 INFO:teuthology.orchestra.run.smithi148.stdout: python3-requests noarch 2.25.1-8.el9 CentOS-BaseOS 113 k 2024-02-13T14:07:10.401 INFO:teuthology.orchestra.run.smithi148.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-02-13T14:07:10.401 INFO:teuthology.orchestra.run.smithi148.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-02-13T14:07:10.401 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-02-13T14:07:10.401 INFO:teuthology.orchestra.run.smithi148.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-02-13T14:07:10.401 INFO:teuthology.orchestra.run.smithi148.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-02-13T14:07:10.402 INFO:teuthology.orchestra.run.smithi148.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-02-13T14:07:10.402 INFO:teuthology.orchestra.run.smithi148.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-02-13T14:07:10.402 INFO:teuthology.orchestra.run.smithi148.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-02-13T14:07:10.402 INFO:teuthology.orchestra.run.smithi148.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-02-13T14:07:10.402 INFO:teuthology.orchestra.run.smithi148.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-02-13T14:07:10.402 INFO:teuthology.orchestra.run.smithi148.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-02-13T14:07:10.402 INFO:teuthology.orchestra.run.smithi148.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-02-13T14:07:10.402 INFO:teuthology.orchestra.run.smithi148.stdout: qatlib x86_64 23.11.0-1.el9 CentOS-AppStream 215 k 2024-02-13T14:07:10.402 INFO:teuthology.orchestra.run.smithi148.stdout: qatzip-libs x86_64 1.1.2-2.el9 CentOS-AppStream 43 k 2024-02-13T14:07:10.402 INFO:teuthology.orchestra.run.smithi148.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-02-13T14:07:10.402 INFO:teuthology.orchestra.run.smithi148.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-02-13T14:07:10.402 INFO:teuthology.orchestra.run.smithi148.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-02-13T14:07:10.402 INFO:teuthology.orchestra.run.smithi148.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-02-13T14:07:10.402 INFO:teuthology.orchestra.run.smithi148.stdout:Installing weak dependencies: 2024-02-13T14:07:10.403 INFO:teuthology.orchestra.run.smithi148.stdout: qatlib-service x86_64 23.11.0-1.el9 CentOS-AppStream 34 k 2024-02-13T14:07:10.403 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-02-13T14:07:10.403 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction Summary 2024-02-13T14:07:10.403 INFO:teuthology.orchestra.run.smithi148.stdout:============================================================================================ 2024-02-13T14:07:10.403 INFO:teuthology.orchestra.run.smithi148.stdout:Install 131 Packages 2024-02-13T14:07:10.403 INFO:teuthology.orchestra.run.smithi148.stdout:Upgrade 2 Packages 2024-02-13T14:07:10.403 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-02-13T14:07:10.403 INFO:teuthology.orchestra.run.smithi148.stdout:Total download size: 195 M 2024-02-13T14:07:10.404 INFO:teuthology.orchestra.run.smithi148.stdout:Downloading Packages: 2024-02-13T14:07:10.777 INFO:teuthology.orchestra.run.smithi087.stdout:Extra Packages for Enterprise Linux 3.2 MB/s | 20 MB 00:06 2024-02-13T14:07:11.268 INFO:teuthology.orchestra.run.smithi148.stdout:(1/133): ceph-19.0.0-1423.ge1b79d5a.el9.x86_64. 36 kB/s | 6.6 kB 00:00 2024-02-13T14:07:11.669 INFO:teuthology.orchestra.run.smithi148.stdout:(2/133): ceph-fuse-19.0.0-1423.ge1b79d5a.el9.x8 2.2 MB/s | 885 kB 00:00 2024-02-13T14:07:11.727 INFO:teuthology.orchestra.run.smithi148.stdout:(3/133): ceph-immutable-object-cache-19.0.0-142 2.4 MB/s | 145 kB 00:00 2024-02-13T14:07:12.161 INFO:teuthology.orchestra.run.smithi148.stdout:(4/133): ceph-mds-19.0.0-1423.ge1b79d5a.el9.x86 4.9 MB/s | 2.1 MB 00:00 2024-02-13T14:07:12.353 INFO:teuthology.orchestra.run.smithi148.stdout:(5/133): ceph-mgr-19.0.0-1423.ge1b79d5a.el9.x86 7.8 MB/s | 1.5 MB 00:00 2024-02-13T14:07:12.878 INFO:teuthology.orchestra.run.smithi148.stdout:(6/133): ceph-mon-19.0.0-1423.ge1b79d5a.el9.x86 8.9 MB/s | 4.7 MB 00:00 2024-02-13T14:07:13.220 INFO:teuthology.orchestra.run.smithi148.stdout:(7/133): ceph-common-19.0.0-1423.ge1b79d5a.el9. 9.3 MB/s | 20 MB 00:02 2024-02-13T14:07:15.683 INFO:teuthology.orchestra.run.smithi087.stdout:lab-extras 11 kB/s | 1.7 kB 00:00 2024-02-13T14:07:18.321 INFO:teuthology.orchestra.run.smithi087.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-02-13T14:07:18.322 INFO:teuthology.orchestra.run.smithi087.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-02-13T14:07:18.409 INFO:teuthology.orchestra.run.smithi087.stdout:Dependencies resolved. 2024-02-13T14:07:18.417 INFO:teuthology.orchestra.run.smithi087.stdout:============================================================================================ 2024-02-13T14:07:18.417 INFO:teuthology.orchestra.run.smithi087.stdout: Package Arch Version Repository Size 2024-02-13T14:07:18.418 INFO:teuthology.orchestra.run.smithi087.stdout:============================================================================================ 2024-02-13T14:07:18.418 INFO:teuthology.orchestra.run.smithi087.stdout:Installing: 2024-02-13T14:07:18.418 INFO:teuthology.orchestra.run.smithi087.stdout: ceph x86_64 2:19.0.0-1423.ge1b79d5a.el9 ceph 6.6 k 2024-02-13T14:07:18.418 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-base x86_64 2:19.0.0-1423.ge1b79d5a.el9 ceph 5.3 M 2024-02-13T14:07:18.418 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-fuse x86_64 2:19.0.0-1423.ge1b79d5a.el9 ceph 885 k 2024-02-13T14:07:18.418 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1423.ge1b79d5a.el9 ceph 145 k 2024-02-13T14:07:18.418 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mgr x86_64 2:19.0.0-1423.ge1b79d5a.el9 ceph 1.5 M 2024-02-13T14:07:18.418 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1423.ge1b79d5a.el9 ceph-noarch 139 k 2024-02-13T14:07:18.418 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1423.ge1b79d5a.el9 ceph-noarch 1.8 M 2024-02-13T14:07:18.418 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1423.ge1b79d5a.el9 ceph-noarch 7.4 M 2024-02-13T14:07:18.418 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mgr-rook noarch 2:19.0.0-1423.ge1b79d5a.el9 ceph-noarch 52 k 2024-02-13T14:07:18.418 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-radosgw x86_64 2:19.0.0-1423.ge1b79d5a.el9 ceph 9.7 M 2024-02-13T14:07:18.418 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-test x86_64 2:19.0.0-1423.ge1b79d5a.el9 ceph 47 M 2024-02-13T14:07:18.418 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-volume noarch 2:19.0.0-1423.ge1b79d5a.el9 ceph-noarch 264 k 2024-02-13T14:07:18.419 INFO:teuthology.orchestra.run.smithi087.stdout: cephadm noarch 2:19.0.0-1423.ge1b79d5a.el9 ceph-noarch 760 k 2024-02-13T14:07:18.419 INFO:teuthology.orchestra.run.smithi087.stdout: libcephfs-devel x86_64 2:19.0.0-1423.ge1b79d5a.el9 ceph 31 k 2024-02-13T14:07:18.419 INFO:teuthology.orchestra.run.smithi087.stdout: libcephfs2 x86_64 2:19.0.0-1423.ge1b79d5a.el9 ceph 743 k 2024-02-13T14:07:18.419 INFO:teuthology.orchestra.run.smithi087.stdout: librados-devel x86_64 2:19.0.0-1423.ge1b79d5a.el9 ceph 127 k 2024-02-13T14:07:18.419 INFO:teuthology.orchestra.run.smithi087.stdout: python3-cephfs x86_64 2:19.0.0-1423.ge1b79d5a.el9 ceph 162 k 2024-02-13T14:07:18.419 INFO:teuthology.orchestra.run.smithi087.stdout: python3-pytest noarch 6.2.2-6.el9 CentOS-AppStream 474 k 2024-02-13T14:07:18.419 INFO:teuthology.orchestra.run.smithi087.stdout: python3-rados x86_64 2:19.0.0-1423.ge1b79d5a.el9 ceph 321 k 2024-02-13T14:07:18.419 INFO:teuthology.orchestra.run.smithi087.stdout: python3-rbd x86_64 2:19.0.0-1423.ge1b79d5a.el9 ceph 298 k 2024-02-13T14:07:18.419 INFO:teuthology.orchestra.run.smithi087.stdout: python3-rgw x86_64 2:19.0.0-1423.ge1b79d5a.el9 ceph 99 k 2024-02-13T14:07:18.419 INFO:teuthology.orchestra.run.smithi087.stdout: rbd-fuse x86_64 2:19.0.0-1423.ge1b79d5a.el9 ceph 85 k 2024-02-13T14:07:18.419 INFO:teuthology.orchestra.run.smithi087.stdout: rbd-mirror x86_64 2:19.0.0-1423.ge1b79d5a.el9 ceph 3.1 M 2024-02-13T14:07:18.419 INFO:teuthology.orchestra.run.smithi087.stdout: rbd-nbd x86_64 2:19.0.0-1423.ge1b79d5a.el9 ceph 171 k 2024-02-13T14:07:18.419 INFO:teuthology.orchestra.run.smithi087.stdout:Upgrading: 2024-02-13T14:07:18.419 INFO:teuthology.orchestra.run.smithi087.stdout: librados2 x86_64 2:19.0.0-1423.ge1b79d5a.el9 ceph 3.4 M 2024-02-13T14:07:18.419 INFO:teuthology.orchestra.run.smithi087.stdout: librbd1 x86_64 2:19.0.0-1423.ge1b79d5a.el9 ceph 3.1 M 2024-02-13T14:07:18.420 INFO:teuthology.orchestra.run.smithi087.stdout:Installing dependencies: 2024-02-13T14:07:18.420 INFO:teuthology.orchestra.run.smithi087.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-02-13T14:07:18.420 INFO:teuthology.orchestra.run.smithi087.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-02-13T14:07:18.420 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-common x86_64 2:19.0.0-1423.ge1b79d5a.el9 ceph 20 M 2024-02-13T14:07:18.420 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1423.ge1b79d5a.el9 ceph-noarch 31 k 2024-02-13T14:07:18.420 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mds x86_64 2:19.0.0-1423.ge1b79d5a.el9 ceph 2.1 M 2024-02-13T14:07:18.420 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1423.ge1b79d5a.el9 ceph-noarch 248 k 2024-02-13T14:07:18.420 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mon x86_64 2:19.0.0-1423.ge1b79d5a.el9 ceph 4.7 M 2024-02-13T14:07:18.420 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-osd x86_64 2:19.0.0-1423.ge1b79d5a.el9 ceph 17 M 2024-02-13T14:07:18.420 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1423.ge1b79d5a.el9 ceph-noarch 15 k 2024-02-13T14:07:18.420 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-selinux x86_64 2:19.0.0-1423.ge1b79d5a.el9 ceph 25 k 2024-02-13T14:07:18.420 INFO:teuthology.orchestra.run.smithi087.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-02-13T14:07:18.420 INFO:teuthology.orchestra.run.smithi087.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-02-13T14:07:18.420 INFO:teuthology.orchestra.run.smithi087.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-02-13T14:07:18.421 INFO:teuthology.orchestra.run.smithi087.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-02-13T14:07:18.421 INFO:teuthology.orchestra.run.smithi087.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-02-13T14:07:18.421 INFO:teuthology.orchestra.run.smithi087.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-02-13T14:07:18.421 INFO:teuthology.orchestra.run.smithi087.stdout: jq x86_64 1.6-16.el9 CentOS-BaseOS 186 k 2024-02-13T14:07:18.421 INFO:teuthology.orchestra.run.smithi087.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-02-13T14:07:18.421 INFO:teuthology.orchestra.run.smithi087.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-02-13T14:07:18.421 INFO:teuthology.orchestra.run.smithi087.stdout: libcephsqlite x86_64 2:19.0.0-1423.ge1b79d5a.el9 ceph 163 k 2024-02-13T14:07:18.421 INFO:teuthology.orchestra.run.smithi087.stdout: libgfortran x86_64 11.4.1-3.el9 CentOS-BaseOS 802 k 2024-02-13T14:07:18.421 INFO:teuthology.orchestra.run.smithi087.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-02-13T14:07:18.421 INFO:teuthology.orchestra.run.smithi087.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-02-13T14:07:18.421 INFO:teuthology.orchestra.run.smithi087.stdout: libquadmath x86_64 11.4.1-3.el9 CentOS-BaseOS 194 k 2024-02-13T14:07:18.421 INFO:teuthology.orchestra.run.smithi087.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-02-13T14:07:18.421 INFO:teuthology.orchestra.run.smithi087.stdout: libradosstriper1 x86_64 2:19.0.0-1423.ge1b79d5a.el9 ceph 499 k 2024-02-13T14:07:18.421 INFO:teuthology.orchestra.run.smithi087.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-02-13T14:07:18.422 INFO:teuthology.orchestra.run.smithi087.stdout: librgw2 x86_64 2:19.0.0-1423.ge1b79d5a.el9 ceph 5.1 M 2024-02-13T14:07:18.422 INFO:teuthology.orchestra.run.smithi087.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-02-13T14:07:18.422 INFO:teuthology.orchestra.run.smithi087.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-02-13T14:07:18.422 INFO:teuthology.orchestra.run.smithi087.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-02-13T14:07:18.422 INFO:teuthology.orchestra.run.smithi087.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-02-13T14:07:18.422 INFO:teuthology.orchestra.run.smithi087.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-02-13T14:07:18.422 INFO:teuthology.orchestra.run.smithi087.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-BaseOS 217 k 2024-02-13T14:07:18.422 INFO:teuthology.orchestra.run.smithi087.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-02-13T14:07:18.422 INFO:teuthology.orchestra.run.smithi087.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-02-13T14:07:18.422 INFO:teuthology.orchestra.run.smithi087.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-02-13T14:07:18.422 INFO:teuthology.orchestra.run.smithi087.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-02-13T14:07:18.422 INFO:teuthology.orchestra.run.smithi087.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-02-13T14:07:18.422 INFO:teuthology.orchestra.run.smithi087.stdout: python3-asyncssh noarch 2.13.2-3.el9 epel 548 k 2024-02-13T14:07:18.422 INFO:teuthology.orchestra.run.smithi087.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-02-13T14:07:18.422 INFO:teuthology.orchestra.run.smithi087.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-02-13T14:07:18.423 INFO:teuthology.orchestra.run.smithi087.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-02-13T14:07:18.423 INFO:teuthology.orchestra.run.smithi087.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-02-13T14:07:18.423 INFO:teuthology.orchestra.run.smithi087.stdout: python3-ceph-argparse x86_64 2:19.0.0-1423.ge1b79d5a.el9 ceph 45 k 2024-02-13T14:07:18.423 INFO:teuthology.orchestra.run.smithi087.stdout: python3-ceph-common x86_64 2:19.0.0-1423.ge1b79d5a.el9 ceph 132 k 2024-02-13T14:07:18.423 INFO:teuthology.orchestra.run.smithi087.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-02-13T14:07:18.423 INFO:teuthology.orchestra.run.smithi087.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-02-13T14:07:18.423 INFO:teuthology.orchestra.run.smithi087.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-02-13T14:07:18.423 INFO:teuthology.orchestra.run.smithi087.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-02-13T14:07:18.423 INFO:teuthology.orchestra.run.smithi087.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-02-13T14:07:18.423 INFO:teuthology.orchestra.run.smithi087.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-02-13T14:07:18.423 INFO:teuthology.orchestra.run.smithi087.stdout: python3-google-auth noarch 1:2.27.0-1.el9 epel 203 k 2024-02-13T14:07:18.423 INFO:teuthology.orchestra.run.smithi087.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-02-13T14:07:18.423 INFO:teuthology.orchestra.run.smithi087.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-02-13T14:07:18.423 INFO:teuthology.orchestra.run.smithi087.stdout: python3-iniconfig noarch 1.1.1-7.el9 CentOS-AppStream 16 k 2024-02-13T14:07:18.424 INFO:teuthology.orchestra.run.smithi087.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-02-13T14:07:18.424 INFO:teuthology.orchestra.run.smithi087.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-02-13T14:07:18.424 INFO:teuthology.orchestra.run.smithi087.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-02-13T14:07:18.424 INFO:teuthology.orchestra.run.smithi087.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-02-13T14:07:18.424 INFO:teuthology.orchestra.run.smithi087.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-02-13T14:07:18.424 INFO:teuthology.orchestra.run.smithi087.stdout: python3-jinja2 noarch 2.11.3-5.el9 CentOS-AppStream 227 k 2024-02-13T14:07:18.424 INFO:teuthology.orchestra.run.smithi087.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-02-13T14:07:18.424 INFO:teuthology.orchestra.run.smithi087.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-02-13T14:07:18.424 INFO:teuthology.orchestra.run.smithi087.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-02-13T14:07:18.424 INFO:teuthology.orchestra.run.smithi087.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-02-13T14:07:18.424 INFO:teuthology.orchestra.run.smithi087.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-02-13T14:07:18.424 INFO:teuthology.orchestra.run.smithi087.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-02-13T14:07:18.424 INFO:teuthology.orchestra.run.smithi087.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-02-13T14:07:18.424 INFO:teuthology.orchestra.run.smithi087.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-02-13T14:07:18.425 INFO:teuthology.orchestra.run.smithi087.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-02-13T14:07:18.425 INFO:teuthology.orchestra.run.smithi087.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-02-13T14:07:18.425 INFO:teuthology.orchestra.run.smithi087.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-02-13T14:07:18.425 INFO:teuthology.orchestra.run.smithi087.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-02-13T14:07:18.425 INFO:teuthology.orchestra.run.smithi087.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-02-13T14:07:18.425 INFO:teuthology.orchestra.run.smithi087.stdout: python3-pluggy noarch 0.13.1-7.el9 CentOS-AppStream 38 k 2024-02-13T14:07:18.425 INFO:teuthology.orchestra.run.smithi087.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-02-13T14:07:18.425 INFO:teuthology.orchestra.run.smithi087.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-02-13T14:07:18.425 INFO:teuthology.orchestra.run.smithi087.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-02-13T14:07:18.425 INFO:teuthology.orchestra.run.smithi087.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-02-13T14:07:18.425 INFO:teuthology.orchestra.run.smithi087.stdout: python3-py noarch 1.10.0-6.el9 CentOS-AppStream 427 k 2024-02-13T14:07:18.425 INFO:teuthology.orchestra.run.smithi087.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-02-13T14:07:18.425 INFO:teuthology.orchestra.run.smithi087.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-02-13T14:07:18.425 INFO:teuthology.orchestra.run.smithi087.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-02-13T14:07:18.426 INFO:teuthology.orchestra.run.smithi087.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-02-13T14:07:18.426 INFO:teuthology.orchestra.run.smithi087.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-02-13T14:07:18.426 INFO:teuthology.orchestra.run.smithi087.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-02-13T14:07:18.426 INFO:teuthology.orchestra.run.smithi087.stdout: python3-requests noarch 2.25.1-8.el9 CentOS-BaseOS 113 k 2024-02-13T14:07:18.426 INFO:teuthology.orchestra.run.smithi087.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-02-13T14:07:18.426 INFO:teuthology.orchestra.run.smithi087.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-02-13T14:07:18.426 INFO:teuthology.orchestra.run.smithi087.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-02-13T14:07:18.426 INFO:teuthology.orchestra.run.smithi087.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-02-13T14:07:18.426 INFO:teuthology.orchestra.run.smithi087.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-02-13T14:07:18.426 INFO:teuthology.orchestra.run.smithi087.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-02-13T14:07:18.426 INFO:teuthology.orchestra.run.smithi087.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-02-13T14:07:18.426 INFO:teuthology.orchestra.run.smithi087.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-02-13T14:07:18.426 INFO:teuthology.orchestra.run.smithi087.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-02-13T14:07:18.426 INFO:teuthology.orchestra.run.smithi087.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-02-13T14:07:18.426 INFO:teuthology.orchestra.run.smithi087.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-02-13T14:07:18.427 INFO:teuthology.orchestra.run.smithi087.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-02-13T14:07:18.427 INFO:teuthology.orchestra.run.smithi087.stdout: qatlib x86_64 23.11.0-1.el9 CentOS-AppStream 215 k 2024-02-13T14:07:18.427 INFO:teuthology.orchestra.run.smithi087.stdout: qatzip-libs x86_64 1.1.2-2.el9 CentOS-AppStream 43 k 2024-02-13T14:07:18.427 INFO:teuthology.orchestra.run.smithi087.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-02-13T14:07:18.427 INFO:teuthology.orchestra.run.smithi087.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-02-13T14:07:18.427 INFO:teuthology.orchestra.run.smithi087.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-02-13T14:07:18.427 INFO:teuthology.orchestra.run.smithi087.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-02-13T14:07:18.427 INFO:teuthology.orchestra.run.smithi087.stdout:Installing weak dependencies: 2024-02-13T14:07:18.427 INFO:teuthology.orchestra.run.smithi087.stdout: qatlib-service x86_64 23.11.0-1.el9 CentOS-AppStream 34 k 2024-02-13T14:07:18.427 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-02-13T14:07:18.427 INFO:teuthology.orchestra.run.smithi087.stdout:Transaction Summary 2024-02-13T14:07:18.427 INFO:teuthology.orchestra.run.smithi087.stdout:============================================================================================ 2024-02-13T14:07:18.427 INFO:teuthology.orchestra.run.smithi087.stdout:Install 131 Packages 2024-02-13T14:07:18.427 INFO:teuthology.orchestra.run.smithi087.stdout:Upgrade 2 Packages 2024-02-13T14:07:18.428 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-02-13T14:07:18.428 INFO:teuthology.orchestra.run.smithi087.stdout:Total download size: 195 M 2024-02-13T14:07:18.428 INFO:teuthology.orchestra.run.smithi087.stdout:Downloading Packages: 2024-02-13T14:07:19.189 INFO:teuthology.orchestra.run.smithi087.stdout:(1/133): ceph-19.0.0-1423.ge1b79d5a.el9.x86_64. 24 kB/s | 6.6 kB 00:00 2024-02-13T14:07:19.551 INFO:teuthology.orchestra.run.smithi087.stdout:(2/133): ceph-fuse-19.0.0-1423.ge1b79d5a.el9.x8 2.4 MB/s | 885 kB 00:00 2024-02-13T14:07:19.618 INFO:teuthology.orchestra.run.smithi087.stdout:(3/133): ceph-immutable-object-cache-19.0.0-142 2.1 MB/s | 145 kB 00:00 2024-02-13T14:07:20.118 INFO:teuthology.orchestra.run.smithi087.stdout:(4/133): ceph-mds-19.0.0-1423.ge1b79d5a.el9.x86 4.2 MB/s | 2.1 MB 00:00 2024-02-13T14:07:20.443 INFO:teuthology.orchestra.run.smithi087.stdout:(5/133): ceph-mgr-19.0.0-1423.ge1b79d5a.el9.x86 4.6 MB/s | 1.5 MB 00:00 2024-02-13T14:07:20.670 INFO:teuthology.orchestra.run.smithi148.stdout:(8/133): ceph-radosgw-19.0.0-1423.ge1b79d5a.el9 1.3 MB/s | 9.7 MB 00:07 2024-02-13T14:07:20.703 INFO:teuthology.orchestra.run.smithi148.stdout:(9/133): ceph-selinux-19.0.0-1423.ge1b79d5a.el9 761 kB/s | 25 kB 00:00 2024-02-13T14:07:20.927 INFO:teuthology.orchestra.run.smithi087.stdout:(6/133): ceph-mon-19.0.0-1423.ge1b79d5a.el9.x86 9.7 MB/s | 4.7 MB 00:00 2024-02-13T14:07:20.970 INFO:teuthology.orchestra.run.smithi148.stdout:(10/133): ceph-osd-19.0.0-1423.ge1b79d5a.el9.x8 2.1 MB/s | 17 MB 00:08 2024-02-13T14:07:21.037 INFO:teuthology.orchestra.run.smithi148.stdout:(11/133): libcephfs-devel-19.0.0-1423.ge1b79d5a 471 kB/s | 31 kB 00:00 2024-02-13T14:07:21.145 INFO:teuthology.orchestra.run.smithi148.stdout:(12/133): libcephfs2-19.0.0-1423.ge1b79d5a.el9. 6.7 MB/s | 743 kB 00:00 2024-02-13T14:07:21.271 INFO:teuthology.orchestra.run.smithi148.stdout:(13/133): libcephsqlite-19.0.0-1423.ge1b79d5a.e 1.3 MB/s | 163 kB 00:00 2024-02-13T14:07:21.363 INFO:teuthology.orchestra.run.smithi148.stdout:(14/133): librados-devel-19.0.0-1423.ge1b79d5a. 1.4 MB/s | 127 kB 00:00 2024-02-13T14:07:21.454 INFO:teuthology.orchestra.run.smithi148.stdout:(15/133): libradosstriper1-19.0.0-1423.ge1b79d5 5.3 MB/s | 499 kB 00:00 2024-02-13T14:07:21.938 INFO:teuthology.orchestra.run.smithi148.stdout:(16/133): librgw2-19.0.0-1423.ge1b79d5a.el9.x86 11 MB/s | 5.1 MB 00:00 2024-02-13T14:07:22.005 INFO:teuthology.orchestra.run.smithi148.stdout:(17/133): python3-ceph-argparse-19.0.0-1423.ge1 678 kB/s | 45 kB 00:00 2024-02-13T14:07:22.072 INFO:teuthology.orchestra.run.smithi148.stdout:(18/133): python3-ceph-common-19.0.0-1423.ge1b7 1.9 MB/s | 132 kB 00:00 2024-02-13T14:07:22.156 INFO:teuthology.orchestra.run.smithi148.stdout:(19/133): python3-cephfs-19.0.0-1423.ge1b79d5a. 1.9 MB/s | 162 kB 00:00 2024-02-13T14:07:22.231 INFO:teuthology.orchestra.run.smithi148.stdout:(20/133): python3-rados-19.0.0-1423.ge1b79d5a.e 4.2 MB/s | 321 kB 00:00 2024-02-13T14:07:22.306 INFO:teuthology.orchestra.run.smithi148.stdout:(21/133): python3-rbd-19.0.0-1423.ge1b79d5a.el9 3.9 MB/s | 298 kB 00:00 2024-02-13T14:07:22.373 INFO:teuthology.orchestra.run.smithi148.stdout:(22/133): python3-rgw-19.0.0-1423.ge1b79d5a.el9 1.5 MB/s | 99 kB 00:00 2024-02-13T14:07:22.482 INFO:teuthology.orchestra.run.smithi148.stdout:(23/133): rbd-fuse-19.0.0-1423.ge1b79d5a.el9.x8 785 kB/s | 85 kB 00:00 2024-02-13T14:07:22.799 INFO:teuthology.orchestra.run.smithi148.stdout:(24/133): rbd-mirror-19.0.0-1423.ge1b79d5a.el9. 9.8 MB/s | 3.1 MB 00:00 2024-02-13T14:07:22.827 INFO:teuthology.orchestra.run.smithi087.stdout:(7/133): ceph-osd-19.0.0-1423.ge1b79d5a.el9.x86 8.8 MB/s | 17 MB 00:01 2024-02-13T14:07:22.866 INFO:teuthology.orchestra.run.smithi148.stdout:(25/133): rbd-nbd-19.0.0-1423.ge1b79d5a.el9.x86 2.5 MB/s | 171 kB 00:00 2024-02-13T14:07:22.933 INFO:teuthology.orchestra.run.smithi148.stdout:(26/133): ceph-grafana-dashboards-19.0.0-1423.g 473 kB/s | 31 kB 00:00 2024-02-13T14:07:23.058 INFO:teuthology.orchestra.run.smithi148.stdout:(27/133): ceph-mgr-cephadm-19.0.0-1423.ge1b79d5 1.1 MB/s | 139 kB 00:00 2024-02-13T14:07:23.300 INFO:teuthology.orchestra.run.smithi148.stdout:(28/133): ceph-mgr-dashboard-19.0.0-1423.ge1b79 7.3 MB/s | 1.8 MB 00:00 2024-02-13T14:07:24.042 INFO:teuthology.orchestra.run.smithi148.stdout:(29/133): ceph-mgr-diskprediction-local-19.0.0- 10 MB/s | 7.4 MB 00:00 2024-02-13T14:07:24.151 INFO:teuthology.orchestra.run.smithi148.stdout:(30/133): ceph-mgr-modules-core-19.0.0-1423.ge1 2.2 MB/s | 248 kB 00:00 2024-02-13T14:07:24.209 INFO:teuthology.orchestra.run.smithi148.stdout:(31/133): ceph-mgr-rook-19.0.0-1423.ge1b79d5a.e 899 kB/s | 52 kB 00:00 2024-02-13T14:07:24.293 INFO:teuthology.orchestra.run.smithi148.stdout:(32/133): ceph-prometheus-alerts-19.0.0-1423.ge 184 kB/s | 15 kB 00:00 2024-02-13T14:07:24.294 INFO:teuthology.orchestra.run.smithi087.stdout:(8/133): ceph-radosgw-19.0.0-1423.ge1b79d5a.el9 6.6 MB/s | 9.7 MB 00:01 2024-02-13T14:07:24.336 INFO:teuthology.orchestra.run.smithi087.stdout:(9/133): ceph-selinux-19.0.0-1423.ge1b79d5a.el9 609 kB/s | 25 kB 00:00 2024-02-13T14:07:24.735 INFO:teuthology.orchestra.run.smithi148.stdout:(33/133): ceph-test-19.0.0-1423.ge1b79d5a.el9.x 12 MB/s | 47 MB 00:04 2024-02-13T14:07:24.760 INFO:teuthology.orchestra.run.smithi148.stdout:(34/133): ceph-volume-19.0.0-1423.ge1b79d5a.el9 565 kB/s | 264 kB 00:00 2024-02-13T14:07:24.860 INFO:teuthology.orchestra.run.smithi148.stdout:(35/133): cephadm-19.0.0-1423.ge1b79d5a.el9.noa 5.9 MB/s | 760 kB 00:00 2024-02-13T14:07:24.885 INFO:teuthology.orchestra.run.smithi148.stdout:(36/133): boost-program-options-1.75.0-8.el9.x8 845 kB/s | 106 kB 00:00 2024-02-13T14:07:24.902 INFO:teuthology.orchestra.run.smithi148.stdout:(37/133): flexiblas-3.0.4-8.el9.x86_64.rpm 758 kB/s | 32 kB 00:00 2024-02-13T14:07:24.961 INFO:teuthology.orchestra.run.smithi148.stdout:(38/133): flexiblas-openblas-openmp-3.0.4-8.el9 295 kB/s | 17 kB 00:00 2024-02-13T14:07:25.037 INFO:teuthology.orchestra.run.smithi148.stdout:(39/133): flexiblas-netlib-3.0.4-8.el9.x86_64.r 20 MB/s | 3.0 MB 00:00 2024-02-13T14:07:25.062 INFO:teuthology.orchestra.run.smithi148.stdout:(40/133): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.6 MB/s | 159 kB 00:00 2024-02-13T14:07:25.087 INFO:teuthology.orchestra.run.smithi148.stdout:(41/133): librabbitmq-0.11.0-7.el9.x86_64.rpm 890 kB/s | 44 kB 00:00 2024-02-13T14:07:25.112 INFO:teuthology.orchestra.run.smithi148.stdout:(42/133): librdkafka-1.6.1-102.el9.x86_64.rpm 13 MB/s | 661 kB 00:00 2024-02-13T14:07:25.138 INFO:teuthology.orchestra.run.smithi148.stdout:(43/133): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.6 MB/s | 282 kB 00:00 2024-02-13T14:07:25.163 INFO:teuthology.orchestra.run.smithi148.stdout:(44/133): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 187 kB 00:00 2024-02-13T14:07:25.188 INFO:teuthology.orchestra.run.smithi148.stdout:(45/133): openblas-0.3.21-2.el9.x86_64.rpm 689 kB/s | 34 kB 00:00 2024-02-13T14:07:25.322 INFO:teuthology.orchestra.run.smithi148.stdout:(46/133): openblas-openmp-0.3.21-2.el9.x86_64.r 30 MB/s | 4.7 MB 00:00 2024-02-13T14:07:25.355 INFO:teuthology.orchestra.run.smithi148.stdout:(47/133): protobuf-3.14.0-13.el9.x86_64.rpm 6.1 MB/s | 1.0 MB 00:00 2024-02-13T14:07:25.406 INFO:teuthology.orchestra.run.smithi148.stdout:(48/133): python3-attrs-20.3.0-7.el9.noarch.rpm 951 kB/s | 78 kB 00:00 2024-02-13T14:07:25.556 INFO:teuthology.orchestra.run.smithi148.stdout:(49/133): python3-babel-2.9.1-2.el9.noarch.rpm 29 MB/s | 5.8 MB 00:00 2024-02-13T14:07:25.573 INFO:teuthology.orchestra.run.smithi148.stdout:(50/133): python3-iniconfig-1.1.1-7.el9.noarch. 100 kB/s | 16 kB 00:00 2024-02-13T14:07:25.598 INFO:teuthology.orchestra.run.smithi148.stdout:(51/133): python3-jinja2-2.11.3-5.el9.noarch.rp 5.3 MB/s | 227 kB 00:00 2024-02-13T14:07:25.615 INFO:teuthology.orchestra.run.smithi148.stdout:(52/133): python3-jsonpatch-1.21-16.el9.noarch. 586 kB/s | 24 kB 00:00 2024-02-13T14:07:25.641 INFO:teuthology.orchestra.run.smithi148.stdout:(53/133): python3-jsonpointer-2.0-4.el9.noarch. 432 kB/s | 18 kB 00:00 2024-02-13T14:07:25.666 INFO:teuthology.orchestra.run.smithi148.stdout:(54/133): python3-mako-1.1.4-6.el9.noarch.rpm 2.8 MB/s | 144 kB 00:00 2024-02-13T14:07:25.691 INFO:teuthology.orchestra.run.smithi148.stdout:(55/133): python3-markupsafe-1.1.1-12.el9.x86_6 633 kB/s | 32 kB 00:00 2024-02-13T14:07:25.842 INFO:teuthology.orchestra.run.smithi148.stdout:(56/133): python3-numpy-1.20.1-5.el9.x86_64.rpm 27 MB/s | 4.8 MB 00:00 2024-02-13T14:07:25.866 INFO:teuthology.orchestra.run.smithi148.stdout:(57/133): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.2 MB/s | 217 kB 00:00 2024-02-13T14:07:25.892 INFO:teuthology.orchestra.run.smithi148.stdout:(58/133): python3-oauthlib-3.1.1-5.el9.noarch.r 3.4 MB/s | 174 kB 00:00 2024-02-13T14:07:25.917 INFO:teuthology.orchestra.run.smithi148.stdout:(59/133): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 69 kB 00:00 2024-02-13T14:07:25.942 INFO:teuthology.orchestra.run.smithi148.stdout:(60/133): python3-pluggy-0.13.1-7.el9.noarch.rp 757 kB/s | 38 kB 00:00 2024-02-13T14:07:25.968 INFO:teuthology.orchestra.run.smithi148.stdout:(61/133): python3-prettytable-0.7.2-27.el9.noar 822 kB/s | 41 kB 00:00 2024-02-13T14:07:25.993 INFO:teuthology.orchestra.run.smithi148.stdout:(62/133): python3-protobuf-3.14.0-13.el9.noarch 4.6 MB/s | 238 kB 00:00 2024-02-13T14:07:26.018 INFO:teuthology.orchestra.run.smithi148.stdout:(63/133): python3-py-1.10.0-6.el9.noarch.rpm 8.4 MB/s | 427 kB 00:00 2024-02-13T14:07:26.043 INFO:teuthology.orchestra.run.smithi148.stdout:(64/133): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.6 MB/s | 132 kB 00:00 2024-02-13T14:07:26.069 INFO:teuthology.orchestra.run.smithi148.stdout:(65/133): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 211 kB 00:00 2024-02-13T14:07:26.094 INFO:teuthology.orchestra.run.smithi148.stdout:(66/133): python3-pytest-6.2.2-6.el9.noarch.rpm 9.3 MB/s | 474 kB 00:00 2024-02-13T14:07:26.119 INFO:teuthology.orchestra.run.smithi148.stdout:(67/133): python3-pytz-2021.1-5.el9.noarch.rpm 958 kB/s | 47 kB 00:00 2024-02-13T14:07:26.144 INFO:teuthology.orchestra.run.smithi148.stdout:(68/133): python3-requests-oauthlib-1.3.0-12.el 854 kB/s | 43 kB 00:00 2024-02-13T14:07:26.303 INFO:teuthology.orchestra.run.smithi148.stdout:(69/133): python3-toml-0.10.2-6.el9.noarch.rpm 246 kB/s | 39 kB 00:00 2024-02-13T14:07:26.570 INFO:teuthology.orchestra.run.smithi148.stdout:(70/133): python3-scipy-1.6.2-8.el9.x86_64.rpm 35 MB/s | 16 MB 00:00 2024-02-13T14:07:26.595 INFO:teuthology.orchestra.run.smithi148.stdout:(71/133): qatlib-23.11.0-1.el9.x86_64.rpm 743 kB/s | 215 kB 00:00 2024-02-13T14:07:26.621 INFO:teuthology.orchestra.run.smithi148.stdout:(72/133): qatlib-service-23.11.0-1.el9.x86_64.r 681 kB/s | 34 kB 00:00 2024-02-13T14:07:26.648 INFO:teuthology.orchestra.run.smithi148.stdout:(73/133): qatzip-libs-1.1.2-2.el9.x86_64.rpm 868 kB/s | 43 kB 00:00 2024-02-13T14:07:26.671 INFO:teuthology.orchestra.run.smithi148.stdout:(74/133): socat-1.7.4.1-5.el9.x86_64.rpm 5.9 MB/s | 300 kB 00:00 2024-02-13T14:07:26.738 INFO:teuthology.orchestra.run.smithi148.stdout:(75/133): ceph-base-19.0.0-1423.ge1b79d5a.el9.x 350 kB/s | 5.3 MB 00:15 2024-02-13T14:07:26.763 INFO:teuthology.orchestra.run.smithi148.stdout:(76/133): xmlstarlet-1.6.1-20.el9.x86_64.rpm 536 kB/s | 63 kB 00:00 2024-02-13T14:07:26.789 INFO:teuthology.orchestra.run.smithi148.stdout:(77/133): fuse-2.9.9-15.el9.x86_64.rpm 672 kB/s | 79 kB 00:00 2024-02-13T14:07:26.814 INFO:teuthology.orchestra.run.smithi148.stdout:(78/133): jq-1.6-16.el9.x86_64.rpm 3.6 MB/s | 186 kB 00:00 2024-02-13T14:07:26.839 INFO:teuthology.orchestra.run.smithi148.stdout:(79/133): libgfortran-11.4.1-3.el9.x86_64.rpm 10 MB/s | 802 kB 00:00 2024-02-13T14:07:26.864 INFO:teuthology.orchestra.run.smithi148.stdout:(80/133): libquadmath-11.4.1-3.el9.x86_64.rpm 2.5 MB/s | 194 kB 00:00 2024-02-13T14:07:26.890 INFO:teuthology.orchestra.run.smithi148.stdout:(81/133): oniguruma-6.9.6-1.el9.5.x86_64.rpm 2.9 MB/s | 217 kB 00:00 2024-02-13T14:07:26.920 INFO:teuthology.orchestra.run.smithi087.stdout:(10/133): ceph-test-19.0.0-1423.ge1b79d5a.el9.x 18 MB/s | 47 MB 00:02 2024-02-13T14:07:26.923 INFO:teuthology.orchestra.run.smithi148.stdout:(82/133): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.0 MB/s | 241 kB 00:00 2024-02-13T14:07:26.957 INFO:teuthology.orchestra.run.smithi148.stdout:(83/133): python3-cryptography-36.0.1-4.el9.x86 12 MB/s | 1.1 MB 00:00 2024-02-13T14:07:26.978 INFO:teuthology.orchestra.run.smithi087.stdout:(11/133): libcephfs-devel-19.0.0-1423.ge1b79d5a 539 kB/s | 31 kB 00:00 2024-02-13T14:07:26.982 INFO:teuthology.orchestra.run.smithi148.stdout:(84/133): python3-ply-3.11-14.el9.noarch.rpm 1.1 MB/s | 103 kB 00:00 2024-02-13T14:07:27.007 INFO:teuthology.orchestra.run.smithi148.stdout:(85/133): python3-pycparser-2.20-6.el9.noarch.r 1.5 MB/s | 124 kB 00:00 2024-02-13T14:07:27.033 INFO:teuthology.orchestra.run.smithi148.stdout:(86/133): python3-requests-2.25.1-8.el9.noarch. 2.2 MB/s | 113 kB 00:00 2024-02-13T14:07:27.058 INFO:teuthology.orchestra.run.smithi148.stdout:(87/133): python3-urllib3-1.26.5-5.el9.noarch.r 2.4 MB/s | 187 kB 00:00 2024-02-13T14:07:27.103 INFO:teuthology.orchestra.run.smithi087.stdout:(12/133): libcephfs2-19.0.0-1423.ge1b79d5a.el9. 5.8 MB/s | 743 kB 00:00 2024-02-13T14:07:27.133 INFO:teuthology.orchestra.run.smithi148.stdout:(88/133): python3-devel-3.9.17-2.el9.x86_64.rpm 1.9 MB/s | 245 kB 00:00 2024-02-13T14:07:27.167 INFO:teuthology.orchestra.run.smithi148.stdout:(89/133): lua-devel-5.4.4-4.el9.x86_64.rpm 205 kB/s | 22 kB 00:00 2024-02-13T14:07:27.178 INFO:teuthology.orchestra.run.smithi087.stdout:(13/133): libcephsqlite-19.0.0-1423.ge1b79d5a.e 2.1 MB/s | 163 kB 00:00 2024-02-13T14:07:27.245 INFO:teuthology.orchestra.run.smithi087.stdout:(14/133): librados-devel-19.0.0-1423.ge1b79d5a. 1.9 MB/s | 127 kB 00:00 2024-02-13T14:07:27.296 INFO:teuthology.orchestra.run.smithi087.stdout:(15/133): libradosstriper1-19.0.0-1423.ge1b79d5 9.8 MB/s | 499 kB 00:00 2024-02-13T14:07:27.434 INFO:teuthology.orchestra.run.smithi148.stdout:(90/133): gperftools-libs-2.9.1-2.el9.x86_64.rp 1.3 MB/s | 309 kB 00:00 2024-02-13T14:07:27.467 INFO:teuthology.orchestra.run.smithi148.stdout:(91/133): abseil-cpp-20211102.0-3.el9.x86_64.rp 1.6 MB/s | 548 kB 00:00 2024-02-13T14:07:27.492 INFO:teuthology.orchestra.run.smithi148.stdout:(92/133): grpc-data-1.46.7-10.el9.noarch.rpm 334 kB/s | 19 kB 00:00 2024-02-13T14:07:27.526 INFO:teuthology.orchestra.run.smithi148.stdout:(93/133): protobuf-compiler-3.14.0-13.el9.x86_6 1.8 MB/s | 863 kB 00:00 2024-02-13T14:07:27.551 INFO:teuthology.orchestra.run.smithi148.stdout:(94/133): libarrow-doc-9.0.0-10.el9.noarch.rpm 443 kB/s | 26 kB 00:00 2024-02-13T14:07:27.596 INFO:teuthology.orchestra.run.smithi087.stdout:(16/133): librgw2-19.0.0-1423.ge1b79d5a.el9.x86 17 MB/s | 5.1 MB 00:00 2024-02-13T14:07:27.618 INFO:teuthology.orchestra.run.smithi148.stdout:(95/133): libunwind-1.6.2-1.el9.x86_64.rpm 1.6 MB/s | 67 kB 00:00 2024-02-13T14:07:27.638 INFO:teuthology.orchestra.run.smithi087.stdout:(17/133): python3-ceph-argparse-19.0.0-1423.ge1 1.1 MB/s | 45 kB 00:00 2024-02-13T14:07:27.685 INFO:teuthology.orchestra.run.smithi148.stdout:(96/133): luarocks-3.9.2-1.el9.noarch.rpm 2.2 MB/s | 151 kB 00:00 2024-02-13T14:07:27.696 INFO:teuthology.orchestra.run.smithi087.stdout:(18/133): python3-ceph-common-19.0.0-1423.ge1b7 2.2 MB/s | 132 kB 00:00 2024-02-13T14:07:27.718 INFO:teuthology.orchestra.run.smithi148.stdout:(97/133): liboath-2.6.7-2.el9.x86_64.rpm 253 kB/s | 49 kB 00:00 2024-02-13T14:07:27.755 INFO:teuthology.orchestra.run.smithi087.stdout:(19/133): python3-cephfs-19.0.0-1423.ge1b79d5a. 2.7 MB/s | 162 kB 00:00 2024-02-13T14:07:27.831 INFO:teuthology.orchestra.run.smithi087.stdout:(20/133): python3-rados-19.0.0-1423.ge1b79d5a.e 4.2 MB/s | 321 kB 00:00 2024-02-13T14:07:27.871 INFO:teuthology.orchestra.run.smithi148.stdout:(98/133): python3-asyncssh-2.13.2-3.el9.noarch. 3.6 MB/s | 548 kB 00:00 2024-02-13T14:07:27.894 INFO:teuthology.orchestra.run.smithi148.stdout:(99/133): parquet-libs-9.0.0-10.el9.x86_64.rpm 3.9 MB/s | 838 kB 00:00 2024-02-13T14:07:27.914 INFO:teuthology.orchestra.run.smithi087.stdout:(21/133): python3-rbd-19.0.0-1423.ge1b79d5a.el9 3.5 MB/s | 298 kB 00:00 2024-02-13T14:07:27.919 INFO:teuthology.orchestra.run.smithi148.stdout:(100/133): python3-bcrypt-3.2.2-1.el9.x86_64.rp 866 kB/s | 43 kB 00:00 2024-02-13T14:07:27.953 INFO:teuthology.orchestra.run.smithi148.stdout:(101/133): python3-cachetools-4.2.4-1.el9.noarc 549 kB/s | 32 kB 00:00 2024-02-13T14:07:27.978 INFO:teuthology.orchestra.run.smithi148.stdout:(102/133): python3-certifi-2021.10.8-2.el9.noar 257 kB/s | 15 kB 00:00 2024-02-13T14:07:27.989 INFO:teuthology.orchestra.run.smithi087.stdout:(22/133): python3-rgw-19.0.0-1423.ge1b79d5a.el9 1.3 MB/s | 99 kB 00:00 2024-02-13T14:07:28.031 INFO:teuthology.orchestra.run.smithi087.stdout:(23/133): rbd-fuse-19.0.0-1423.ge1b79d5a.el9.x8 2.0 MB/s | 85 kB 00:00 2024-02-13T14:07:28.037 INFO:teuthology.orchestra.run.smithi148.stdout:(103/133): python3-cheroot-8.6.0-4.el9.noarch.r 2.0 MB/s | 172 kB 00:00 2024-02-13T14:07:28.129 INFO:teuthology.orchestra.run.smithi148.stdout:(104/133): python3-google-auth-2.27.0-1.el9.noa 2.2 MB/s | 203 kB 00:00 2024-02-13T14:07:28.154 INFO:teuthology.orchestra.run.smithi148.stdout:(105/133): python3-cherrypy-18.6.1-2.el9.noarch 2.0 MB/s | 358 kB 00:00 2024-02-13T14:07:28.173 INFO:teuthology.orchestra.run.smithi087.stdout:(24/133): ceph-base-19.0.0-1423.ge1b79d5a.el9.x 591 kB/s | 5.3 MB 00:09 2024-02-13T14:07:28.229 INFO:teuthology.orchestra.run.smithi148.stdout:(106/133): python3-grpcio-tools-1.46.7-10.el9.x 1.9 MB/s | 144 kB 00:00 2024-02-13T14:07:28.265 INFO:teuthology.orchestra.run.smithi087.stdout:(25/133): rbd-mirror-19.0.0-1423.ge1b79d5a.el9. 13 MB/s | 3.1 MB 00:00 2024-02-13T14:07:28.280 INFO:teuthology.orchestra.run.smithi148.stdout:(107/133): python3-jaraco-8.2.1-3.el9.noarch.rp 214 kB/s | 11 kB 00:00 2024-02-13T14:07:28.307 INFO:teuthology.orchestra.run.smithi087.stdout:(26/133): ceph-grafana-dashboards-19.0.0-1423.g 759 kB/s | 31 kB 00:00 2024-02-13T14:07:28.330 INFO:teuthology.orchestra.run.smithi148.stdout:(108/133): python3-jaraco-classes-3.2.1-5.el9.n 355 kB/s | 18 kB 00:00 2024-02-13T14:07:28.357 INFO:teuthology.orchestra.run.smithi087.stdout:(27/133): ceph-mgr-cephadm-19.0.0-1423.ge1b79d5 2.7 MB/s | 139 kB 00:00 2024-02-13T14:07:28.380 INFO:teuthology.orchestra.run.smithi148.stdout:(109/133): python3-jaraco-collections-3.0.0-8.e 463 kB/s | 23 kB 00:00 2024-02-13T14:07:28.383 INFO:teuthology.orchestra.run.smithi087.stdout:(28/133): rbd-nbd-19.0.0-1423.ge1b79d5a.el9.x86 816 kB/s | 171 kB 00:00 2024-02-13T14:07:28.430 INFO:teuthology.orchestra.run.smithi148.stdout:(110/133): python3-jaraco-functools-3.5.0-2.el9 389 kB/s | 19 kB 00:00 2024-02-13T14:07:28.489 INFO:teuthology.orchestra.run.smithi148.stdout:(111/133): libarrow-9.0.0-10.el9.x86_64.rpm 4.3 MB/s | 4.4 MB 00:01 2024-02-13T14:07:28.491 INFO:teuthology.orchestra.run.smithi087.stdout:(29/133): ceph-mgr-dashboard-19.0.0-1423.ge1b79 13 MB/s | 1.8 MB 00:00 2024-02-13T14:07:28.514 INFO:teuthology.orchestra.run.smithi148.stdout:(112/133): python3-jaraco-text-3.2.0-6.el9.noar 235 kB/s | 20 kB 00:00 2024-02-13T14:07:28.541 INFO:teuthology.orchestra.run.smithi087.stdout:(30/133): ceph-mgr-modules-core-19.0.0-1423.ge1 4.9 MB/s | 248 kB 00:00 2024-02-13T14:07:28.565 INFO:teuthology.orchestra.run.smithi148.stdout:(113/133): python3-logutils-0.3.5-21.el9.noarch 927 kB/s | 46 kB 00:00 2024-02-13T14:07:28.592 INFO:teuthology.orchestra.run.smithi087.stdout:(31/133): ceph-mgr-rook-19.0.0-1423.ge1b79d5a.e 1.0 MB/s | 52 kB 00:00 2024-02-13T14:07:28.650 INFO:teuthology.orchestra.run.smithi087.stdout:(32/133): ceph-prometheus-alerts-19.0.0-1423.ge 264 kB/s | 15 kB 00:00 2024-02-13T14:07:28.701 INFO:teuthology.orchestra.run.smithi087.stdout:(33/133): ceph-volume-19.0.0-1423.ge1b79d5a.el9 5.2 MB/s | 264 kB 00:00 2024-02-13T14:07:28.715 INFO:teuthology.orchestra.run.smithi148.stdout:(114/133): python3-grpcio-1.46.7-10.el9.x86_64. 3.5 MB/s | 2.0 MB 00:00 2024-02-13T14:07:28.757 INFO:teuthology.orchestra.run.smithi148.stdout:(115/133): python3-more-itertools-8.12.0-2.el9. 411 kB/s | 79 kB 00:00 2024-02-13T14:07:28.784 INFO:teuthology.orchestra.run.smithi087.stdout:(34/133): cephadm-19.0.0-1423.ge1b79d5a.el9.noa 8.9 MB/s | 760 kB 00:00 2024-02-13T14:07:28.797 INFO:teuthology.orchestra.run.smithi148.stdout:(116/133): python3-natsort-7.1.1-5.el9.noarch.r 703 kB/s | 58 kB 00:00 2024-02-13T14:07:28.824 INFO:teuthology.orchestra.run.smithi148.stdout:(117/133): python3-kubernetes-26.1.0-2.el9.noar 3.0 MB/s | 1.0 MB 00:00 2024-02-13T14:07:28.850 INFO:teuthology.orchestra.run.smithi148.stdout:(118/133): python3-portend-3.1.0-2.el9.noarch.r 318 kB/s | 16 kB 00:00 2024-02-13T14:07:28.875 INFO:teuthology.orchestra.run.smithi148.stdout:(119/133): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.7 MB/s | 90 kB 00:00 2024-02-13T14:07:28.900 INFO:teuthology.orchestra.run.smithi148.stdout:(120/133): python3-repoze-lru-0.7-16.el9.noarch 613 kB/s | 31 kB 00:00 2024-02-13T14:07:28.926 INFO:teuthology.orchestra.run.smithi087.stdout:(35/133): boost-program-options-1.75.0-8.el9.x8 747 kB/s | 106 kB 00:00 2024-02-13T14:07:28.945 INFO:teuthology.orchestra.run.smithi148.stdout:(121/133): python3-routes-2.5.1-5.el9.noarch.rp 2.8 MB/s | 188 kB 00:00 2024-02-13T14:07:28.967 INFO:teuthology.orchestra.run.smithi148.stdout:(122/133): python3-rsa-4.9-2.el9.noarch.rpm 886 kB/s | 59 kB 00:00 2024-02-13T14:07:28.992 INFO:teuthology.orchestra.run.smithi148.stdout:(123/133): python3-tempora-5.0.0-2.el9.noarch.r 716 kB/s | 36 kB 00:00 2024-02-13T14:07:29.002 INFO:teuthology.orchestra.run.smithi087.stdout:(36/133): flexiblas-3.0.4-8.el9.x86_64.rpm 421 kB/s | 32 kB 00:00 2024-02-13T14:07:29.018 INFO:teuthology.orchestra.run.smithi148.stdout:(124/133): python3-typing-extensions-4.4.0-2.el 1.0 MB/s | 51 kB 00:00 2024-02-13T14:07:29.052 INFO:teuthology.orchestra.run.smithi148.stdout:(125/133): python3-webob-1.8.7-6.el9.noarch.rpm 3.8 MB/s | 230 kB 00:00 2024-02-13T14:07:29.077 INFO:teuthology.orchestra.run.smithi148.stdout:(126/133): python3-websocket-client-1.2.3-2.el9 1.5 MB/s | 90 kB 00:00 2024-02-13T14:07:29.127 INFO:teuthology.orchestra.run.smithi148.stdout:(127/133): python3-zc-lockfile-2.0-10.el9.noarc 401 kB/s | 20 kB 00:00 2024-02-13T14:07:29.160 INFO:teuthology.orchestra.run.smithi087.stdout:(37/133): flexiblas-netlib-3.0.4-8.el9.x86_64.r 19 MB/s | 3.0 MB 00:00 2024-02-13T14:07:29.169 INFO:teuthology.orchestra.run.smithi148.stdout:(128/133): python3-werkzeug-2.0.3-3.el9.1.noarc 3.6 MB/s | 427 kB 00:00 2024-02-13T14:07:29.194 INFO:teuthology.orchestra.run.smithi087.stdout:(38/133): flexiblas-openblas-openmp-3.0.4-8.el9 516 kB/s | 17 kB 00:00 2024-02-13T14:07:29.194 INFO:teuthology.orchestra.run.smithi148.stdout:(129/133): python3-pecan-1.4.2-3.el9.noarch.rpm 623 kB/s | 272 kB 00:00 2024-02-13T14:07:29.219 INFO:teuthology.orchestra.run.smithi148.stdout:(130/133): re2-20211101-3.el9.x86_64.rpm 2.0 MB/s | 192 kB 00:00 2024-02-13T14:07:29.236 INFO:teuthology.orchestra.run.smithi087.stdout:(39/133): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.7 MB/s | 159 kB 00:00 2024-02-13T14:07:29.270 INFO:teuthology.orchestra.run.smithi087.stdout:(40/133): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.3 MB/s | 44 kB 00:00 2024-02-13T14:07:29.320 INFO:teuthology.orchestra.run.smithi087.stdout:(41/133): librdkafka-1.6.1-102.el9.x86_64.rpm 13 MB/s | 661 kB 00:00 2024-02-13T14:07:29.629 INFO:teuthology.orchestra.run.smithi087.stdout:(42/133): lttng-ust-2.12.0-6.el9.x86_64.rpm 926 kB/s | 282 kB 00:00 2024-02-13T14:07:29.671 INFO:teuthology.orchestra.run.smithi087.stdout:(43/133): lua-5.4.4-4.el9.x86_64.rpm 4.4 MB/s | 187 kB 00:00 2024-02-13T14:07:29.686 INFO:teuthology.orchestra.run.smithi148.stdout:(131/133): librados2-19.0.0-1423.ge1b79d5a.el9. 6.9 MB/s | 3.4 MB 00:00 2024-02-13T14:07:29.704 INFO:teuthology.orchestra.run.smithi087.stdout:(44/133): openblas-0.3.21-2.el9.x86_64.rpm 1.0 MB/s | 34 kB 00:00 2024-02-13T14:07:29.728 INFO:teuthology.orchestra.run.smithi148.stdout:(132/133): thrift-0.15.0-2.el9.x86_64.rpm 2.8 MB/s | 1.6 MB 00:00 2024-02-13T14:07:29.871 INFO:teuthology.orchestra.run.smithi087.stdout:(45/133): openblas-openmp-0.3.21-2.el9.x86_64.r 28 MB/s | 4.7 MB 00:00 2024-02-13T14:07:29.938 INFO:teuthology.orchestra.run.smithi087.stdout:(46/133): protobuf-3.14.0-13.el9.x86_64.rpm 15 MB/s | 1.0 MB 00:00 2024-02-13T14:07:29.972 INFO:teuthology.orchestra.run.smithi087.stdout:(47/133): python3-attrs-20.3.0-7.el9.noarch.rpm 2.3 MB/s | 78 kB 00:00 2024-02-13T14:07:30.156 INFO:teuthology.orchestra.run.smithi087.stdout:(48/133): python3-babel-2.9.1-2.el9.noarch.rpm 32 MB/s | 5.8 MB 00:00 2024-02-13T14:07:30.197 INFO:teuthology.orchestra.run.smithi087.stdout:(49/133): python3-iniconfig-1.1.1-7.el9.noarch. 396 kB/s | 16 kB 00:00 2024-02-13T14:07:30.273 INFO:teuthology.orchestra.run.smithi087.stdout:(50/133): python3-jinja2-2.11.3-5.el9.noarch.rp 3.0 MB/s | 227 kB 00:00 2024-02-13T14:07:30.306 INFO:teuthology.orchestra.run.smithi087.stdout:(51/133): python3-jsonpatch-1.21-16.el9.noarch. 734 kB/s | 24 kB 00:00 2024-02-13T14:07:30.340 INFO:teuthology.orchestra.run.smithi087.stdout:(52/133): python3-jsonpointer-2.0-4.el9.noarch. 541 kB/s | 18 kB 00:00 2024-02-13T14:07:30.382 INFO:teuthology.orchestra.run.smithi087.stdout:(53/133): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 144 kB 00:00 2024-02-13T14:07:30.416 INFO:teuthology.orchestra.run.smithi087.stdout:(54/133): python3-markupsafe-1.1.1-12.el9.x86_6 948 kB/s | 32 kB 00:00 2024-02-13T14:07:30.616 INFO:teuthology.orchestra.run.smithi087.stdout:(55/133): python3-numpy-1.20.1-5.el9.x86_64.rpm 24 MB/s | 4.8 MB 00:00 2024-02-13T14:07:30.683 INFO:teuthology.orchestra.run.smithi087.stdout:(56/133): python3-numpy-f2py-1.20.1-5.el9.x86_6 3.2 MB/s | 217 kB 00:00 2024-02-13T14:07:30.750 INFO:teuthology.orchestra.run.smithi087.stdout:(57/133): python3-oauthlib-3.1.1-5.el9.noarch.r 2.6 MB/s | 174 kB 00:00 2024-02-13T14:07:30.817 INFO:teuthology.orchestra.run.smithi087.stdout:(58/133): python3-packaging-20.9-5.el9.noarch.r 1.0 MB/s | 69 kB 00:00 2024-02-13T14:07:30.875 INFO:teuthology.orchestra.run.smithi087.stdout:(59/133): python3-pluggy-0.13.1-7.el9.noarch.rp 649 kB/s | 38 kB 00:00 2024-02-13T14:07:30.935 INFO:teuthology.orchestra.run.smithi087.stdout:(60/133): python3-prettytable-0.7.2-27.el9.noar 704 kB/s | 41 kB 00:00 2024-02-13T14:07:31.001 INFO:teuthology.orchestra.run.smithi087.stdout:(61/133): python3-protobuf-3.14.0-13.el9.noarch 3.5 MB/s | 238 kB 00:00 2024-02-13T14:07:31.052 INFO:teuthology.orchestra.run.smithi087.stdout:(62/133): python3-py-1.10.0-6.el9.noarch.rpm 8.4 MB/s | 427 kB 00:00 2024-02-13T14:07:31.094 INFO:teuthology.orchestra.run.smithi087.stdout:(63/133): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 132 kB 00:00 2024-02-13T14:07:31.136 INFO:teuthology.orchestra.run.smithi087.stdout:(64/133): python3-pyasn1-modules-0.4.8-6.el9.no 4.9 MB/s | 211 kB 00:00 2024-02-13T14:07:31.194 INFO:teuthology.orchestra.run.smithi087.stdout:(65/133): python3-pytest-6.2.2-6.el9.noarch.rpm 8.0 MB/s | 474 kB 00:00 2024-02-13T14:07:31.236 INFO:teuthology.orchestra.run.smithi087.stdout:(66/133): python3-pytz-2021.1-5.el9.noarch.rpm 1.1 MB/s | 47 kB 00:00 2024-02-13T14:07:31.278 INFO:teuthology.orchestra.run.smithi087.stdout:(67/133): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 43 kB 00:00 2024-02-13T14:07:31.329 INFO:teuthology.orchestra.run.smithi148.stdout:(133/133): librbd1-19.0.0-1423.ge1b79d5a.el9.x8 1.5 MB/s | 3.1 MB 00:02 2024-02-13T14:07:31.334 INFO:teuthology.orchestra.run.smithi148.stdout:-------------------------------------------------------------------------------- 2024-02-13T14:07:31.334 INFO:teuthology.orchestra.run.smithi148.stdout:Total 9.3 MB/s | 195 MB 00:20 2024-02-13T14:07:31.380 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction check 2024-02-13T14:07:31.485 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction check succeeded. 2024-02-13T14:07:31.485 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction test 2024-02-13T14:07:31.995 INFO:teuthology.orchestra.run.smithi087.stdout:(68/133): python3-scipy-1.6.2-8.el9.x86_64.rpm 22 MB/s | 16 MB 00:00 2024-02-13T14:07:32.054 INFO:teuthology.orchestra.run.smithi087.stdout:(69/133): python3-toml-0.10.2-6.el9.noarch.rpm 666 kB/s | 39 kB 00:00 2024-02-13T14:07:32.104 INFO:teuthology.orchestra.run.smithi087.stdout:(70/133): qatlib-23.11.0-1.el9.x86_64.rpm 4.2 MB/s | 215 kB 00:00 2024-02-13T14:07:32.146 INFO:teuthology.orchestra.run.smithi087.stdout:(71/133): qatlib-service-23.11.0-1.el9.x86_64.r 814 kB/s | 34 kB 00:00 2024-02-13T14:07:32.188 INFO:teuthology.orchestra.run.smithi087.stdout:(72/133): qatzip-libs-1.1.2-2.el9.x86_64.rpm 1.0 MB/s | 43 kB 00:00 2024-02-13T14:07:32.246 INFO:teuthology.orchestra.run.smithi087.stdout:(73/133): socat-1.7.4.1-5.el9.x86_64.rpm 5.0 MB/s | 300 kB 00:00 2024-02-13T14:07:32.355 INFO:teuthology.orchestra.run.smithi087.stdout:(74/133): ceph-mgr-diskprediction-local-19.0.0- 1.9 MB/s | 7.4 MB 00:03 2024-02-13T14:07:32.380 INFO:teuthology.orchestra.run.smithi087.stdout:(75/133): xmlstarlet-1.6.1-20.el9.x86_64.rpm 469 kB/s | 63 kB 00:00 2024-02-13T14:07:32.406 INFO:teuthology.orchestra.run.smithi087.stdout:(76/133): fuse-2.9.9-15.el9.x86_64.rpm 1.5 MB/s | 79 kB 00:00 2024-02-13T14:07:32.431 INFO:teuthology.orchestra.run.smithi087.stdout:(77/133): jq-1.6-16.el9.x86_64.rpm 3.6 MB/s | 186 kB 00:00 2024-02-13T14:07:32.481 INFO:teuthology.orchestra.run.smithi087.stdout:(78/133): libgfortran-11.4.1-3.el9.x86_64.rpm 10 MB/s | 802 kB 00:00 2024-02-13T14:07:32.506 INFO:teuthology.orchestra.run.smithi087.stdout:(79/133): libquadmath-11.4.1-3.el9.x86_64.rpm 2.5 MB/s | 194 kB 00:00 2024-02-13T14:07:32.540 INFO:teuthology.orchestra.run.smithi087.stdout:(80/133): oniguruma-6.9.6-1.el9.5.x86_64.rpm 3.6 MB/s | 217 kB 00:00 2024-02-13T14:07:32.565 INFO:teuthology.orchestra.run.smithi087.stdout:(81/133): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.0 MB/s | 241 kB 00:00 2024-02-13T14:07:32.633 INFO:teuthology.orchestra.run.smithi087.stdout:(82/133): python3-cryptography-36.0.1-4.el9.x86 12 MB/s | 1.1 MB 00:00 2024-02-13T14:07:32.657 INFO:teuthology.orchestra.run.smithi087.stdout:(83/133): python3-ply-3.11-14.el9.noarch.rpm 1.1 MB/s | 103 kB 00:00 2024-02-13T14:07:32.683 INFO:teuthology.orchestra.run.smithi087.stdout:(84/133): python3-pycparser-2.20-6.el9.noarch.r 2.4 MB/s | 124 kB 00:00 2024-02-13T14:07:32.716 INFO:teuthology.orchestra.run.smithi087.stdout:(85/133): python3-requests-2.25.1-8.el9.noarch. 1.9 MB/s | 113 kB 00:00 2024-02-13T14:07:32.742 INFO:teuthology.orchestra.run.smithi087.stdout:(86/133): python3-urllib3-1.26.5-5.el9.noarch.r 3.1 MB/s | 187 kB 00:00 2024-02-13T14:07:32.867 INFO:teuthology.orchestra.run.smithi087.stdout:(87/133): lua-devel-5.4.4-4.el9.x86_64.rpm 179 kB/s | 22 kB 00:00 2024-02-13T14:07:32.993 INFO:teuthology.orchestra.run.smithi087.stdout:(88/133): python3-devel-3.9.17-2.el9.x86_64.rpm 892 kB/s | 245 kB 00:00 2024-02-13T14:07:33.102 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction test succeeded. 2024-02-13T14:07:33.103 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction 2024-02-13T14:07:33.218 INFO:teuthology.orchestra.run.smithi087.stdout:(89/133): protobuf-compiler-3.14.0-13.el9.x86_6 2.4 MB/s | 863 kB 00:00 2024-02-13T14:07:33.501 INFO:teuthology.orchestra.run.smithi087.stdout:(90/133): ceph-common-19.0.0-1423.ge1b79d5a.el9 1.4 MB/s | 20 MB 00:14 2024-02-13T14:07:33.869 INFO:teuthology.orchestra.run.smithi087.stdout:(91/133): grpc-data-1.46.7-10.el9.noarch.rpm 53 kB/s | 19 kB 00:00 2024-02-13T14:07:33.894 INFO:teuthology.orchestra.run.smithi087.stdout:(92/133): abseil-cpp-20211102.0-3.el9.x86_64.rp 608 kB/s | 548 kB 00:00 2024-02-13T14:07:34.027 INFO:teuthology.orchestra.run.smithi087.stdout:(93/133): gperftools-libs-2.9.1-2.el9.x86_64.rp 382 kB/s | 309 kB 00:00 2024-02-13T14:07:34.069 INFO:teuthology.orchestra.run.smithi087.stdout:(94/133): libarrow-doc-9.0.0-10.el9.noarch.rpm 148 kB/s | 26 kB 00:00 2024-02-13T14:07:34.136 INFO:teuthology.orchestra.run.smithi087.stdout:(95/133): liboath-2.6.7-2.el9.x86_64.rpm 447 kB/s | 49 kB 00:00 2024-02-13T14:07:34.178 INFO:teuthology.orchestra.run.smithi087.stdout:(96/133): libunwind-1.6.2-1.el9.x86_64.rpm 621 kB/s | 67 kB 00:00 2024-02-13T14:07:34.279 INFO:teuthology.orchestra.run.smithi087.stdout:(97/133): luarocks-3.9.2-1.el9.noarch.rpm 1.0 MB/s | 151 kB 00:00 2024-02-13T14:07:34.495 INFO:teuthology.orchestra.run.smithi087.stdout:(98/133): parquet-libs-9.0.0-10.el9.x86_64.rpm 2.6 MB/s | 838 kB 00:00 2024-02-13T14:07:34.595 INFO:teuthology.orchestra.run.smithi087.stdout:(99/133): python3-asyncssh-2.13.2-3.el9.noarch. 1.7 MB/s | 548 kB 00:00 2024-02-13T14:07:34.621 INFO:teuthology.orchestra.run.smithi087.stdout:(100/133): python3-bcrypt-3.2.2-1.el9.x86_64.rp 347 kB/s | 43 kB 00:00 2024-02-13T14:07:34.704 INFO:teuthology.orchestra.run.smithi087.stdout:(101/133): python3-cachetools-4.2.4-1.el9.noarc 296 kB/s | 32 kB 00:00 2024-02-13T14:07:34.846 INFO:teuthology.orchestra.run.smithi087.stdout:(102/133): python3-cheroot-8.6.0-4.el9.noarch.r 1.2 MB/s | 172 kB 00:00 2024-02-13T14:07:34.913 INFO:teuthology.orchestra.run.smithi087.stdout:(103/133): libarrow-9.0.0-10.el9.x86_64.rpm 4.2 MB/s | 4.4 MB 00:01 2024-02-13T14:07:34.938 INFO:teuthology.orchestra.run.smithi087.stdout:(104/133): python3-certifi-2021.10.8-2.el9.noar 47 kB/s | 15 kB 00:00 2024-02-13T14:07:35.047 INFO:teuthology.orchestra.run.smithi087.stdout:(105/133): python3-google-auth-2.27.0-1.el9.noa 1.5 MB/s | 203 kB 00:00 2024-02-13T14:07:35.089 INFO:teuthology.orchestra.run.smithi087.stdout:(106/133): python3-cherrypy-18.6.1-2.el9.noarch 1.4 MB/s | 358 kB 00:00 2024-02-13T14:07:35.181 INFO:teuthology.orchestra.run.smithi087.stdout:(107/133): python3-grpcio-1.46.7-10.el9.x86_64. 8.4 MB/s | 2.0 MB 00:00 2024-02-13T14:07:35.206 INFO:teuthology.orchestra.run.smithi087.stdout:(108/133): python3-jaraco-8.2.1-3.el9.noarch.rp 91 kB/s | 11 kB 00:00 2024-02-13T14:07:35.232 INFO:teuthology.orchestra.run.smithi087.stdout:(109/133): python3-grpcio-tools-1.46.7-10.el9.x 785 kB/s | 144 kB 00:00 2024-02-13T14:07:35.273 INFO:teuthology.orchestra.run.smithi087.stdout:(110/133): python3-jaraco-classes-3.2.1-5.el9.n 192 kB/s | 18 kB 00:00 2024-02-13T14:07:35.299 INFO:teuthology.orchestra.run.smithi087.stdout:(111/133): python3-jaraco-collections-3.0.0-8.e 251 kB/s | 23 kB 00:00 2024-02-13T14:07:35.324 INFO:teuthology.orchestra.run.smithi087.stdout:(112/133): python3-jaraco-functools-3.5.0-2.el9 211 kB/s | 19 kB 00:00 2024-02-13T14:07:35.349 INFO:teuthology.orchestra.run.smithi087.stdout:(113/133): python3-jaraco-text-3.2.0-6.el9.noar 261 kB/s | 20 kB 00:00 2024-02-13T14:07:35.399 INFO:teuthology.orchestra.run.smithi087.stdout:(114/133): python3-kubernetes-26.1.0-2.el9.noar 10 MB/s | 1.0 MB 00:00 2024-02-13T14:07:35.425 INFO:teuthology.orchestra.run.smithi087.stdout:(115/133): python3-logutils-0.3.5-21.el9.noarch 461 kB/s | 46 kB 00:00 2024-02-13T14:07:35.450 INFO:teuthology.orchestra.run.smithi087.stdout:(116/133): python3-more-itertools-8.12.0-2.el9. 783 kB/s | 79 kB 00:00 2024-02-13T14:07:35.492 INFO:teuthology.orchestra.run.smithi087.stdout:(117/133): python3-natsort-7.1.1-5.el9.noarch.r 626 kB/s | 58 kB 00:00 2024-02-13T14:07:35.526 INFO:teuthology.orchestra.run.smithi087.stdout:(118/133): python3-pecan-1.4.2-3.el9.noarch.rpm 2.6 MB/s | 272 kB 00:00 2024-02-13T14:07:35.551 INFO:teuthology.orchestra.run.smithi087.stdout:(119/133): python3-portend-3.1.0-2.el9.noarch.r 163 kB/s | 16 kB 00:00 2024-02-13T14:07:35.584 INFO:teuthology.orchestra.run.smithi087.stdout:(120/133): python3-pyOpenSSL-21.0.0-1.el9.noarc 976 kB/s | 90 kB 00:00 2024-02-13T14:07:35.635 INFO:teuthology.orchestra.run.smithi087.stdout:(121/133): python3-repoze-lru-0.7-16.el9.noarch 282 kB/s | 31 kB 00:00 2024-02-13T14:07:35.668 INFO:teuthology.orchestra.run.smithi087.stdout:(122/133): python3-routes-2.5.1-5.el9.noarch.rp 1.6 MB/s | 188 kB 00:00 2024-02-13T14:07:35.694 INFO:teuthology.orchestra.run.smithi087.stdout:(123/133): python3-rsa-4.9-2.el9.noarch.rpm 541 kB/s | 59 kB 00:00 2024-02-13T14:07:35.719 INFO:teuthology.orchestra.run.smithi087.stdout:(124/133): python3-tempora-5.0.0-2.el9.noarch.r 427 kB/s | 36 kB 00:00 2024-02-13T14:07:35.761 INFO:teuthology.orchestra.run.smithi087.stdout:(125/133): python3-typing-extensions-4.4.0-2.el 553 kB/s | 51 kB 00:00 2024-02-13T14:07:35.786 INFO:teuthology.orchestra.run.smithi087.stdout:(126/133): python3-webob-1.8.7-6.el9.noarch.rpm 2.4 MB/s | 230 kB 00:00 2024-02-13T14:07:35.812 INFO:teuthology.orchestra.run.smithi087.stdout:(127/133): python3-websocket-client-1.2.3-2.el9 972 kB/s | 90 kB 00:00 2024-02-13T14:07:35.854 INFO:teuthology.orchestra.run.smithi087.stdout:(128/133): python3-zc-lockfile-2.0-10.el9.noarc 298 kB/s | 20 kB 00:00 2024-02-13T14:07:35.895 INFO:teuthology.orchestra.run.smithi087.stdout:(129/133): python3-werkzeug-2.0.3-3.el9.1.noarc 3.1 MB/s | 427 kB 00:00 2024-02-13T14:07:35.920 INFO:teuthology.orchestra.run.smithi087.stdout:(130/133): re2-20211101-3.el9.x86_64.rpm 1.7 MB/s | 192 kB 00:00 2024-02-13T14:07:35.979 INFO:teuthology.orchestra.run.smithi087.stdout:(131/133): thrift-0.15.0-2.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2024-02-13T14:07:36.221 INFO:teuthology.orchestra.run.smithi087.stdout:(132/133): librados2-19.0.0-1423.ge1b79d5a.el9. 10 MB/s | 3.4 MB 00:00 2024-02-13T14:07:36.680 INFO:teuthology.orchestra.run.smithi087.stdout:(133/133): librbd1-19.0.0-1423.ge1b79d5a.el9.x8 4.1 MB/s | 3.1 MB 00:00 2024-02-13T14:07:36.686 INFO:teuthology.orchestra.run.smithi087.stdout:-------------------------------------------------------------------------------- 2024-02-13T14:07:36.686 INFO:teuthology.orchestra.run.smithi087.stdout:Total 11 MB/s | 195 MB 00:18 2024-02-13T14:07:36.731 INFO:teuthology.orchestra.run.smithi087.stdout:Running transaction check 2024-02-13T14:07:36.835 INFO:teuthology.orchestra.run.smithi087.stdout:Transaction check succeeded. 2024-02-13T14:07:36.835 INFO:teuthology.orchestra.run.smithi087.stdout:Running transaction test 2024-02-13T14:07:38.336 INFO:teuthology.orchestra.run.smithi148.stdout: Preparing : 1/1 2024-02-13T14:07:38.414 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/135 2024-02-13T14:07:38.447 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/135 2024-02-13T14:07:38.496 INFO:teuthology.orchestra.run.smithi087.stdout:Transaction test succeeded. 2024-02-13T14:07:38.497 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/135 2024-02-13T14:07:38.497 INFO:teuthology.orchestra.run.smithi087.stdout:Running transaction 2024-02-13T14:07:38.840 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 4/135 2024-02-13T14:07:38.898 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : librados2-2:19.0.0-1423.ge1b79d5a.el9.x86_64 5/135 2024-02-13T14:07:39.020 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: librados2-2:19.0.0-1423.ge1b79d5a.el9.x86_64 5/135 2024-02-13T14:07:39.049 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libcephfs2-2:19.0.0-1423.ge1b79d5a.el9.x86_64 6/135 2024-02-13T14:07:39.141 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: libcephfs2-2:19.0.0-1423.ge1b79d5a.el9.x86_64 6/135 2024-02-13T14:07:39.196 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-rados-2:19.0.0-1423.ge1b79d5a.el9.x86_64 7/135 2024-02-13T14:07:39.229 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/135 2024-02-13T14:07:39.265 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/135 2024-02-13T14:07:39.296 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libcephsqlite-2:19.0.0-1423.ge1b79d5a.el9.x86_64 10/135 2024-02-13T14:07:39.391 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1423.ge1b79d5a.el9.x86_64 10/135 2024-02-13T14:07:39.414 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libradosstriper1-2:19.0.0-1423.ge1b79d5a.el9.x86 11/135 2024-02-13T14:07:39.468 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1423.ge1b79d5a.el9.x86 11/135 2024-02-13T14:07:39.490 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 12/135 2024-02-13T14:07:39.505 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 12/135 2024-02-13T14:07:39.649 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 13/135 2024-02-13T14:07:39.682 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 13/135 2024-02-13T14:07:39.790 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : re2-1:20211101-3.el9.x86_64 14/135 2024-02-13T14:07:39.886 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 15/135 2024-02-13T14:07:39.922 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 16/135 2024-02-13T14:07:39.994 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : liboath-2.6.7-2.el9.x86_64 17/135 2024-02-13T14:07:40.044 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 18/135 2024-02-13T14:07:40.078 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 19/135 2024-02-13T14:07:40.119 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 20/135 2024-02-13T14:07:40.156 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 21/135 2024-02-13T14:07:40.186 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 22/135 2024-02-13T14:07:40.226 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 23/135 2024-02-13T14:07:40.290 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 24/135 2024-02-13T14:07:40.335 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 25/135 2024-02-13T14:07:40.368 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-packaging-20.9-5.el9.noarch 26/135 2024-02-13T14:07:40.420 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 27/135 2024-02-13T14:07:40.453 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 28/135 2024-02-13T14:07:40.480 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : lua-5.4.4-4.el9.x86_64 29/135 2024-02-13T14:07:40.558 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 30/135 2024-02-13T14:07:40.594 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-ceph-common-2:19.0.0-1423.ge1b79d5a.el9. 31/135 2024-02-13T14:07:40.637 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-ceph-argparse-2:19.0.0-1423.ge1b79d5a.el 32/135 2024-02-13T14:07:40.666 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-cephfs-2:19.0.0-1423.ge1b79d5a.el9.x86_6 33/135 2024-02-13T14:07:40.735 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/135 2024-02-13T14:07:40.789 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/135 2024-02-13T14:07:40.863 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/135 2024-02-13T14:07:40.997 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/135 2024-02-13T14:07:41.049 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/135 2024-02-13T14:07:41.574 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/135 2024-02-13T14:07:41.653 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/135 2024-02-13T14:07:41.691 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/135 2024-02-13T14:07:41.725 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 42/135 2024-02-13T14:07:41.760 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 43/135 2024-02-13T14:07:41.791 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 44/135 2024-02-13T14:07:41.835 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 45/135 2024-02-13T14:07:41.873 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : librados-devel-2:19.0.0-1423.ge1b79d5a.el9.x86_6 46/135 2024-02-13T14:07:41.904 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/135 2024-02-13T14:07:41.940 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 48/135 2024-02-13T14:07:41.995 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 49/135 2024-02-13T14:07:42.067 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 50/135 2024-02-13T14:07:42.128 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 51/135 2024-02-13T14:07:42.169 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 52/135 2024-02-13T14:07:42.216 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 53/135 2024-02-13T14:07:42.261 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 54/135 2024-02-13T14:07:42.303 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 55/135 2024-02-13T14:07:42.439 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 56/135 2024-02-13T14:07:42.480 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 57/135 2024-02-13T14:07:42.521 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/135 2024-02-13T14:07:42.616 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/135 2024-02-13T14:07:42.655 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-google-auth-1:2.27.0-1.el9.noarch 60/135 2024-02-13T14:07:42.697 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/135 2024-02-13T14:07:42.729 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/135 2024-02-13T14:07:42.753 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/135 2024-02-13T14:07:42.837 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/135 2024-02-13T14:07:42.964 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/135 2024-02-13T14:07:43.064 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/135 2024-02-13T14:07:43.111 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 67/135 2024-02-13T14:07:43.168 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-ply-3.11-14.el9.noarch 68/135 2024-02-13T14:07:43.224 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 69/135 2024-02-13T14:07:43.416 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 70/135 2024-02-13T14:07:43.458 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 71/135 2024-02-13T14:07:43.536 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 72/135 2024-02-13T14:07:43.598 INFO:teuthology.orchestra.run.smithi087.stdout: Preparing : 1/1 2024-02-13T14:07:43.642 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 73/135 2024-02-13T14:07:43.648 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/135 2024-02-13T14:07:43.674 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/135 2024-02-13T14:07:43.737 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/135 2024-02-13T14:07:43.782 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 74/135 2024-02-13T14:07:43.832 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-asyncssh-2.13.2-3.el9.noarch 75/135 2024-02-13T14:07:43.865 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 76/135 2024-02-13T14:07:43.897 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 77/135 2024-02-13T14:07:43.932 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : jq-1.6-16.el9.x86_64 78/135 2024-02-13T14:07:43.965 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : fuse-2.9.9-15.el9.x86_64 79/135 2024-02-13T14:07:44.010 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 80/135 2024-02-13T14:07:44.072 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 4/135 2024-02-13T14:07:44.133 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 81/135 2024-02-13T14:07:44.134 INFO:teuthology.orchestra.run.smithi087.stdout: Upgrading : librados2-2:19.0.0-1423.ge1b79d5a.el9.x86_64 5/135 2024-02-13T14:07:44.230 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-py-1.10.0-6.el9.noarch 82/135 2024-02-13T14:07:44.255 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: librados2-2:19.0.0-1423.ge1b79d5a.el9.x86_64 5/135 2024-02-13T14:07:44.282 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 83/135 2024-02-13T14:07:44.301 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : libcephfs2-2:19.0.0-1423.ge1b79d5a.el9.x86_64 6/135 2024-02-13T14:07:44.313 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 84/135 2024-02-13T14:07:44.352 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 85/135 2024-02-13T14:07:44.376 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: libcephfs2-2:19.0.0-1423.ge1b79d5a.el9.x86_64 6/135 2024-02-13T14:07:44.428 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-rados-2:19.0.0-1423.ge1b79d5a.el9.x86_64 7/135 2024-02-13T14:07:44.450 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-pluggy-0.13.1-7.el9.noarch 86/135 2024-02-13T14:07:44.461 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/135 2024-02-13T14:07:44.498 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/135 2024-02-13T14:07:44.506 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 87/135 2024-02-13T14:07:44.526 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : libcephsqlite-2:19.0.0-1423.ge1b79d5a.el9.x86_64 10/135 2024-02-13T14:07:44.613 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1423.ge1b79d5a.el9.x86_64 10/135 2024-02-13T14:07:44.635 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : libradosstriper1-2:19.0.0-1423.ge1b79d5a.el9.x86 11/135 2024-02-13T14:07:44.682 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1423.ge1b79d5a.el9.x86 11/135 2024-02-13T14:07:44.703 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 12/135 2024-02-13T14:07:44.718 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 12/135 2024-02-13T14:07:44.921 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 13/135 2024-02-13T14:07:44.953 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 13/135 2024-02-13T14:07:45.060 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : re2-1:20211101-3.el9.x86_64 14/135 2024-02-13T14:07:45.164 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 15/135 2024-02-13T14:07:45.194 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 16/135 2024-02-13T14:07:45.258 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 88/135 2024-02-13T14:07:45.288 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : liboath-2.6.7-2.el9.x86_64 17/135 2024-02-13T14:07:45.303 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 89/135 2024-02-13T14:07:45.337 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 90/135 2024-02-13T14:07:45.341 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 18/135 2024-02-13T14:07:45.370 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 91/135 2024-02-13T14:07:45.376 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 19/135 2024-02-13T14:07:45.416 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 20/135 2024-02-13T14:07:45.418 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-iniconfig-1.1.1-7.el9.noarch 92/135 2024-02-13T14:07:45.453 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 21/135 2024-02-13T14:07:45.462 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 93/135 2024-02-13T14:07:45.483 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 22/135 2024-02-13T14:07:45.523 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 23/135 2024-02-13T14:07:45.588 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 24/135 2024-02-13T14:07:45.631 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : openblas-0.3.21-2.el9.x86_64 94/135 2024-02-13T14:07:45.649 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 25/135 2024-02-13T14:07:45.660 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 95/135 2024-02-13T14:07:45.690 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-packaging-20.9-5.el9.noarch 26/135 2024-02-13T14:07:45.733 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 27/135 2024-02-13T14:07:45.746 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 96/135 2024-02-13T14:07:45.770 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 28/135 2024-02-13T14:07:45.797 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : lua-5.4.4-4.el9.x86_64 29/135 2024-02-13T14:07:45.867 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 30/135 2024-02-13T14:07:45.902 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-ceph-common-2:19.0.0-1423.ge1b79d5a.el9. 31/135 2024-02-13T14:07:45.946 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-ceph-argparse-2:19.0.0-1423.ge1b79d5a.el 32/135 2024-02-13T14:07:45.974 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-cephfs-2:19.0.0-1423.ge1b79d5a.el9.x86_6 33/135 2024-02-13T14:07:46.043 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/135 2024-02-13T14:07:46.088 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/135 2024-02-13T14:07:46.162 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/135 2024-02-13T14:07:46.305 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/135 2024-02-13T14:07:46.365 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/135 2024-02-13T14:07:46.414 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 97/135 2024-02-13T14:07:46.533 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 98/135 2024-02-13T14:07:46.887 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/135 2024-02-13T14:07:46.961 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/135 2024-02-13T14:07:47.008 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/135 2024-02-13T14:07:47.042 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 42/135 2024-02-13T14:07:47.076 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 43/135 2024-02-13T14:07:47.107 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 44/135 2024-02-13T14:07:47.144 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 45/135 2024-02-13T14:07:47.181 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : librados-devel-2:19.0.0-1423.ge1b79d5a.el9.x86_6 46/135 2024-02-13T14:07:47.211 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/135 2024-02-13T14:07:47.256 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 48/135 2024-02-13T14:07:47.311 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 49/135 2024-02-13T14:07:47.384 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 50/135 2024-02-13T14:07:47.415 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 51/135 2024-02-13T14:07:47.452 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 52/135 2024-02-13T14:07:47.498 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 53/135 2024-02-13T14:07:47.544 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 54/135 2024-02-13T14:07:47.586 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 55/135 2024-02-13T14:07:47.720 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 56/135 2024-02-13T14:07:47.762 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 57/135 2024-02-13T14:07:47.800 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 99/135 2024-02-13T14:07:47.804 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/135 2024-02-13T14:07:47.882 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 100/135 2024-02-13T14:07:47.898 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/135 2024-02-13T14:07:47.947 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-google-auth-1:2.27.0-1.el9.noarch 60/135 2024-02-13T14:07:47.987 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/135 2024-02-13T14:07:48.019 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/135 2024-02-13T14:07:48.044 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/135 2024-02-13T14:07:48.126 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/135 2024-02-13T14:07:48.195 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 101/135 2024-02-13T14:07:48.219 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : librbd1-2:19.0.0-1423.ge1b79d5a.el9.x86_64 102/135 2024-02-13T14:07:48.253 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/135 2024-02-13T14:07:48.309 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: librbd1-2:19.0.0-1423.ge1b79d5a.el9.x86_64 102/135 2024-02-13T14:07:48.341 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-rbd-2:19.0.0-1423.ge1b79d5a.el9.x86_64 103/135 2024-02-13T14:07:48.352 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/135 2024-02-13T14:07:48.377 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 104/135 2024-02-13T14:07:48.401 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 67/135 2024-02-13T14:07:48.450 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-ply-3.11-14.el9.noarch 68/135 2024-02-13T14:07:48.506 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 69/135 2024-02-13T14:07:48.698 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 70/135 2024-02-13T14:07:48.764 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 71/135 2024-02-13T14:07:48.832 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 105/135 2024-02-13T14:07:48.841 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 72/135 2024-02-13T14:07:48.855 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : librgw2-2:19.0.0-1423.ge1b79d5a.el9.x86_64 106/135 2024-02-13T14:07:48.926 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: librgw2-2:19.0.0-1423.ge1b79d5a.el9.x86_64 106/135 2024-02-13T14:07:48.930 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 73/135 2024-02-13T14:07:48.949 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-rgw-2:19.0.0-1423.ge1b79d5a.el9.x86_64 107/135 2024-02-13T14:07:49.070 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 74/135 2024-02-13T14:07:49.113 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-asyncssh-2.13.2-3.el9.noarch 75/135 2024-02-13T14:07:49.147 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 76/135 2024-02-13T14:07:49.180 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 77/135 2024-02-13T14:07:49.214 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : jq-1.6-16.el9.x86_64 78/135 2024-02-13T14:07:49.246 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : fuse-2.9.9-15.el9.x86_64 79/135 2024-02-13T14:07:49.291 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 80/135 2024-02-13T14:07:49.414 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 81/135 2024-02-13T14:07:49.502 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-py-1.10.0-6.el9.noarch 82/135 2024-02-13T14:07:49.545 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 83/135 2024-02-13T14:07:49.576 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 84/135 2024-02-13T14:07:49.607 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 85/135 2024-02-13T14:07:49.703 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-pluggy-0.13.1-7.el9.noarch 86/135 2024-02-13T14:07:49.760 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 87/135 2024-02-13T14:07:50.494 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 88/135 2024-02-13T14:07:50.541 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 89/135 2024-02-13T14:07:50.575 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 90/135 2024-02-13T14:07:50.617 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 91/135 2024-02-13T14:07:50.673 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-iniconfig-1.1.1-7.el9.noarch 92/135 2024-02-13T14:07:50.708 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 93/135 2024-02-13T14:07:50.885 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : openblas-0.3.21-2.el9.x86_64 94/135 2024-02-13T14:07:50.914 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 95/135 2024-02-13T14:07:50.992 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 96/135 2024-02-13T14:07:51.153 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-common-2:19.0.0-1423.ge1b79d5a.el9.x86_64 108/135 2024-02-13T14:07:51.186 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-common-2:19.0.0-1423.ge1b79d5a.el9.x86_64 108/135 2024-02-13T14:07:51.658 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 97/135 2024-02-13T14:07:51.739 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-common-2:19.0.0-1423.ge1b79d5a.el9.x86_64 108/135 2024-02-13T14:07:51.771 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 98/135 2024-02-13T14:07:51.781 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-base-2:19.0.0-1423.ge1b79d5a.el9.x86_64 109/135 2024-02-13T14:07:51.864 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-base-2:19.0.0-1423.ge1b79d5a.el9.x86_64 109/135 2024-02-13T14:07:51.864 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-02-13T14:07:51.865 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-02-13T14:07:51.865 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-02-13T14:07:51.908 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-selinux-2:19.0.0-1423.ge1b79d5a.el9.x86_64 110/135 2024-02-13T14:07:53.030 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 99/135 2024-02-13T14:07:53.111 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 100/135 2024-02-13T14:07:53.418 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 101/135 2024-02-13T14:07:53.448 INFO:teuthology.orchestra.run.smithi087.stdout: Upgrading : librbd1-2:19.0.0-1423.ge1b79d5a.el9.x86_64 102/135 2024-02-13T14:07:53.538 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: librbd1-2:19.0.0-1423.ge1b79d5a.el9.x86_64 102/135 2024-02-13T14:07:53.569 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-rbd-2:19.0.0-1423.ge1b79d5a.el9.x86_64 103/135 2024-02-13T14:07:53.606 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 104/135 2024-02-13T14:07:54.052 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 105/135 2024-02-13T14:07:54.084 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : librgw2-2:19.0.0-1423.ge1b79d5a.el9.x86_64 106/135 2024-02-13T14:07:54.139 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: librgw2-2:19.0.0-1423.ge1b79d5a.el9.x86_64 106/135 2024-02-13T14:07:54.161 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-rgw-2:19.0.0-1423.ge1b79d5a.el9.x86_64 107/135 2024-02-13T14:07:56.354 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: ceph-common-2:19.0.0-1423.ge1b79d5a.el9.x86_64 108/135 2024-02-13T14:07:56.396 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-common-2:19.0.0-1423.ge1b79d5a.el9.x86_64 108/135 2024-02-13T14:07:56.934 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: ceph-common-2:19.0.0-1423.ge1b79d5a.el9.x86_64 108/135 2024-02-13T14:07:56.975 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-base-2:19.0.0-1423.ge1b79d5a.el9.x86_64 109/135 2024-02-13T14:07:57.064 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: ceph-base-2:19.0.0-1423.ge1b79d5a.el9.x86_64 109/135 2024-02-13T14:07:57.064 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-02-13T14:07:57.064 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-02-13T14:07:57.065 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-02-13T14:07:57.101 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-selinux-2:19.0.0-1423.ge1b79d5a.el9.x86_64 110/135 2024-02-13T14:08:03.382 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1423.ge1b79d5a.el9.x86_64 110/135 2024-02-13T14:08:03.382 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /sys 2024-02-13T14:08:03.382 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /proc 2024-02-13T14:08:03.382 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /mnt 2024-02-13T14:08:03.382 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /var/tmp 2024-02-13T14:08:03.382 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /home 2024-02-13T14:08:03.382 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /root 2024-02-13T14:08:03.382 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /tmp 2024-02-13T14:08:03.382 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-02-13T14:08:03.545 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-volume-2:19.0.0-1423.ge1b79d5a.el9.noarch 111/135 2024-02-13T14:08:03.560 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-volume-2:19.0.0-1423.ge1b79d5a.el9.noarch 111/135 2024-02-13T14:08:03.560 INFO:teuthology.orchestra.run.smithi148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-13T14:08:03.560 INFO:teuthology.orchestra.run.smithi148.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-02-13T14:08:03.560 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-02-13T14:08:05.040 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-osd-2:19.0.0-1423.ge1b79d5a.el9.x86_64 112/135 2024-02-13T14:08:05.065 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-osd-2:19.0.0-1423.ge1b79d5a.el9.x86_64 112/135 2024-02-13T14:08:05.065 INFO:teuthology.orchestra.run.smithi148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-13T14:08:05.066 INFO:teuthology.orchestra.run.smithi148.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-02-13T14:08:05.066 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-02-13T14:08:05.066 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-02-13T14:08:05.067 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-02-13T14:08:05.273 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mds-2:19.0.0-1423.ge1b79d5a.el9.x86_64 113/135 2024-02-13T14:08:05.296 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-mds-2:19.0.0-1423.ge1b79d5a.el9.x86_64 113/135 2024-02-13T14:08:05.296 INFO:teuthology.orchestra.run.smithi148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-13T14:08:05.296 INFO:teuthology.orchestra.run.smithi148.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-02-13T14:08:05.296 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-02-13T14:08:05.296 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-02-13T14:08:05.296 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-02-13T14:08:05.741 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mon-2:19.0.0-1423.ge1b79d5a.el9.x86_64 114/135 2024-02-13T14:08:05.763 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-mon-2:19.0.0-1423.ge1b79d5a.el9.x86_64 114/135 2024-02-13T14:08:05.763 INFO:teuthology.orchestra.run.smithi148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-13T14:08:05.763 INFO:teuthology.orchestra.run.smithi148.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-02-13T14:08:05.763 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-02-13T14:08:05.764 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-02-13T14:08:05.764 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-02-13T14:08:06.152 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: cephadm-2:19.0.0-1423.ge1b79d5a.el9.noarch 115/135 2024-02-13T14:08:06.181 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : cephadm-2:19.0.0-1423.ge1b79d5a.el9.noarch 115/135 2024-02-13T14:08:06.213 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1423.ge1b79d5a.e 116/135 2024-02-13T14:08:06.273 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1423.ge1b79d5a. 117/135 2024-02-13T14:08:06.296 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1423.ge1b79d5a.el9.noa 118/135 2024-02-13T14:08:06.525 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1423.ge1b79d5a.el9.noa 118/135 2024-02-13T14:08:06.555 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1423.ge1b79d5a.el9.n 119/135 2024-02-13T14:08:07.512 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1423.ge1b79d5a.el9.n 119/135 2024-02-13T14:08:07.531 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1423.ge1b 120/135 2024-02-13T14:08:07.630 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1423.ge1b 120/135 2024-02-13T14:08:07.792 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1423.ge1b79d5a.el 121/135 2024-02-13T14:08:07.825 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mgr-2:19.0.0-1423.ge1b79d5a.el9.x86_64 122/135 2024-02-13T14:08:07.847 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1423.ge1b79d5a.el9.x86_64 122/135 2024-02-13T14:08:07.847 INFO:teuthology.orchestra.run.smithi148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-13T14:08:07.848 INFO:teuthology.orchestra.run.smithi148.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-02-13T14:08:07.848 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-02-13T14:08:07.848 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-02-13T14:08:07.848 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-02-13T14:08:07.892 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mgr-rook-2:19.0.0-1423.ge1b79d5a.el9.noarch 123/135 2024-02-13T14:08:07.902 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1423.ge1b79d5a.el9.noarch 123/135 2024-02-13T14:08:08.748 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-2:19.0.0-1423.ge1b79d5a.el9.x86_64 124/135 2024-02-13T14:08:08.787 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-radosgw-2:19.0.0-1423.ge1b79d5a.el9.x86_64 125/135 2024-02-13T14:08:08.809 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1423.ge1b79d5a.el9.x86_64 125/135 2024-02-13T14:08:08.810 INFO:teuthology.orchestra.run.smithi148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-13T14:08:08.810 INFO:teuthology.orchestra.run.smithi148.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-02-13T14:08:08.810 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-02-13T14:08:08.810 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-02-13T14:08:08.810 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-02-13T14:08:08.816 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1423.ge1b79d5a.el9.x86_64 110/135 2024-02-13T14:08:08.816 INFO:teuthology.orchestra.run.smithi087.stdout:skipping the directory /sys 2024-02-13T14:08:08.816 INFO:teuthology.orchestra.run.smithi087.stdout:skipping the directory /proc 2024-02-13T14:08:08.816 INFO:teuthology.orchestra.run.smithi087.stdout:skipping the directory /mnt 2024-02-13T14:08:08.816 INFO:teuthology.orchestra.run.smithi087.stdout:skipping the directory /var/tmp 2024-02-13T14:08:08.816 INFO:teuthology.orchestra.run.smithi087.stdout:skipping the directory /home 2024-02-13T14:08:08.816 INFO:teuthology.orchestra.run.smithi087.stdout:skipping the directory /root 2024-02-13T14:08:08.817 INFO:teuthology.orchestra.run.smithi087.stdout:skipping the directory /tmp 2024-02-13T14:08:08.817 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-02-13T14:08:08.853 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1423.ge1b79 126/135 2024-02-13T14:08:08.873 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1423.ge1b79 126/135 2024-02-13T14:08:08.873 INFO:teuthology.orchestra.run.smithi148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-13T14:08:08.873 INFO:teuthology.orchestra.run.smithi148.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-02-13T14:08:08.873 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-02-13T14:08:09.070 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-volume-2:19.0.0-1423.ge1b79d5a.el9.noarch 111/135 2024-02-13T14:08:09.087 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: ceph-volume-2:19.0.0-1423.ge1b79d5a.el9.noarch 111/135 2024-02-13T14:08:09.087 INFO:teuthology.orchestra.run.smithi087.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-13T14:08:09.087 INFO:teuthology.orchestra.run.smithi087.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-02-13T14:08:09.087 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-02-13T14:08:09.177 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : rbd-mirror-2:19.0.0-1423.ge1b79d5a.el9.x86_64 127/135 2024-02-13T14:08:09.199 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1423.ge1b79d5a.el9.x86_64 127/135 2024-02-13T14:08:09.199 INFO:teuthology.orchestra.run.smithi148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-13T14:08:09.199 INFO:teuthology.orchestra.run.smithi148.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-02-13T14:08:09.199 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-02-13T14:08:09.199 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-02-13T14:08:09.200 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-02-13T14:08:10.565 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-osd-2:19.0.0-1423.ge1b79d5a.el9.x86_64 112/135 2024-02-13T14:08:10.590 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: ceph-osd-2:19.0.0-1423.ge1b79d5a.el9.x86_64 112/135 2024-02-13T14:08:10.590 INFO:teuthology.orchestra.run.smithi087.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-13T14:08:10.590 INFO:teuthology.orchestra.run.smithi087.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-02-13T14:08:10.590 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-02-13T14:08:10.590 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-02-13T14:08:10.590 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-02-13T14:08:10.799 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-mds-2:19.0.0-1423.ge1b79d5a.el9.x86_64 113/135 2024-02-13T14:08:10.821 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: ceph-mds-2:19.0.0-1423.ge1b79d5a.el9.x86_64 113/135 2024-02-13T14:08:10.821 INFO:teuthology.orchestra.run.smithi087.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-13T14:08:10.821 INFO:teuthology.orchestra.run.smithi087.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-02-13T14:08:10.821 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-02-13T14:08:10.821 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-02-13T14:08:10.821 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-02-13T14:08:11.266 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-mon-2:19.0.0-1423.ge1b79d5a.el9.x86_64 114/135 2024-02-13T14:08:11.288 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: ceph-mon-2:19.0.0-1423.ge1b79d5a.el9.x86_64 114/135 2024-02-13T14:08:11.288 INFO:teuthology.orchestra.run.smithi087.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-13T14:08:11.288 INFO:teuthology.orchestra.run.smithi087.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-02-13T14:08:11.288 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-02-13T14:08:11.288 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-02-13T14:08:11.288 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-02-13T14:08:11.752 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: cephadm-2:19.0.0-1423.ge1b79d5a.el9.noarch 115/135 2024-02-13T14:08:11.773 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : cephadm-2:19.0.0-1423.ge1b79d5a.el9.noarch 115/135 2024-02-13T14:08:11.804 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1423.ge1b79d5a.e 116/135 2024-02-13T14:08:11.865 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1423.ge1b79d5a. 117/135 2024-02-13T14:08:11.888 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1423.ge1b79d5a.el9.noa 118/135 2024-02-13T14:08:12.116 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1423.ge1b79d5a.el9.noa 118/135 2024-02-13T14:08:12.147 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1423.ge1b79d5a.el9.n 119/135 2024-02-13T14:08:13.102 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1423.ge1b79d5a.el9.n 119/135 2024-02-13T14:08:13.131 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1423.ge1b 120/135 2024-02-13T14:08:13.231 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1423.ge1b 120/135 2024-02-13T14:08:13.393 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1423.ge1b79d5a.el 121/135 2024-02-13T14:08:13.416 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-mgr-2:19.0.0-1423.ge1b79d5a.el9.x86_64 122/135 2024-02-13T14:08:13.438 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1423.ge1b79d5a.el9.x86_64 122/135 2024-02-13T14:08:13.438 INFO:teuthology.orchestra.run.smithi087.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-13T14:08:13.438 INFO:teuthology.orchestra.run.smithi087.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-02-13T14:08:13.438 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-02-13T14:08:13.438 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-02-13T14:08:13.439 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-02-13T14:08:13.475 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-mgr-rook-2:19.0.0-1423.ge1b79d5a.el9.noarch 123/135 2024-02-13T14:08:13.486 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1423.ge1b79d5a.el9.noarch 123/135 2024-02-13T14:08:13.681 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-test-2:19.0.0-1423.ge1b79d5a.el9.x86_64 128/135 2024-02-13T14:08:13.722 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : rbd-fuse-2:19.0.0-1423.ge1b79d5a.el9.x86_64 129/135 2024-02-13T14:08:13.840 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : rbd-nbd-2:19.0.0-1423.ge1b79d5a.el9.x86_64 130/135 2024-02-13T14:08:13.961 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-pytest-6.2.2-6.el9.noarch 131/135 2024-02-13T14:08:13.997 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-fuse-2:19.0.0-1423.ge1b79d5a.el9.x86_64 132/135 2024-02-13T14:08:14.028 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libcephfs-devel-2:19.0.0-1423.ge1b79d5a.el9.x86_ 133/135 2024-02-13T14:08:14.028 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 134/135 2024-02-13T14:08:14.120 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 134/135 2024-02-13T14:08:14.120 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 135/135 2024-02-13T14:08:14.334 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-2:19.0.0-1423.ge1b79d5a.el9.x86_64 124/135 2024-02-13T14:08:14.368 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-radosgw-2:19.0.0-1423.ge1b79d5a.el9.x86_64 125/135 2024-02-13T14:08:14.390 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1423.ge1b79d5a.el9.x86_64 125/135 2024-02-13T14:08:14.390 INFO:teuthology.orchestra.run.smithi087.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-13T14:08:14.390 INFO:teuthology.orchestra.run.smithi087.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-02-13T14:08:14.390 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-02-13T14:08:14.390 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-02-13T14:08:14.390 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-02-13T14:08:14.435 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1423.ge1b79 126/135 2024-02-13T14:08:14.456 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1423.ge1b79 126/135 2024-02-13T14:08:14.456 INFO:teuthology.orchestra.run.smithi087.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-13T14:08:14.457 INFO:teuthology.orchestra.run.smithi087.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-02-13T14:08:14.457 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-02-13T14:08:14.761 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : rbd-mirror-2:19.0.0-1423.ge1b79d5a.el9.x86_64 127/135 2024-02-13T14:08:14.783 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1423.ge1b79d5a.el9.x86_64 127/135 2024-02-13T14:08:14.783 INFO:teuthology.orchestra.run.smithi087.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-13T14:08:14.783 INFO:teuthology.orchestra.run.smithi087.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-02-13T14:08:14.783 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-02-13T14:08:14.783 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-02-13T14:08:14.783 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-02-13T14:08:18.719 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 135/135 2024-02-13T14:08:18.720 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-2:19.0.0-1423.ge1b79d5a.el9.x86_64 1/135 2024-02-13T14:08:18.720 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-base-2:19.0.0-1423.ge1b79d5a.el9.x86_64 2/135 2024-02-13T14:08:18.720 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-common-2:19.0.0-1423.ge1b79d5a.el9.x86_64 3/135 2024-02-13T14:08:18.720 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-fuse-2:19.0.0-1423.ge1b79d5a.el9.x86_64 4/135 2024-02-13T14:08:18.720 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1423.ge1b79 5/135 2024-02-13T14:08:18.720 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mds-2:19.0.0-1423.ge1b79d5a.el9.x86_64 6/135 2024-02-13T14:08:18.720 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mgr-2:19.0.0-1423.ge1b79d5a.el9.x86_64 7/135 2024-02-13T14:08:18.720 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mon-2:19.0.0-1423.ge1b79d5a.el9.x86_64 8/135 2024-02-13T14:08:18.720 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-osd-2:19.0.0-1423.ge1b79d5a.el9.x86_64 9/135 2024-02-13T14:08:18.720 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-radosgw-2:19.0.0-1423.ge1b79d5a.el9.x86_64 10/135 2024-02-13T14:08:18.720 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-selinux-2:19.0.0-1423.ge1b79d5a.el9.x86_64 11/135 2024-02-13T14:08:18.720 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-test-2:19.0.0-1423.ge1b79d5a.el9.x86_64 12/135 2024-02-13T14:08:18.720 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libcephfs-devel-2:19.0.0-1423.ge1b79d5a.el9.x86_ 13/135 2024-02-13T14:08:18.720 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libcephfs2-2:19.0.0-1423.ge1b79d5a.el9.x86_64 14/135 2024-02-13T14:08:18.721 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libcephsqlite-2:19.0.0-1423.ge1b79d5a.el9.x86_64 15/135 2024-02-13T14:08:18.721 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librados-devel-2:19.0.0-1423.ge1b79d5a.el9.x86_6 16/135 2024-02-13T14:08:18.721 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libradosstriper1-2:19.0.0-1423.ge1b79d5a.el9.x86 17/135 2024-02-13T14:08:18.721 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librgw2-2:19.0.0-1423.ge1b79d5a.el9.x86_64 18/135 2024-02-13T14:08:18.721 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1423.ge1b79d5a.el 19/135 2024-02-13T14:08:18.721 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-ceph-common-2:19.0.0-1423.ge1b79d5a.el9. 20/135 2024-02-13T14:08:18.721 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-cephfs-2:19.0.0-1423.ge1b79d5a.el9.x86_6 21/135 2024-02-13T14:08:18.721 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-rados-2:19.0.0-1423.ge1b79d5a.el9.x86_64 22/135 2024-02-13T14:08:18.721 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-rbd-2:19.0.0-1423.ge1b79d5a.el9.x86_64 23/135 2024-02-13T14:08:18.721 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-rgw-2:19.0.0-1423.ge1b79d5a.el9.x86_64 24/135 2024-02-13T14:08:18.721 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : rbd-fuse-2:19.0.0-1423.ge1b79d5a.el9.x86_64 25/135 2024-02-13T14:08:18.721 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : rbd-mirror-2:19.0.0-1423.ge1b79d5a.el9.x86_64 26/135 2024-02-13T14:08:18.721 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : rbd-nbd-2:19.0.0-1423.ge1b79d5a.el9.x86_64 27/135 2024-02-13T14:08:18.722 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1423.ge1b79d5a. 28/135 2024-02-13T14:08:18.722 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1423.ge1b79d5a.el9.noa 29/135 2024-02-13T14:08:18.722 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1423.ge1b79d5a.el9.n 30/135 2024-02-13T14:08:18.732 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1423.ge1b 31/135 2024-02-13T14:08:18.732 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1423.ge1b79d5a.el 32/135 2024-02-13T14:08:18.732 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1423.ge1b79d5a.el9.noarch 33/135 2024-02-13T14:08:18.732 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1423.ge1b79d5a.e 34/135 2024-02-13T14:08:18.733 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-volume-2:19.0.0-1423.ge1b79d5a.el9.noarch 35/135 2024-02-13T14:08:18.733 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : cephadm-2:19.0.0-1423.ge1b79d5a.el9.noarch 36/135 2024-02-13T14:08:18.733 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 37/135 2024-02-13T14:08:18.733 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 38/135 2024-02-13T14:08:18.733 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 39/135 2024-02-13T14:08:18.733 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 40/135 2024-02-13T14:08:18.733 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 41/135 2024-02-13T14:08:18.733 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 42/135 2024-02-13T14:08:18.733 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 43/135 2024-02-13T14:08:18.733 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 44/135 2024-02-13T14:08:18.733 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : lua-5.4.4-4.el9.x86_64 45/135 2024-02-13T14:08:18.733 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 46/135 2024-02-13T14:08:18.734 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 47/135 2024-02-13T14:08:18.734 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 48/135 2024-02-13T14:08:18.734 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 49/135 2024-02-13T14:08:18.734 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 50/135 2024-02-13T14:08:18.734 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-iniconfig-1.1.1-7.el9.noarch 51/135 2024-02-13T14:08:18.734 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 52/135 2024-02-13T14:08:18.734 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 53/135 2024-02-13T14:08:18.734 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 54/135 2024-02-13T14:08:18.734 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 55/135 2024-02-13T14:08:18.734 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 56/135 2024-02-13T14:08:18.734 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 57/135 2024-02-13T14:08:18.734 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 58/135 2024-02-13T14:08:18.735 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 59/135 2024-02-13T14:08:18.735 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 60/135 2024-02-13T14:08:18.735 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-pluggy-0.13.1-7.el9.noarch 61/135 2024-02-13T14:08:18.735 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 62/135 2024-02-13T14:08:18.735 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 63/135 2024-02-13T14:08:18.735 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-py-1.10.0-6.el9.noarch 64/135 2024-02-13T14:08:18.735 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 65/135 2024-02-13T14:08:18.735 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 66/135 2024-02-13T14:08:18.735 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-pytest-6.2.2-6.el9.noarch 67/135 2024-02-13T14:08:18.735 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 68/135 2024-02-13T14:08:18.735 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 69/135 2024-02-13T14:08:18.735 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 70/135 2024-02-13T14:08:18.735 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 71/135 2024-02-13T14:08:18.736 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 72/135 2024-02-13T14:08:18.736 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 73/135 2024-02-13T14:08:18.736 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 74/135 2024-02-13T14:08:18.736 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 75/135 2024-02-13T14:08:18.736 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 76/135 2024-02-13T14:08:18.736 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 77/135 2024-02-13T14:08:18.736 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : jq-1.6-16.el9.x86_64 78/135 2024-02-13T14:08:18.736 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 79/135 2024-02-13T14:08:18.736 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 80/135 2024-02-13T14:08:18.736 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 81/135 2024-02-13T14:08:18.736 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 82/135 2024-02-13T14:08:18.736 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 83/135 2024-02-13T14:08:18.737 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-ply-3.11-14.el9.noarch 84/135 2024-02-13T14:08:18.737 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 85/135 2024-02-13T14:08:18.737 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 86/135 2024-02-13T14:08:18.737 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 87/135 2024-02-13T14:08:18.737 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 88/135 2024-02-13T14:08:18.737 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 89/135 2024-02-13T14:08:18.737 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 90/135 2024-02-13T14:08:18.737 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 91/135 2024-02-13T14:08:18.737 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 92/135 2024-02-13T14:08:18.737 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 93/135 2024-02-13T14:08:18.737 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 94/135 2024-02-13T14:08:18.737 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 95/135 2024-02-13T14:08:18.737 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 96/135 2024-02-13T14:08:18.737 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 97/135 2024-02-13T14:08:18.738 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 98/135 2024-02-13T14:08:18.738 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 99/135 2024-02-13T14:08:18.738 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-asyncssh-2.13.2-3.el9.noarch 100/135 2024-02-13T14:08:18.738 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 101/135 2024-02-13T14:08:18.738 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 102/135 2024-02-13T14:08:18.738 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 103/135 2024-02-13T14:08:18.738 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 104/135 2024-02-13T14:08:18.738 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 105/135 2024-02-13T14:08:18.738 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-google-auth-1:2.27.0-1.el9.noarch 106/135 2024-02-13T14:08:18.738 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 107/135 2024-02-13T14:08:18.738 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 108/135 2024-02-13T14:08:18.738 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 109/135 2024-02-13T14:08:18.738 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 110/135 2024-02-13T14:08:18.738 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 111/135 2024-02-13T14:08:18.739 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 112/135 2024-02-13T14:08:18.739 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 113/135 2024-02-13T14:08:18.739 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 114/135 2024-02-13T14:08:18.739 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 115/135 2024-02-13T14:08:18.739 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 116/135 2024-02-13T14:08:18.739 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 117/135 2024-02-13T14:08:18.739 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 118/135 2024-02-13T14:08:18.739 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 119/135 2024-02-13T14:08:18.739 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 120/135 2024-02-13T14:08:18.739 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 121/135 2024-02-13T14:08:18.739 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 122/135 2024-02-13T14:08:18.739 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 123/135 2024-02-13T14:08:18.739 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 124/135 2024-02-13T14:08:18.739 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 125/135 2024-02-13T14:08:18.739 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 126/135 2024-02-13T14:08:18.740 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 127/135 2024-02-13T14:08:18.740 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 128/135 2024-02-13T14:08:18.740 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 129/135 2024-02-13T14:08:18.740 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : re2-1:20211101-3.el9.x86_64 130/135 2024-02-13T14:08:18.740 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 131/135 2024-02-13T14:08:18.740 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librados2-2:19.0.0-1423.ge1b79d5a.el9.x86_64 132/135 2024-02-13T14:08:18.740 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 133/135 2024-02-13T14:08:18.740 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librbd1-2:19.0.0-1423.ge1b79d5a.el9.x86_64 134/135 2024-02-13T14:08:19.189 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-test-2:19.0.0-1423.ge1b79d5a.el9.x86_64 128/135 2024-02-13T14:08:19.238 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : rbd-fuse-2:19.0.0-1423.ge1b79d5a.el9.x86_64 129/135 2024-02-13T14:08:19.355 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : rbd-nbd-2:19.0.0-1423.ge1b79d5a.el9.x86_64 130/135 2024-02-13T14:08:19.483 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-pytest-6.2.2-6.el9.noarch 131/135 2024-02-13T14:08:19.513 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-fuse-2:19.0.0-1423.ge1b79d5a.el9.x86_64 132/135 2024-02-13T14:08:19.544 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : libcephfs-devel-2:19.0.0-1423.ge1b79d5a.el9.x86_ 133/135 2024-02-13T14:08:19.544 INFO:teuthology.orchestra.run.smithi087.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 134/135 2024-02-13T14:08:19.620 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 134/135 2024-02-13T14:08:19.620 INFO:teuthology.orchestra.run.smithi087.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 135/135 2024-02-13T14:08:20.727 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 135/135 2024-02-13T14:08:20.727 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-02-13T14:08:20.727 INFO:teuthology.orchestra.run.smithi148.stdout:Upgraded: 2024-02-13T14:08:20.727 INFO:teuthology.orchestra.run.smithi148.stdout: librados2-2:19.0.0-1423.ge1b79d5a.el9.x86_64 2024-02-13T14:08:20.727 INFO:teuthology.orchestra.run.smithi148.stdout: librbd1-2:19.0.0-1423.ge1b79d5a.el9.x86_64 2024-02-13T14:08:20.727 INFO:teuthology.orchestra.run.smithi148.stdout:Installed: 2024-02-13T14:08:20.727 INFO:teuthology.orchestra.run.smithi148.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-02-13T14:08:20.728 INFO:teuthology.orchestra.run.smithi148.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-02-13T14:08:20.728 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-2:19.0.0-1423.ge1b79d5a.el9.x86_64 2024-02-13T14:08:20.728 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-base-2:19.0.0-1423.ge1b79d5a.el9.x86_64 2024-02-13T14:08:20.728 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-common-2:19.0.0-1423.ge1b79d5a.el9.x86_64 2024-02-13T14:08:20.728 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-fuse-2:19.0.0-1423.ge1b79d5a.el9.x86_64 2024-02-13T14:08:20.728 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-grafana-dashboards-2:19.0.0-1423.ge1b79d5a.el9.noarch 2024-02-13T14:08:20.728 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-immutable-object-cache-2:19.0.0-1423.ge1b79d5a.el9.x86_64 2024-02-13T14:08:20.728 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mds-2:19.0.0-1423.ge1b79d5a.el9.x86_64 2024-02-13T14:08:20.728 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-2:19.0.0-1423.ge1b79d5a.el9.x86_64 2024-02-13T14:08:20.728 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-cephadm-2:19.0.0-1423.ge1b79d5a.el9.noarch 2024-02-13T14:08:20.728 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-dashboard-2:19.0.0-1423.ge1b79d5a.el9.noarch 2024-02-13T14:08:20.728 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1423.ge1b79d5a.el9.noarch 2024-02-13T14:08:20.729 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-modules-core-2:19.0.0-1423.ge1b79d5a.el9.noarch 2024-02-13T14:08:20.729 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-rook-2:19.0.0-1423.ge1b79d5a.el9.noarch 2024-02-13T14:08:20.729 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mon-2:19.0.0-1423.ge1b79d5a.el9.x86_64 2024-02-13T14:08:20.729 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-osd-2:19.0.0-1423.ge1b79d5a.el9.x86_64 2024-02-13T14:08:20.729 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-prometheus-alerts-2:19.0.0-1423.ge1b79d5a.el9.noarch 2024-02-13T14:08:20.729 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-radosgw-2:19.0.0-1423.ge1b79d5a.el9.x86_64 2024-02-13T14:08:20.729 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-selinux-2:19.0.0-1423.ge1b79d5a.el9.x86_64 2024-02-13T14:08:20.729 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-test-2:19.0.0-1423.ge1b79d5a.el9.x86_64 2024-02-13T14:08:20.729 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-volume-2:19.0.0-1423.ge1b79d5a.el9.noarch 2024-02-13T14:08:20.729 INFO:teuthology.orchestra.run.smithi148.stdout: cephadm-2:19.0.0-1423.ge1b79d5a.el9.noarch 2024-02-13T14:08:20.729 INFO:teuthology.orchestra.run.smithi148.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-02-13T14:08:20.729 INFO:teuthology.orchestra.run.smithi148.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-02-13T14:08:20.729 INFO:teuthology.orchestra.run.smithi148.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-02-13T14:08:20.730 INFO:teuthology.orchestra.run.smithi148.stdout: fuse-2.9.9-15.el9.x86_64 2024-02-13T14:08:20.730 INFO:teuthology.orchestra.run.smithi148.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-02-13T14:08:20.730 INFO:teuthology.orchestra.run.smithi148.stdout: grpc-data-1.46.7-10.el9.noarch 2024-02-13T14:08:20.730 INFO:teuthology.orchestra.run.smithi148.stdout: jq-1.6-16.el9.x86_64 2024-02-13T14:08:20.730 INFO:teuthology.orchestra.run.smithi148.stdout: libarrow-9.0.0-10.el9.x86_64 2024-02-13T14:08:20.730 INFO:teuthology.orchestra.run.smithi148.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-02-13T14:08:20.730 INFO:teuthology.orchestra.run.smithi148.stdout: libcephfs-devel-2:19.0.0-1423.ge1b79d5a.el9.x86_64 2024-02-13T14:08:20.730 INFO:teuthology.orchestra.run.smithi148.stdout: libcephfs2-2:19.0.0-1423.ge1b79d5a.el9.x86_64 2024-02-13T14:08:20.730 INFO:teuthology.orchestra.run.smithi148.stdout: libcephsqlite-2:19.0.0-1423.ge1b79d5a.el9.x86_64 2024-02-13T14:08:20.730 INFO:teuthology.orchestra.run.smithi148.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-02-13T14:08:20.730 INFO:teuthology.orchestra.run.smithi148.stdout: liboath-2.6.7-2.el9.x86_64 2024-02-13T14:08:20.730 INFO:teuthology.orchestra.run.smithi148.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-02-13T14:08:20.730 INFO:teuthology.orchestra.run.smithi148.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-02-13T14:08:20.730 INFO:teuthology.orchestra.run.smithi148.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-02-13T14:08:20.731 INFO:teuthology.orchestra.run.smithi148.stdout: librados-devel-2:19.0.0-1423.ge1b79d5a.el9.x86_64 2024-02-13T14:08:20.731 INFO:teuthology.orchestra.run.smithi148.stdout: libradosstriper1-2:19.0.0-1423.ge1b79d5a.el9.x86_64 2024-02-13T14:08:20.731 INFO:teuthology.orchestra.run.smithi148.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-02-13T14:08:20.731 INFO:teuthology.orchestra.run.smithi148.stdout: librgw2-2:19.0.0-1423.ge1b79d5a.el9.x86_64 2024-02-13T14:08:20.731 INFO:teuthology.orchestra.run.smithi148.stdout: libunwind-1.6.2-1.el9.x86_64 2024-02-13T14:08:20.731 INFO:teuthology.orchestra.run.smithi148.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-02-13T14:08:20.731 INFO:teuthology.orchestra.run.smithi148.stdout: lua-5.4.4-4.el9.x86_64 2024-02-13T14:08:20.731 INFO:teuthology.orchestra.run.smithi148.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-02-13T14:08:20.731 INFO:teuthology.orchestra.run.smithi148.stdout: luarocks-3.9.2-1.el9.noarch 2024-02-13T14:08:20.731 INFO:teuthology.orchestra.run.smithi148.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-02-13T14:08:20.731 INFO:teuthology.orchestra.run.smithi148.stdout: openblas-0.3.21-2.el9.x86_64 2024-02-13T14:08:20.731 INFO:teuthology.orchestra.run.smithi148.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-02-13T14:08:20.732 INFO:teuthology.orchestra.run.smithi148.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-02-13T14:08:20.732 INFO:teuthology.orchestra.run.smithi148.stdout: protobuf-3.14.0-13.el9.x86_64 2024-02-13T14:08:20.732 INFO:teuthology.orchestra.run.smithi148.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-02-13T14:08:20.732 INFO:teuthology.orchestra.run.smithi148.stdout: python3-asyncssh-2.13.2-3.el9.noarch 2024-02-13T14:08:20.732 INFO:teuthology.orchestra.run.smithi148.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-02-13T14:08:20.732 INFO:teuthology.orchestra.run.smithi148.stdout: python3-babel-2.9.1-2.el9.noarch 2024-02-13T14:08:20.732 INFO:teuthology.orchestra.run.smithi148.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-02-13T14:08:20.732 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-02-13T14:08:20.732 INFO:teuthology.orchestra.run.smithi148.stdout: python3-ceph-argparse-2:19.0.0-1423.ge1b79d5a.el9.x86_64 2024-02-13T14:08:20.732 INFO:teuthology.orchestra.run.smithi148.stdout: python3-ceph-common-2:19.0.0-1423.ge1b79d5a.el9.x86_64 2024-02-13T14:08:20.732 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cephfs-2:19.0.0-1423.ge1b79d5a.el9.x86_64 2024-02-13T14:08:20.732 INFO:teuthology.orchestra.run.smithi148.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-02-13T14:08:20.732 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-02-13T14:08:20.733 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-02-13T14:08:20.733 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-02-13T14:08:20.733 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-02-13T14:08:20.733 INFO:teuthology.orchestra.run.smithi148.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-02-13T14:08:20.733 INFO:teuthology.orchestra.run.smithi148.stdout: python3-google-auth-1:2.27.0-1.el9.noarch 2024-02-13T14:08:20.733 INFO:teuthology.orchestra.run.smithi148.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-02-13T14:08:20.733 INFO:teuthology.orchestra.run.smithi148.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-02-13T14:08:20.733 INFO:teuthology.orchestra.run.smithi148.stdout: python3-iniconfig-1.1.1-7.el9.noarch 2024-02-13T14:08:20.733 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-02-13T14:08:20.733 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-02-13T14:08:20.734 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-02-13T14:08:20.734 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-02-13T14:08:20.734 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-02-13T14:08:20.734 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-02-13T14:08:20.734 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-02-13T14:08:20.734 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-02-13T14:08:20.734 INFO:teuthology.orchestra.run.smithi148.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-02-13T14:08:20.734 INFO:teuthology.orchestra.run.smithi148.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-02-13T14:08:20.734 INFO:teuthology.orchestra.run.smithi148.stdout: python3-mako-1.1.4-6.el9.noarch 2024-02-13T14:08:20.734 INFO:teuthology.orchestra.run.smithi148.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-02-13T14:08:20.734 INFO:teuthology.orchestra.run.smithi148.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-02-13T14:08:20.734 INFO:teuthology.orchestra.run.smithi148.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-02-13T14:08:20.734 INFO:teuthology.orchestra.run.smithi148.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-02-13T14:08:20.735 INFO:teuthology.orchestra.run.smithi148.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-02-13T14:08:20.735 INFO:teuthology.orchestra.run.smithi148.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-02-13T14:08:20.735 INFO:teuthology.orchestra.run.smithi148.stdout: python3-packaging-20.9-5.el9.noarch 2024-02-13T14:08:20.735 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-02-13T14:08:20.735 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pluggy-0.13.1-7.el9.noarch 2024-02-13T14:08:20.735 INFO:teuthology.orchestra.run.smithi148.stdout: python3-ply-3.11-14.el9.noarch 2024-02-13T14:08:20.735 INFO:teuthology.orchestra.run.smithi148.stdout: python3-portend-3.1.0-2.el9.noarch 2024-02-13T14:08:20.735 INFO:teuthology.orchestra.run.smithi148.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-02-13T14:08:20.735 INFO:teuthology.orchestra.run.smithi148.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-02-13T14:08:20.735 INFO:teuthology.orchestra.run.smithi148.stdout: python3-py-1.10.0-6.el9.noarch 2024-02-13T14:08:20.735 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-02-13T14:08:20.735 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-02-13T14:08:20.736 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-02-13T14:08:20.736 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pycparser-2.20-6.el9.noarch 2024-02-13T14:08:20.736 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pytest-6.2.2-6.el9.noarch 2024-02-13T14:08:20.736 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pytz-2021.1-5.el9.noarch 2024-02-13T14:08:20.736 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rados-2:19.0.0-1423.ge1b79d5a.el9.x86_64 2024-02-13T14:08:20.736 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rbd-2:19.0.0-1423.ge1b79d5a.el9.x86_64 2024-02-13T14:08:20.736 INFO:teuthology.orchestra.run.smithi148.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-02-13T14:08:20.736 INFO:teuthology.orchestra.run.smithi148.stdout: python3-requests-2.25.1-8.el9.noarch 2024-02-13T14:08:20.736 INFO:teuthology.orchestra.run.smithi148.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-02-13T14:08:20.737 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rgw-2:19.0.0-1423.ge1b79d5a.el9.x86_64 2024-02-13T14:08:20.737 INFO:teuthology.orchestra.run.smithi148.stdout: python3-routes-2.5.1-5.el9.noarch 2024-02-13T14:08:20.737 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rsa-4.9-2.el9.noarch 2024-02-13T14:08:20.737 INFO:teuthology.orchestra.run.smithi148.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-02-13T14:08:20.737 INFO:teuthology.orchestra.run.smithi148.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-02-13T14:08:20.737 INFO:teuthology.orchestra.run.smithi148.stdout: python3-toml-0.10.2-6.el9.noarch 2024-02-13T14:08:20.737 INFO:teuthology.orchestra.run.smithi148.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-02-13T14:08:20.737 INFO:teuthology.orchestra.run.smithi148.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-02-13T14:08:20.738 INFO:teuthology.orchestra.run.smithi148.stdout: python3-webob-1.8.7-6.el9.noarch 2024-02-13T14:08:20.738 INFO:teuthology.orchestra.run.smithi148.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-02-13T14:08:20.738 INFO:teuthology.orchestra.run.smithi148.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-02-13T14:08:20.738 INFO:teuthology.orchestra.run.smithi148.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-02-13T14:08:20.738 INFO:teuthology.orchestra.run.smithi148.stdout: qatlib-23.11.0-1.el9.x86_64 2024-02-13T14:08:20.738 INFO:teuthology.orchestra.run.smithi148.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-02-13T14:08:20.738 INFO:teuthology.orchestra.run.smithi148.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-02-13T14:08:20.738 INFO:teuthology.orchestra.run.smithi148.stdout: rbd-fuse-2:19.0.0-1423.ge1b79d5a.el9.x86_64 2024-02-13T14:08:20.738 INFO:teuthology.orchestra.run.smithi148.stdout: rbd-mirror-2:19.0.0-1423.ge1b79d5a.el9.x86_64 2024-02-13T14:08:20.738 INFO:teuthology.orchestra.run.smithi148.stdout: rbd-nbd-2:19.0.0-1423.ge1b79d5a.el9.x86_64 2024-02-13T14:08:20.739 INFO:teuthology.orchestra.run.smithi148.stdout: re2-1:20211101-3.el9.x86_64 2024-02-13T14:08:20.739 INFO:teuthology.orchestra.run.smithi148.stdout: socat-1.7.4.1-5.el9.x86_64 2024-02-13T14:08:20.739 INFO:teuthology.orchestra.run.smithi148.stdout: thrift-0.15.0-2.el9.x86_64 2024-02-13T14:08:20.740 INFO:teuthology.orchestra.run.smithi148.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-02-13T14:08:20.740 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-02-13T14:08:20.740 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-02-13T14:08:21.026 DEBUG:teuthology.parallel:result is None 2024-02-13T14:08:24.179 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 135/135 2024-02-13T14:08:24.180 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-2:19.0.0-1423.ge1b79d5a.el9.x86_64 1/135 2024-02-13T14:08:24.180 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-base-2:19.0.0-1423.ge1b79d5a.el9.x86_64 2/135 2024-02-13T14:08:24.180 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-common-2:19.0.0-1423.ge1b79d5a.el9.x86_64 3/135 2024-02-13T14:08:24.180 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-fuse-2:19.0.0-1423.ge1b79d5a.el9.x86_64 4/135 2024-02-13T14:08:24.180 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1423.ge1b79 5/135 2024-02-13T14:08:24.180 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-mds-2:19.0.0-1423.ge1b79d5a.el9.x86_64 6/135 2024-02-13T14:08:24.181 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-mgr-2:19.0.0-1423.ge1b79d5a.el9.x86_64 7/135 2024-02-13T14:08:24.181 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-mon-2:19.0.0-1423.ge1b79d5a.el9.x86_64 8/135 2024-02-13T14:08:24.181 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-osd-2:19.0.0-1423.ge1b79d5a.el9.x86_64 9/135 2024-02-13T14:08:24.181 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-radosgw-2:19.0.0-1423.ge1b79d5a.el9.x86_64 10/135 2024-02-13T14:08:24.181 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-selinux-2:19.0.0-1423.ge1b79d5a.el9.x86_64 11/135 2024-02-13T14:08:24.181 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-test-2:19.0.0-1423.ge1b79d5a.el9.x86_64 12/135 2024-02-13T14:08:24.182 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : libcephfs-devel-2:19.0.0-1423.ge1b79d5a.el9.x86_ 13/135 2024-02-13T14:08:24.182 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : libcephfs2-2:19.0.0-1423.ge1b79d5a.el9.x86_64 14/135 2024-02-13T14:08:24.182 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : libcephsqlite-2:19.0.0-1423.ge1b79d5a.el9.x86_64 15/135 2024-02-13T14:08:24.182 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : librados-devel-2:19.0.0-1423.ge1b79d5a.el9.x86_6 16/135 2024-02-13T14:08:24.182 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : libradosstriper1-2:19.0.0-1423.ge1b79d5a.el9.x86 17/135 2024-02-13T14:08:24.183 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : librgw2-2:19.0.0-1423.ge1b79d5a.el9.x86_64 18/135 2024-02-13T14:08:24.183 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1423.ge1b79d5a.el 19/135 2024-02-13T14:08:24.183 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-ceph-common-2:19.0.0-1423.ge1b79d5a.el9. 20/135 2024-02-13T14:08:24.183 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-cephfs-2:19.0.0-1423.ge1b79d5a.el9.x86_6 21/135 2024-02-13T14:08:24.183 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-rados-2:19.0.0-1423.ge1b79d5a.el9.x86_64 22/135 2024-02-13T14:08:24.183 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-rbd-2:19.0.0-1423.ge1b79d5a.el9.x86_64 23/135 2024-02-13T14:08:24.184 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-rgw-2:19.0.0-1423.ge1b79d5a.el9.x86_64 24/135 2024-02-13T14:08:24.184 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : rbd-fuse-2:19.0.0-1423.ge1b79d5a.el9.x86_64 25/135 2024-02-13T14:08:24.184 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : rbd-mirror-2:19.0.0-1423.ge1b79d5a.el9.x86_64 26/135 2024-02-13T14:08:24.184 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : rbd-nbd-2:19.0.0-1423.ge1b79d5a.el9.x86_64 27/135 2024-02-13T14:08:24.184 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1423.ge1b79d5a. 28/135 2024-02-13T14:08:24.184 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1423.ge1b79d5a.el9.noa 29/135 2024-02-13T14:08:24.185 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1423.ge1b79d5a.el9.n 30/135 2024-02-13T14:08:24.185 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1423.ge1b 31/135 2024-02-13T14:08:24.185 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1423.ge1b79d5a.el 32/135 2024-02-13T14:08:24.185 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1423.ge1b79d5a.el9.noarch 33/135 2024-02-13T14:08:24.185 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1423.ge1b79d5a.e 34/135 2024-02-13T14:08:24.185 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-volume-2:19.0.0-1423.ge1b79d5a.el9.noarch 35/135 2024-02-13T14:08:24.186 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : cephadm-2:19.0.0-1423.ge1b79d5a.el9.noarch 36/135 2024-02-13T14:08:24.186 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 37/135 2024-02-13T14:08:24.186 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 38/135 2024-02-13T14:08:24.186 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 39/135 2024-02-13T14:08:24.186 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 40/135 2024-02-13T14:08:24.186 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 41/135 2024-02-13T14:08:24.187 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 42/135 2024-02-13T14:08:24.187 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 43/135 2024-02-13T14:08:24.187 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 44/135 2024-02-13T14:08:24.187 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : lua-5.4.4-4.el9.x86_64 45/135 2024-02-13T14:08:24.187 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 46/135 2024-02-13T14:08:24.188 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 47/135 2024-02-13T14:08:24.188 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 48/135 2024-02-13T14:08:24.188 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 49/135 2024-02-13T14:08:24.188 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 50/135 2024-02-13T14:08:24.188 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-iniconfig-1.1.1-7.el9.noarch 51/135 2024-02-13T14:08:24.188 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 52/135 2024-02-13T14:08:24.189 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 53/135 2024-02-13T14:08:24.189 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 54/135 2024-02-13T14:08:24.189 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 55/135 2024-02-13T14:08:24.189 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 56/135 2024-02-13T14:08:24.189 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 57/135 2024-02-13T14:08:24.190 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 58/135 2024-02-13T14:08:24.190 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 59/135 2024-02-13T14:08:24.190 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 60/135 2024-02-13T14:08:24.190 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-pluggy-0.13.1-7.el9.noarch 61/135 2024-02-13T14:08:24.191 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 62/135 2024-02-13T14:08:24.191 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 63/135 2024-02-13T14:08:24.191 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-py-1.10.0-6.el9.noarch 64/135 2024-02-13T14:08:24.191 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 65/135 2024-02-13T14:08:24.191 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 66/135 2024-02-13T14:08:24.191 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-pytest-6.2.2-6.el9.noarch 67/135 2024-02-13T14:08:24.192 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 68/135 2024-02-13T14:08:24.192 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 69/135 2024-02-13T14:08:24.192 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 70/135 2024-02-13T14:08:24.192 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 71/135 2024-02-13T14:08:24.192 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 72/135 2024-02-13T14:08:24.192 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 73/135 2024-02-13T14:08:24.193 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 74/135 2024-02-13T14:08:24.193 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 75/135 2024-02-13T14:08:24.193 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 76/135 2024-02-13T14:08:24.193 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 77/135 2024-02-13T14:08:24.193 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : jq-1.6-16.el9.x86_64 78/135 2024-02-13T14:08:24.193 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 79/135 2024-02-13T14:08:24.194 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 80/135 2024-02-13T14:08:24.194 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 81/135 2024-02-13T14:08:24.194 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 82/135 2024-02-13T14:08:24.194 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 83/135 2024-02-13T14:08:24.194 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-ply-3.11-14.el9.noarch 84/135 2024-02-13T14:08:24.194 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 85/135 2024-02-13T14:08:24.195 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 86/135 2024-02-13T14:08:24.195 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 87/135 2024-02-13T14:08:24.195 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 88/135 2024-02-13T14:08:24.195 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 89/135 2024-02-13T14:08:24.195 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 90/135 2024-02-13T14:08:24.195 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 91/135 2024-02-13T14:08:24.195 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 92/135 2024-02-13T14:08:24.196 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 93/135 2024-02-13T14:08:24.196 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 94/135 2024-02-13T14:08:24.196 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 95/135 2024-02-13T14:08:24.196 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 96/135 2024-02-13T14:08:24.196 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 97/135 2024-02-13T14:08:24.196 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 98/135 2024-02-13T14:08:24.197 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 99/135 2024-02-13T14:08:24.197 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-asyncssh-2.13.2-3.el9.noarch 100/135 2024-02-13T14:08:24.197 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 101/135 2024-02-13T14:08:24.197 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 102/135 2024-02-13T14:08:24.197 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 103/135 2024-02-13T14:08:24.198 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 104/135 2024-02-13T14:08:24.198 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 105/135 2024-02-13T14:08:24.198 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-google-auth-1:2.27.0-1.el9.noarch 106/135 2024-02-13T14:08:24.198 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 107/135 2024-02-13T14:08:24.198 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 108/135 2024-02-13T14:08:24.198 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 109/135 2024-02-13T14:08:24.198 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 110/135 2024-02-13T14:08:24.198 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 111/135 2024-02-13T14:08:24.199 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 112/135 2024-02-13T14:08:24.199 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 113/135 2024-02-13T14:08:24.199 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 114/135 2024-02-13T14:08:24.199 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 115/135 2024-02-13T14:08:24.199 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 116/135 2024-02-13T14:08:24.199 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 117/135 2024-02-13T14:08:24.199 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 118/135 2024-02-13T14:08:24.199 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 119/135 2024-02-13T14:08:24.204 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 120/135 2024-02-13T14:08:24.205 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 121/135 2024-02-13T14:08:24.205 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 122/135 2024-02-13T14:08:24.205 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 123/135 2024-02-13T14:08:24.205 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 124/135 2024-02-13T14:08:24.205 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 125/135 2024-02-13T14:08:24.205 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 126/135 2024-02-13T14:08:24.205 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 127/135 2024-02-13T14:08:24.205 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 128/135 2024-02-13T14:08:24.205 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 129/135 2024-02-13T14:08:24.205 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : re2-1:20211101-3.el9.x86_64 130/135 2024-02-13T14:08:24.205 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 131/135 2024-02-13T14:08:24.205 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : librados2-2:19.0.0-1423.ge1b79d5a.el9.x86_64 132/135 2024-02-13T14:08:24.206 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 133/135 2024-02-13T14:08:24.206 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : librbd1-2:19.0.0-1423.ge1b79d5a.el9.x86_64 134/135 2024-02-13T14:08:26.317 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 135/135 2024-02-13T14:08:26.317 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-02-13T14:08:26.318 INFO:teuthology.orchestra.run.smithi087.stdout:Upgraded: 2024-02-13T14:08:26.318 INFO:teuthology.orchestra.run.smithi087.stdout: librados2-2:19.0.0-1423.ge1b79d5a.el9.x86_64 2024-02-13T14:08:26.318 INFO:teuthology.orchestra.run.smithi087.stdout: librbd1-2:19.0.0-1423.ge1b79d5a.el9.x86_64 2024-02-13T14:08:26.318 INFO:teuthology.orchestra.run.smithi087.stdout:Installed: 2024-02-13T14:08:26.318 INFO:teuthology.orchestra.run.smithi087.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-02-13T14:08:26.318 INFO:teuthology.orchestra.run.smithi087.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-02-13T14:08:26.318 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-2:19.0.0-1423.ge1b79d5a.el9.x86_64 2024-02-13T14:08:26.318 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-base-2:19.0.0-1423.ge1b79d5a.el9.x86_64 2024-02-13T14:08:26.318 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-common-2:19.0.0-1423.ge1b79d5a.el9.x86_64 2024-02-13T14:08:26.318 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-fuse-2:19.0.0-1423.ge1b79d5a.el9.x86_64 2024-02-13T14:08:26.318 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-grafana-dashboards-2:19.0.0-1423.ge1b79d5a.el9.noarch 2024-02-13T14:08:26.318 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-immutable-object-cache-2:19.0.0-1423.ge1b79d5a.el9.x86_64 2024-02-13T14:08:26.318 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mds-2:19.0.0-1423.ge1b79d5a.el9.x86_64 2024-02-13T14:08:26.319 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mgr-2:19.0.0-1423.ge1b79d5a.el9.x86_64 2024-02-13T14:08:26.319 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mgr-cephadm-2:19.0.0-1423.ge1b79d5a.el9.noarch 2024-02-13T14:08:26.319 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mgr-dashboard-2:19.0.0-1423.ge1b79d5a.el9.noarch 2024-02-13T14:08:26.319 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1423.ge1b79d5a.el9.noarch 2024-02-13T14:08:26.319 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mgr-modules-core-2:19.0.0-1423.ge1b79d5a.el9.noarch 2024-02-13T14:08:26.319 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mgr-rook-2:19.0.0-1423.ge1b79d5a.el9.noarch 2024-02-13T14:08:26.319 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mon-2:19.0.0-1423.ge1b79d5a.el9.x86_64 2024-02-13T14:08:26.319 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-osd-2:19.0.0-1423.ge1b79d5a.el9.x86_64 2024-02-13T14:08:26.319 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-prometheus-alerts-2:19.0.0-1423.ge1b79d5a.el9.noarch 2024-02-13T14:08:26.319 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-radosgw-2:19.0.0-1423.ge1b79d5a.el9.x86_64 2024-02-13T14:08:26.319 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-selinux-2:19.0.0-1423.ge1b79d5a.el9.x86_64 2024-02-13T14:08:26.319 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-test-2:19.0.0-1423.ge1b79d5a.el9.x86_64 2024-02-13T14:08:26.319 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-volume-2:19.0.0-1423.ge1b79d5a.el9.noarch 2024-02-13T14:08:26.319 INFO:teuthology.orchestra.run.smithi087.stdout: cephadm-2:19.0.0-1423.ge1b79d5a.el9.noarch 2024-02-13T14:08:26.320 INFO:teuthology.orchestra.run.smithi087.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-02-13T14:08:26.320 INFO:teuthology.orchestra.run.smithi087.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-02-13T14:08:26.320 INFO:teuthology.orchestra.run.smithi087.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-02-13T14:08:26.320 INFO:teuthology.orchestra.run.smithi087.stdout: fuse-2.9.9-15.el9.x86_64 2024-02-13T14:08:26.320 INFO:teuthology.orchestra.run.smithi087.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-02-13T14:08:26.320 INFO:teuthology.orchestra.run.smithi087.stdout: grpc-data-1.46.7-10.el9.noarch 2024-02-13T14:08:26.320 INFO:teuthology.orchestra.run.smithi087.stdout: jq-1.6-16.el9.x86_64 2024-02-13T14:08:26.320 INFO:teuthology.orchestra.run.smithi087.stdout: libarrow-9.0.0-10.el9.x86_64 2024-02-13T14:08:26.320 INFO:teuthology.orchestra.run.smithi087.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-02-13T14:08:26.320 INFO:teuthology.orchestra.run.smithi087.stdout: libcephfs-devel-2:19.0.0-1423.ge1b79d5a.el9.x86_64 2024-02-13T14:08:26.320 INFO:teuthology.orchestra.run.smithi087.stdout: libcephfs2-2:19.0.0-1423.ge1b79d5a.el9.x86_64 2024-02-13T14:08:26.320 INFO:teuthology.orchestra.run.smithi087.stdout: libcephsqlite-2:19.0.0-1423.ge1b79d5a.el9.x86_64 2024-02-13T14:08:26.320 INFO:teuthology.orchestra.run.smithi087.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-02-13T14:08:26.321 INFO:teuthology.orchestra.run.smithi087.stdout: liboath-2.6.7-2.el9.x86_64 2024-02-13T14:08:26.321 INFO:teuthology.orchestra.run.smithi087.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-02-13T14:08:26.321 INFO:teuthology.orchestra.run.smithi087.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-02-13T14:08:26.321 INFO:teuthology.orchestra.run.smithi087.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-02-13T14:08:26.321 INFO:teuthology.orchestra.run.smithi087.stdout: librados-devel-2:19.0.0-1423.ge1b79d5a.el9.x86_64 2024-02-13T14:08:26.321 INFO:teuthology.orchestra.run.smithi087.stdout: libradosstriper1-2:19.0.0-1423.ge1b79d5a.el9.x86_64 2024-02-13T14:08:26.321 INFO:teuthology.orchestra.run.smithi087.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-02-13T14:08:26.321 INFO:teuthology.orchestra.run.smithi087.stdout: librgw2-2:19.0.0-1423.ge1b79d5a.el9.x86_64 2024-02-13T14:08:26.321 INFO:teuthology.orchestra.run.smithi087.stdout: libunwind-1.6.2-1.el9.x86_64 2024-02-13T14:08:26.321 INFO:teuthology.orchestra.run.smithi087.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-02-13T14:08:26.321 INFO:teuthology.orchestra.run.smithi087.stdout: lua-5.4.4-4.el9.x86_64 2024-02-13T14:08:26.321 INFO:teuthology.orchestra.run.smithi087.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-02-13T14:08:26.321 INFO:teuthology.orchestra.run.smithi087.stdout: luarocks-3.9.2-1.el9.noarch 2024-02-13T14:08:26.321 INFO:teuthology.orchestra.run.smithi087.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-02-13T14:08:26.322 INFO:teuthology.orchestra.run.smithi087.stdout: openblas-0.3.21-2.el9.x86_64 2024-02-13T14:08:26.322 INFO:teuthology.orchestra.run.smithi087.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-02-13T14:08:26.322 INFO:teuthology.orchestra.run.smithi087.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-02-13T14:08:26.322 INFO:teuthology.orchestra.run.smithi087.stdout: protobuf-3.14.0-13.el9.x86_64 2024-02-13T14:08:26.322 INFO:teuthology.orchestra.run.smithi087.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-02-13T14:08:26.322 INFO:teuthology.orchestra.run.smithi087.stdout: python3-asyncssh-2.13.2-3.el9.noarch 2024-02-13T14:08:26.322 INFO:teuthology.orchestra.run.smithi087.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-02-13T14:08:26.322 INFO:teuthology.orchestra.run.smithi087.stdout: python3-babel-2.9.1-2.el9.noarch 2024-02-13T14:08:26.322 INFO:teuthology.orchestra.run.smithi087.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-02-13T14:08:26.322 INFO:teuthology.orchestra.run.smithi087.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-02-13T14:08:26.322 INFO:teuthology.orchestra.run.smithi087.stdout: python3-ceph-argparse-2:19.0.0-1423.ge1b79d5a.el9.x86_64 2024-02-13T14:08:26.322 INFO:teuthology.orchestra.run.smithi087.stdout: python3-ceph-common-2:19.0.0-1423.ge1b79d5a.el9.x86_64 2024-02-13T14:08:26.322 INFO:teuthology.orchestra.run.smithi087.stdout: python3-cephfs-2:19.0.0-1423.ge1b79d5a.el9.x86_64 2024-02-13T14:08:26.322 INFO:teuthology.orchestra.run.smithi087.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-02-13T14:08:26.323 INFO:teuthology.orchestra.run.smithi087.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-02-13T14:08:26.323 INFO:teuthology.orchestra.run.smithi087.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-02-13T14:08:26.323 INFO:teuthology.orchestra.run.smithi087.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-02-13T14:08:26.323 INFO:teuthology.orchestra.run.smithi087.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-02-13T14:08:26.323 INFO:teuthology.orchestra.run.smithi087.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-02-13T14:08:26.323 INFO:teuthology.orchestra.run.smithi087.stdout: python3-google-auth-1:2.27.0-1.el9.noarch 2024-02-13T14:08:26.323 INFO:teuthology.orchestra.run.smithi087.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-02-13T14:08:26.323 INFO:teuthology.orchestra.run.smithi087.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-02-13T14:08:26.323 INFO:teuthology.orchestra.run.smithi087.stdout: python3-iniconfig-1.1.1-7.el9.noarch 2024-02-13T14:08:26.323 INFO:teuthology.orchestra.run.smithi087.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-02-13T14:08:26.323 INFO:teuthology.orchestra.run.smithi087.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-02-13T14:08:26.323 INFO:teuthology.orchestra.run.smithi087.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-02-13T14:08:26.323 INFO:teuthology.orchestra.run.smithi087.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-02-13T14:08:26.323 INFO:teuthology.orchestra.run.smithi087.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-02-13T14:08:26.324 INFO:teuthology.orchestra.run.smithi087.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-02-13T14:08:26.324 INFO:teuthology.orchestra.run.smithi087.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-02-13T14:08:26.324 INFO:teuthology.orchestra.run.smithi087.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-02-13T14:08:26.324 INFO:teuthology.orchestra.run.smithi087.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-02-13T14:08:26.324 INFO:teuthology.orchestra.run.smithi087.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-02-13T14:08:26.324 INFO:teuthology.orchestra.run.smithi087.stdout: python3-mako-1.1.4-6.el9.noarch 2024-02-13T14:08:26.324 INFO:teuthology.orchestra.run.smithi087.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-02-13T14:08:26.324 INFO:teuthology.orchestra.run.smithi087.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-02-13T14:08:26.324 INFO:teuthology.orchestra.run.smithi087.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-02-13T14:08:26.324 INFO:teuthology.orchestra.run.smithi087.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-02-13T14:08:26.324 INFO:teuthology.orchestra.run.smithi087.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-02-13T14:08:26.324 INFO:teuthology.orchestra.run.smithi087.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-02-13T14:08:26.324 INFO:teuthology.orchestra.run.smithi087.stdout: python3-packaging-20.9-5.el9.noarch 2024-02-13T14:08:26.324 INFO:teuthology.orchestra.run.smithi087.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-02-13T14:08:26.325 INFO:teuthology.orchestra.run.smithi087.stdout: python3-pluggy-0.13.1-7.el9.noarch 2024-02-13T14:08:26.325 INFO:teuthology.orchestra.run.smithi087.stdout: python3-ply-3.11-14.el9.noarch 2024-02-13T14:08:26.325 INFO:teuthology.orchestra.run.smithi087.stdout: python3-portend-3.1.0-2.el9.noarch 2024-02-13T14:08:26.325 INFO:teuthology.orchestra.run.smithi087.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-02-13T14:08:26.325 INFO:teuthology.orchestra.run.smithi087.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-02-13T14:08:26.325 INFO:teuthology.orchestra.run.smithi087.stdout: python3-py-1.10.0-6.el9.noarch 2024-02-13T14:08:26.325 INFO:teuthology.orchestra.run.smithi087.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-02-13T14:08:26.325 INFO:teuthology.orchestra.run.smithi087.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-02-13T14:08:26.325 INFO:teuthology.orchestra.run.smithi087.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-02-13T14:08:26.325 INFO:teuthology.orchestra.run.smithi087.stdout: python3-pycparser-2.20-6.el9.noarch 2024-02-13T14:08:26.325 INFO:teuthology.orchestra.run.smithi087.stdout: python3-pytest-6.2.2-6.el9.noarch 2024-02-13T14:08:26.325 INFO:teuthology.orchestra.run.smithi087.stdout: python3-pytz-2021.1-5.el9.noarch 2024-02-13T14:08:26.325 INFO:teuthology.orchestra.run.smithi087.stdout: python3-rados-2:19.0.0-1423.ge1b79d5a.el9.x86_64 2024-02-13T14:08:26.325 INFO:teuthology.orchestra.run.smithi087.stdout: python3-rbd-2:19.0.0-1423.ge1b79d5a.el9.x86_64 2024-02-13T14:08:26.326 INFO:teuthology.orchestra.run.smithi087.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-02-13T14:08:26.326 INFO:teuthology.orchestra.run.smithi087.stdout: python3-requests-2.25.1-8.el9.noarch 2024-02-13T14:08:26.326 INFO:teuthology.orchestra.run.smithi087.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-02-13T14:08:26.326 INFO:teuthology.orchestra.run.smithi087.stdout: python3-rgw-2:19.0.0-1423.ge1b79d5a.el9.x86_64 2024-02-13T14:08:26.326 INFO:teuthology.orchestra.run.smithi087.stdout: python3-routes-2.5.1-5.el9.noarch 2024-02-13T14:08:26.326 INFO:teuthology.orchestra.run.smithi087.stdout: python3-rsa-4.9-2.el9.noarch 2024-02-13T14:08:26.326 INFO:teuthology.orchestra.run.smithi087.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-02-13T14:08:26.326 INFO:teuthology.orchestra.run.smithi087.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-02-13T14:08:26.326 INFO:teuthology.orchestra.run.smithi087.stdout: python3-toml-0.10.2-6.el9.noarch 2024-02-13T14:08:26.326 INFO:teuthology.orchestra.run.smithi087.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-02-13T14:08:26.326 INFO:teuthology.orchestra.run.smithi087.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-02-13T14:08:26.326 INFO:teuthology.orchestra.run.smithi087.stdout: python3-webob-1.8.7-6.el9.noarch 2024-02-13T14:08:26.326 INFO:teuthology.orchestra.run.smithi087.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-02-13T14:08:26.326 INFO:teuthology.orchestra.run.smithi087.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-02-13T14:08:26.327 INFO:teuthology.orchestra.run.smithi087.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-02-13T14:08:26.327 INFO:teuthology.orchestra.run.smithi087.stdout: qatlib-23.11.0-1.el9.x86_64 2024-02-13T14:08:26.327 INFO:teuthology.orchestra.run.smithi087.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-02-13T14:08:26.327 INFO:teuthology.orchestra.run.smithi087.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-02-13T14:08:26.327 INFO:teuthology.orchestra.run.smithi087.stdout: rbd-fuse-2:19.0.0-1423.ge1b79d5a.el9.x86_64 2024-02-13T14:08:26.327 INFO:teuthology.orchestra.run.smithi087.stdout: rbd-mirror-2:19.0.0-1423.ge1b79d5a.el9.x86_64 2024-02-13T14:08:26.327 INFO:teuthology.orchestra.run.smithi087.stdout: rbd-nbd-2:19.0.0-1423.ge1b79d5a.el9.x86_64 2024-02-13T14:08:26.327 INFO:teuthology.orchestra.run.smithi087.stdout: re2-1:20211101-3.el9.x86_64 2024-02-13T14:08:26.327 INFO:teuthology.orchestra.run.smithi087.stdout: socat-1.7.4.1-5.el9.x86_64 2024-02-13T14:08:26.327 INFO:teuthology.orchestra.run.smithi087.stdout: thrift-0.15.0-2.el9.x86_64 2024-02-13T14:08:26.328 INFO:teuthology.orchestra.run.smithi087.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-02-13T14:08:26.328 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-02-13T14:08:26.328 INFO:teuthology.orchestra.run.smithi087.stdout:Complete! 2024-02-13T14:08:26.605 DEBUG:teuthology.parallel:result is None 2024-02-13T14:08:26.606 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=e1b79d5a4193a484176863ab96c0ca28c00e1662 2024-02-13T14:08:26.822 DEBUG:teuthology.orchestra.run.smithi087:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-02-13T14:08:26.844 INFO:teuthology.orchestra.run.smithi087.stdout:19.0.0-1423.ge1b79d5a.el9 2024-02-13T14:08:26.844 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1423.ge1b79d5a.el9 2024-02-13T14:08:26.844 INFO:teuthology.task.install:The correct ceph version 19.0.0-1423.ge1b79d5a is installed. 2024-02-13T14:08:26.846 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=e1b79d5a4193a484176863ab96c0ca28c00e1662 2024-02-13T14:08:27.061 DEBUG:teuthology.orchestra.run.smithi148:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-02-13T14:08:27.086 INFO:teuthology.orchestra.run.smithi148.stdout:19.0.0-1423.ge1b79d5a.el9 2024-02-13T14:08:27.086 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1423.ge1b79d5a.el9 2024-02-13T14:08:27.086 INFO:teuthology.task.install:The correct ceph version 19.0.0-1423.ge1b79d5a is installed. 2024-02-13T14:08:27.088 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-02-13T14:08:27.088 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-02-13T14:08:27.089 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-02-13T14:08:27.118 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-02-13T14:08:27.118 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-02-13T14:08:27.159 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-02-13T14:08:27.160 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-02-13T14:08:27.160 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd of=/usr/bin/daemon-helper 2024-02-13T14:08:27.188 DEBUG:teuthology.orchestra.run.smithi087:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-02-13T14:08:27.255 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-02-13T14:08:27.255 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/usr/bin/daemon-helper 2024-02-13T14:08:27.286 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-02-13T14:08:27.356 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-02-13T14:08:27.357 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-02-13T14:08:27.357 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd of=/usr/bin/adjust-ulimits 2024-02-13T14:08:27.383 DEBUG:teuthology.orchestra.run.smithi087:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-02-13T14:08:27.450 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-02-13T14:08:27.450 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/usr/bin/adjust-ulimits 2024-02-13T14:08:27.480 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-02-13T14:08:27.548 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-02-13T14:08:27.549 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-02-13T14:08:27.549 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd of=/usr/bin/stdin-killer 2024-02-13T14:08:27.575 DEBUG:teuthology.orchestra.run.smithi087:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-02-13T14:08:27.642 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-02-13T14:08:27.642 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/usr/bin/stdin-killer 2024-02-13T14:08:27.671 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-02-13T14:08:27.739 INFO:teuthology.run_tasks:Running task cephadm... 2024-02-13T14:08:27.872 INFO:tasks.cephadm:Config: {'conf': {'mgr': {'debug mgr': 20, 'debug ms': 1}, 'global': {'mon election default strategy': 3, 'ms type': 'async'}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'debug ms': 1, 'debug osd': 20, 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'but it is still running', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(PG_', '\\(OSD_', '\\(OBJECT_', '\\(POOL_APP_NOT_ENABLED\\)'], 'sha1': 'e1b79d5a4193a484176863ab96c0ca28c00e1662', 'cephadm_mode': 'root'} 2024-02-13T14:08:27.872 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662 2024-02-13T14:08:27.873 INFO:tasks.cephadm:Cluster fsid is 5c04b074-ca79-11ee-95b9-87774f69a715 2024-02-13T14:08:27.873 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-02-13T14:08:27.873 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.87', 'mon.c': '[v2:172.21.15.87:3301,v1:172.21.15.87:6790]', 'mon.b': '172.21.15.148'} 2024-02-13T14:08:27.873 INFO:tasks.cephadm:First mon is mon.a on smithi087 2024-02-13T14:08:27.873 INFO:tasks.cephadm:First mgr is y 2024-02-13T14:08:27.873 INFO:tasks.cephadm:Normalizing hostnames... 2024-02-13T14:08:27.873 DEBUG:teuthology.orchestra.run.smithi087:> sudo hostname $(hostname -s) 2024-02-13T14:08:27.901 DEBUG:teuthology.orchestra.run.smithi148:> sudo hostname $(hostname -s) 2024-02-13T14:08:27.928 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-02-13T14:08:27.929 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=e1b79d5a4193a484176863ab96c0ca28c00e1662 2024-02-13T14:08:28.065 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'e1b79d5a4193a484176863ab96c0ca28c00e1662', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/76678/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-1423-ge1b79d5a', 'node_name': '172.21.2.1+braggi01', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-1423.ge1b79d5a'}, 'url': 'https://4.chacra.ceph.com/r/ceph/wip-phlogistonjohn-testing-2024-02-12-1654/e1b79d5a4193a484176863ab96c0ca28c00e1662/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-02-12 23:59:26.532406', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-phlogistonjohn-testing-2024-02-12-1654', 'chacra_url': 'https://4.chacra.ceph.com/repos/ceph/wip-phlogistonjohn-testing-2024-02-12-1654/e1b79d5a4193a484176863ab96c0ca28c00e1662/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-02-13T14:08:28.201 INFO:tasks.util.chacra:got chacra host 4.chacra.ceph.com, ref wip-phlogistonjohn-testing-2024-02-12-1654, sha1 e1b79d5a4193a484176863ab96c0ca28c00e1662 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=e1b79d5a4193a484176863ab96c0ca28c00e1662 2024-02-13T14:08:28.203 INFO:tasks.cephadm:Discovered cachra url: https://4.chacra.ceph.com/binaries/ceph/wip-phlogistonjohn-testing-2024-02-12-1654/e1b79d5a4193a484176863ab96c0ca28c00e1662/centos/9/x86_64/flavors/default/cephadm 2024-02-13T14:08:28.203 DEBUG:teuthology.orchestra.run.smithi087:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-phlogistonjohn-testing-2024-02-12-1654/e1b79d5a4193a484176863ab96c0ca28c00e1662/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-02-13T14:08:28.881 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r--. 1 ubuntu ubuntu 779193 Feb 13 14:08 /home/ubuntu/cephtest/cephadm 2024-02-13T14:08:28.882 DEBUG:teuthology.orchestra.run.smithi148:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-phlogistonjohn-testing-2024-02-12-1654/e1b79d5a4193a484176863ab96c0ca28c00e1662/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-02-13T14:08:29.511 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r--. 1 ubuntu ubuntu 779193 Feb 13 14:08 /home/ubuntu/cephtest/cephadm 2024-02-13T14:08:29.511 DEBUG:teuthology.orchestra.run.smithi087:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-02-13T14:08:29.531 DEBUG:teuthology.orchestra.run.smithi148:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-02-13T14:08:29.575 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662 on all hosts... 2024-02-13T14:08:29.576 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662 pull 2024-02-13T14:08:29.579 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662 pull 2024-02-13T14:08:30.424 INFO:teuthology.orchestra.run.smithi087.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662... 2024-02-13T14:08:30.467 INFO:teuthology.orchestra.run.smithi148.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662... 2024-02-13T14:09:18.073 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-02-13T14:09:18.074 INFO:teuthology.orchestra.run.smithi087.stdout: "ceph_version": "ceph version 19.0.0-1423-ge1b79d5a (e1b79d5a4193a484176863ab96c0ca28c00e1662) squid (dev)", 2024-02-13T14:09:18.074 INFO:teuthology.orchestra.run.smithi087.stdout: "image_id": "28480620e5edd298d55c0e6673dcbad9ed23edd37269028a11063c1b8925676a", 2024-02-13T14:09:18.074 INFO:teuthology.orchestra.run.smithi087.stdout: "repo_digests": [ 2024-02-13T14:09:18.074 INFO:teuthology.orchestra.run.smithi087.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f" 2024-02-13T14:09:18.074 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-02-13T14:09:18.074 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-02-13T14:09:21.768 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-02-13T14:09:21.768 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_version": "ceph version 19.0.0-1423-ge1b79d5a (e1b79d5a4193a484176863ab96c0ca28c00e1662) squid (dev)", 2024-02-13T14:09:21.768 INFO:teuthology.orchestra.run.smithi148.stdout: "image_id": "28480620e5edd298d55c0e6673dcbad9ed23edd37269028a11063c1b8925676a", 2024-02-13T14:09:21.768 INFO:teuthology.orchestra.run.smithi148.stdout: "repo_digests": [ 2024-02-13T14:09:21.768 INFO:teuthology.orchestra.run.smithi148.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f" 2024-02-13T14:09:21.768 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-02-13T14:09:21.768 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-02-13T14:09:21.792 DEBUG:teuthology.orchestra.run.smithi087:> sudo mkdir -p /etc/ceph 2024-02-13T14:09:21.830 DEBUG:teuthology.orchestra.run.smithi148:> sudo mkdir -p /etc/ceph 2024-02-13T14:09:21.860 DEBUG:teuthology.orchestra.run.smithi087:> sudo chmod 777 /etc/ceph 2024-02-13T14:09:21.900 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod 777 /etc/ceph 2024-02-13T14:09:21.927 INFO:tasks.cephadm:Writing seed config... 2024-02-13T14:09:21.928 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-02-13T14:09:21.928 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-02-13T14:09:21.928 INFO:tasks.cephadm: override: [global] mon election default strategy = 3 2024-02-13T14:09:21.928 INFO:tasks.cephadm: override: [global] ms type = async 2024-02-13T14:09:21.929 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-02-13T14:09:21.929 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-02-13T14:09:21.929 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-02-13T14:09:21.929 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-02-13T14:09:21.929 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-02-13T14:09:21.929 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-02-13T14:09:21.929 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-02-13T14:09:21.929 DEBUG:teuthology.orchestra.run.smithi087:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-02-13T14:09:21.957 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 5c04b074-ca79-11ee-95b9-87774f69a715 mon election default strategy = 3 ms type = async [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true debug ms = 1 debug osd = 20 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true 2024-02-13T14:09:21.957 DEBUG:teuthology.orchestra.run.smithi087:mon.a> sudo journalctl -f -n 0 -u ceph-5c04b074-ca79-11ee-95b9-87774f69a715@mon.a.service 2024-02-13T14:09:22.000 DEBUG:teuthology.orchestra.run.smithi087:mgr.y> sudo journalctl -f -n 0 -u ceph-5c04b074-ca79-11ee-95b9-87774f69a715@mgr.y.service 2024-02-13T14:09:22.042 INFO:tasks.cephadm:Bootstrapping... 2024-02-13T14:09:22.042 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662 -v bootstrap --fsid 5c04b074-ca79-11ee-95b9-87774f69a715 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id y --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.87 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-02-13T14:09:22.236 INFO:teuthology.orchestra.run.smithi087.stdout:-------------------------------------------------------------------------------- 2024-02-13T14:09:22.236 INFO:teuthology.orchestra.run.smithi087.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662', '-v', 'bootstrap', '--fsid', '5c04b074-ca79-11ee-95b9-87774f69a715', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'y', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.87', '--skip-admin-label'] 2024-02-13T14:09:22.261 INFO:teuthology.orchestra.run.smithi087.stdout:/bin/podman: stdout 4.9.3-dev 2024-02-13T14:09:22.261 INFO:teuthology.orchestra.run.smithi087.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-02-13T14:09:22.262 INFO:teuthology.orchestra.run.smithi087.stdout:Verifying podman|docker is present... 2024-02-13T14:09:22.287 INFO:teuthology.orchestra.run.smithi087.stdout:/bin/podman: stdout 4.9.3-dev 2024-02-13T14:09:22.287 INFO:teuthology.orchestra.run.smithi087.stdout:Verifying lvm2 is present... 2024-02-13T14:09:22.287 INFO:teuthology.orchestra.run.smithi087.stdout:Verifying time synchronization is in place... 2024-02-13T14:09:22.294 INFO:teuthology.orchestra.run.smithi087.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-02-13T14:09:22.294 INFO:teuthology.orchestra.run.smithi087.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-02-13T14:09:22.300 INFO:teuthology.orchestra.run.smithi087.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-02-13T14:09:22.300 INFO:teuthology.orchestra.run.smithi087.stdout:systemctl: stdout inactive 2024-02-13T14:09:22.306 INFO:teuthology.orchestra.run.smithi087.stdout:systemctl: stdout enabled 2024-02-13T14:09:22.312 INFO:teuthology.orchestra.run.smithi087.stdout:systemctl: stdout active 2024-02-13T14:09:22.312 INFO:teuthology.orchestra.run.smithi087.stdout:Unit chronyd.service is enabled and running 2024-02-13T14:09:22.312 INFO:teuthology.orchestra.run.smithi087.stdout:Repeating the final host check... 2024-02-13T14:09:22.335 INFO:teuthology.orchestra.run.smithi087.stdout:/bin/podman: stdout 4.9.3-dev 2024-02-13T14:09:22.335 INFO:teuthology.orchestra.run.smithi087.stdout:podman (/bin/podman) version 4.9.3 is present 2024-02-13T14:09:22.335 INFO:teuthology.orchestra.run.smithi087.stdout:systemctl is present 2024-02-13T14:09:22.335 INFO:teuthology.orchestra.run.smithi087.stdout:lvcreate is present 2024-02-13T14:09:22.341 INFO:teuthology.orchestra.run.smithi087.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-02-13T14:09:22.341 INFO:teuthology.orchestra.run.smithi087.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-02-13T14:09:22.346 INFO:teuthology.orchestra.run.smithi087.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-02-13T14:09:22.347 INFO:teuthology.orchestra.run.smithi087.stdout:systemctl: stdout inactive 2024-02-13T14:09:22.352 INFO:teuthology.orchestra.run.smithi087.stdout:systemctl: stdout enabled 2024-02-13T14:09:22.358 INFO:teuthology.orchestra.run.smithi087.stdout:systemctl: stdout active 2024-02-13T14:09:22.358 INFO:teuthology.orchestra.run.smithi087.stdout:Unit chronyd.service is enabled and running 2024-02-13T14:09:22.359 INFO:teuthology.orchestra.run.smithi087.stdout:Host looks OK 2024-02-13T14:09:22.359 INFO:teuthology.orchestra.run.smithi087.stdout:Cluster fsid: 5c04b074-ca79-11ee-95b9-87774f69a715 2024-02-13T14:09:22.359 INFO:teuthology.orchestra.run.smithi087.stdout:Acquiring lock 139760977899040 on /run/cephadm/5c04b074-ca79-11ee-95b9-87774f69a715.lock 2024-02-13T14:09:22.359 INFO:teuthology.orchestra.run.smithi087.stdout:Lock 139760977899040 acquired on /run/cephadm/5c04b074-ca79-11ee-95b9-87774f69a715.lock 2024-02-13T14:09:22.359 INFO:teuthology.orchestra.run.smithi087.stdout:Verifying IP 172.21.15.87 port 3300 ... 2024-02-13T14:09:22.359 INFO:teuthology.orchestra.run.smithi087.stdout:Verifying IP 172.21.15.87 port 6789 ... 2024-02-13T14:09:22.359 INFO:teuthology.orchestra.run.smithi087.stdout:Base mon IP(s) is [172.21.15.87:3300, 172.21.15.87:6789], mon addrv is [v2:172.21.15.87:3300,v1:172.21.15.87:6789] 2024-02-13T14:09:22.362 INFO:teuthology.orchestra.run.smithi087.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.87 metric 100 2024-02-13T14:09:22.362 INFO:teuthology.orchestra.run.smithi087.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.87 metric 100 2024-02-13T14:09:22.365 INFO:teuthology.orchestra.run.smithi087.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-02-13T14:09:22.365 INFO:teuthology.orchestra.run.smithi087.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 256 pref medium 2024-02-13T14:09:22.365 INFO:teuthology.orchestra.run.smithi087.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 1024 expires 1799sec hoplimit 64 pref medium 2024-02-13T14:09:22.367 INFO:teuthology.orchestra.run.smithi087.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-02-13T14:09:22.367 INFO:teuthology.orchestra.run.smithi087.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-02-13T14:09:22.367 INFO:teuthology.orchestra.run.smithi087.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-02-13T14:09:22.368 INFO:teuthology.orchestra.run.smithi087.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-02-13T14:09:22.368 INFO:teuthology.orchestra.run.smithi087.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:d5f5/64 scope link 2024-02-13T14:09:22.368 INFO:teuthology.orchestra.run.smithi087.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-02-13T14:09:22.368 INFO:teuthology.orchestra.run.smithi087.stdout:Mon IP `172.21.15.87` is in CIDR network `172.21.0.0/20` 2024-02-13T14:09:22.368 INFO:teuthology.orchestra.run.smithi087.stdout:Mon IP `172.21.15.87` is in CIDR network `172.21.0.0/20` 2024-02-13T14:09:22.369 INFO:teuthology.orchestra.run.smithi087.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-02-13T14:09:22.369 INFO:teuthology.orchestra.run.smithi087.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-02-13T14:09:22.369 INFO:teuthology.orchestra.run.smithi087.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662... 2024-02-13T14:09:22.804 INFO:teuthology.orchestra.run.smithi087.stdout:/bin/podman: stdout 28480620e5edd298d55c0e6673dcbad9ed23edd37269028a11063c1b8925676a 2024-02-13T14:09:22.804 INFO:teuthology.orchestra.run.smithi087.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662... 2024-02-13T14:09:22.804 INFO:teuthology.orchestra.run.smithi087.stdout:/bin/podman: stderr Getting image source signatures 2024-02-13T14:09:22.804 INFO:teuthology.orchestra.run.smithi087.stdout:/bin/podman: stderr Copying blob sha256:e42eddc1e8b8b944cc016e5274e8a770c05341ed32a5e7a2a56b9220b0323052 2024-02-13T14:09:22.804 INFO:teuthology.orchestra.run.smithi087.stdout:/bin/podman: stderr Copying blob sha256:58df4e03ba066accf3a7e07426bab5f1c867e27a139801c642bdafb42b9a2d9a 2024-02-13T14:09:22.804 INFO:teuthology.orchestra.run.smithi087.stdout:/bin/podman: stderr Copying config sha256:28480620e5edd298d55c0e6673dcbad9ed23edd37269028a11063c1b8925676a 2024-02-13T14:09:22.804 INFO:teuthology.orchestra.run.smithi087.stdout:/bin/podman: stderr Writing manifest to image destination 2024-02-13T14:09:23.762 INFO:teuthology.orchestra.run.smithi087.stdout:ceph: stdout ceph version 19.0.0-1423-ge1b79d5a (e1b79d5a4193a484176863ab96c0ca28c00e1662) squid (dev) 2024-02-13T14:09:23.762 INFO:teuthology.orchestra.run.smithi087.stdout:Ceph version: ceph version 19.0.0-1423-ge1b79d5a (e1b79d5a4193a484176863ab96c0ca28c00e1662) squid (dev) 2024-02-13T14:09:23.763 INFO:teuthology.orchestra.run.smithi087.stdout:Extracting ceph user uid/gid from container image... 2024-02-13T14:09:24.810 INFO:teuthology.orchestra.run.smithi087.stdout:stat: stdout 167 167 2024-02-13T14:09:24.810 INFO:teuthology.orchestra.run.smithi087.stdout:Creating initial keys... 2024-02-13T14:09:25.780 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph-authtool: stdout AQAVeMtlJJEJCxAA83YWkVmUVHOvKCeCfDlUDQ== 2024-02-13T14:09:26.652 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph-authtool: stdout AQAWeMtlDzreBBAAxuybxQ0t6tpZA603V0DrKA== 2024-02-13T14:09:27.632 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph-authtool: stdout AQAWeMtlIdJVOhAAfwyS/oEG7KDEqPozvnLJRw== 2024-02-13T14:09:27.632 INFO:teuthology.orchestra.run.smithi087.stdout:Creating initial monmap... 2024-02-13T14:09:28.605 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-02-13T14:09:28.605 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-02-13T14:09:28.606 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 5c04b074-ca79-11ee-95b9-87774f69a715 2024-02-13T14:09:28.606 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-02-13T14:09:28.606 INFO:teuthology.orchestra.run.smithi087.stdout:monmaptool for a [v2:172.21.15.87:3300,v1:172.21.15.87:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-02-13T14:09:28.606 INFO:teuthology.orchestra.run.smithi087.stdout:setting min_mon_release = quincy 2024-02-13T14:09:28.606 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/monmaptool: set fsid to 5c04b074-ca79-11ee-95b9-87774f69a715 2024-02-13T14:09:28.606 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-02-13T14:09:28.606 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-02-13T14:09:28.606 INFO:teuthology.orchestra.run.smithi087.stdout:Creating mon... 2024-02-13T14:09:29.543 INFO:teuthology.orchestra.run.smithi087.stdout:create mon.a on 2024-02-13T14:09:29.745 INFO:teuthology.orchestra.run.smithi087.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-02-13T14:09:29.889 INFO:teuthology.orchestra.run.smithi087.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-02-13T14:09:30.072 INFO:teuthology.orchestra.run.smithi087.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-5c04b074-ca79-11ee-95b9-87774f69a715.target → /etc/systemd/system/ceph-5c04b074-ca79-11ee-95b9-87774f69a715.target. 2024-02-13T14:09:30.072 INFO:teuthology.orchestra.run.smithi087.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-5c04b074-ca79-11ee-95b9-87774f69a715.target → /etc/systemd/system/ceph-5c04b074-ca79-11ee-95b9-87774f69a715.target. 2024-02-13T14:09:30.302 INFO:teuthology.orchestra.run.smithi087.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-5c04b074-ca79-11ee-95b9-87774f69a715@mon.a 2024-02-13T14:09:30.302 INFO:teuthology.orchestra.run.smithi087.stdout:systemctl: stderr Failed to reset failed state of unit ceph-5c04b074-ca79-11ee-95b9-87774f69a715@mon.a.service: Unit ceph-5c04b074-ca79-11ee-95b9-87774f69a715@mon.a.service not loaded. 2024-02-13T14:09:30.460 INFO:teuthology.orchestra.run.smithi087.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-5c04b074-ca79-11ee-95b9-87774f69a715.target.wants/ceph-5c04b074-ca79-11ee-95b9-87774f69a715@mon.a.service → /etc/systemd/system/ceph-5c04b074-ca79-11ee-95b9-87774f69a715@.service. 2024-02-13T14:09:30.934 INFO:teuthology.orchestra.run.smithi087.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-02-13T14:09:30.934 INFO:teuthology.orchestra.run.smithi087.stdout:systemctl: stdout disabled 2024-02-13T14:09:30.940 INFO:teuthology.orchestra.run.smithi087.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-02-13T14:09:30.940 INFO:teuthology.orchestra.run.smithi087.stdout:systemctl: stdout inactive 2024-02-13T14:09:30.940 INFO:teuthology.orchestra.run.smithi087.stdout:firewalld.service is not enabled 2024-02-13T14:09:30.940 INFO:teuthology.orchestra.run.smithi087.stdout:Not possible to enable service . firewalld.service is not available 2024-02-13T14:09:30.941 INFO:teuthology.orchestra.run.smithi087.stdout:Waiting for mon to start... 2024-02-13T14:09:30.941 INFO:teuthology.orchestra.run.smithi087.stdout:Waiting for mon... 2024-02-13T14:09:30.941 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:30 smithi087 systemd[1]: Started Ceph mon.a for 5c04b074-ca79-11ee-95b9-87774f69a715. 2024-02-13T14:09:31.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:30 smithi087 ceph-mon[26881]: mkfs 5c04b074-ca79-11ee-95b9-87774f69a715 2024-02-13T14:09:31.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:30 smithi087 ceph-mon[26881]: mon.a is new leader, mons a in quorum (ranks 0) 2024-02-13T14:09:32.043 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:31 smithi087 ceph-mon[26881]: mon.a is new leader, mons a in quorum (ranks 0) 2024-02-13T14:09:32.043 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:31 smithi087 ceph-mon[26881]: monmap e1: 1 mons at {a=[v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-13T14:09:32.043 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:31 smithi087 ceph-mon[26881]: fsmap 2024-02-13T14:09:32.043 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:31 smithi087 ceph-mon[26881]: osdmap e1: 0 total, 0 up, 0 in 2024-02-13T14:09:32.043 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:31 smithi087 ceph-mon[26881]: mgrmap e1: no daemons active 2024-02-13T14:09:32.043 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:31 smithi087 ceph-mon[26881]: from='client.? 172.21.15.87:0/2625126055' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-02-13T14:09:32.045 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout cluster: 2024-02-13T14:09:32.047 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout id: 5c04b074-ca79-11ee-95b9-87774f69a715 2024-02-13T14:09:32.047 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-02-13T14:09:32.047 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout 2024-02-13T14:09:32.047 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout services: 2024-02-13T14:09:32.047 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.529352s) 2024-02-13T14:09:32.047 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-02-13T14:09:32.047 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-02-13T14:09:32.047 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout 2024-02-13T14:09:32.047 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout data: 2024-02-13T14:09:32.047 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-02-13T14:09:32.048 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-02-13T14:09:32.048 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-02-13T14:09:32.048 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout pgs: 2024-02-13T14:09:32.048 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout 2024-02-13T14:09:32.048 INFO:teuthology.orchestra.run.smithi087.stdout:mon is available 2024-02-13T14:09:32.048 INFO:teuthology.orchestra.run.smithi087.stdout:Assimilating anything we can from ceph.conf... 2024-02-13T14:09:33.119 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout 2024-02-13T14:09:33.119 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout [global] 2024-02-13T14:09:33.119 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout fsid = 5c04b074-ca79-11ee-95b9-87774f69a715 2024-02-13T14:09:33.119 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.87:3300,v1:172.21.15.87:6789] 2024-02-13T14:09:33.119 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-02-13T14:09:33.119 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-02-13T14:09:33.119 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-02-13T14:09:33.119 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-02-13T14:09:33.119 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout 2024-02-13T14:09:33.119 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout [mgr] 2024-02-13T14:09:33.119 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-02-13T14:09:33.119 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout 2024-02-13T14:09:33.120 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout [osd] 2024-02-13T14:09:33.120 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-02-13T14:09:33.120 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-02-13T14:09:33.120 INFO:teuthology.orchestra.run.smithi087.stdout:Generating new minimal ceph.conf... 2024-02-13T14:09:33.241 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:32 smithi087 ceph-mon[26881]: from='client.? 172.21.15.87:0/3754886268' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-02-13T14:09:33.241 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:32 smithi087 ceph-mon[26881]: from='client.? 172.21.15.87:0/3754886268' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-02-13T14:09:34.229 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:33 smithi087 ceph-mon[26881]: from='client.? 172.21.15.87:0/687717744' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:09:34.232 INFO:teuthology.orchestra.run.smithi087.stdout:Restarting the monitor... 2024-02-13T14:09:34.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:34 smithi087 systemd[1]: Stopping Ceph mon.a for 5c04b074-ca79-11ee-95b9-87774f69a715... 2024-02-13T14:09:34.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:34 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mon-a[26858]: 2024-02-13T14:09:34.475+0000 7f5dac233700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-02-13T14:09:34.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:34 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mon-a[26858]: 2024-02-13T14:09:34.475+0000 7f5dac233700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-02-13T14:09:34.774 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:34 smithi087 podman[27214]: 2024-02-13 14:09:34.496590229 +0000 UTC m=+0.099742185 container died 441b2d39a3678f6bfde601079856d1fc08ac0fc81c672f398e3c66904ea2b4b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, org.label-schema.build-date=20240131, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275) 2024-02-13T14:09:35.149 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:34 smithi087 podman[27214]: 2024-02-13 14:09:34.872466032 +0000 UTC m=+0.475617996 container cleanup 441b2d39a3678f6bfde601079856d1fc08ac0fc81c672f398e3c66904ea2b4b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mon-a, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.2, org.label-schema.build-date=20240131, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5) 2024-02-13T14:09:35.149 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:34 smithi087 bash[27214]: ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mon-a 2024-02-13T14:09:35.491 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 podman[27233]: 2024-02-13 14:09:35.148178815 +0000 UTC m=+0.642882752 container remove 441b2d39a3678f6bfde601079856d1fc08ac0fc81c672f398e3c66904ea2b4b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mon-a, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240131, io.buildah.version=1.33.2, org.label-schema.vendor=CentOS, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True) 2024-02-13T14:09:35.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 systemd[1]: ceph-5c04b074-ca79-11ee-95b9-87774f69a715@mon.a.service: Deactivated successfully. 2024-02-13T14:09:35.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 systemd[1]: Stopped Ceph mon.a for 5c04b074-ca79-11ee-95b9-87774f69a715. 2024-02-13T14:09:35.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 systemd[1]: Starting Ceph mon.a for 5c04b074-ca79-11ee-95b9-87774f69a715... 2024-02-13T14:09:35.769 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 podman[27334]: 2024-02-13 14:09:35.40001428 +0000 UTC m=+0.017824078 image pull 28480620e5edd298d55c0e6673dcbad9ed23edd37269028a11063c1b8925676a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662 2024-02-13T14:09:35.769 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 podman[27334]: 2024-02-13 14:09:35.498869985 +0000 UTC m=+0.116679774 container create 1c16bb4c95da4b569ba6767150c9c2d8fdc9b72940450e33defe962120ec75da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mon-a, io.buildah.version=1.33.2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240131, GIT_BRANCH=HEAD, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, maintainer=Guillaume Abrioux ) 2024-02-13T14:09:35.769 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 podman[27334]: 2024-02-13 14:09:35.692989709 +0000 UTC m=+0.310799503 container init 1c16bb4c95da4b569ba6767150c9c2d8fdc9b72940450e33defe962120ec75da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mon-a, org.label-schema.schema-version=1.0, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, GIT_CLEAN=True, ceph=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.license=GPLv2, io.buildah.version=1.33.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240131, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-02-13T14:09:35.769 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 podman[27334]: 2024-02-13 14:09:35.698112256 +0000 UTC m=+0.315922057 container start 1c16bb4c95da4b569ba6767150c9c2d8fdc9b72940450e33defe962120ec75da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.2, org.label-schema.build-date=20240131, org.label-schema.license=GPLv2, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-02-13T14:09:35.769 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: set uid:gid to 167:167 (ceph:ceph) 2024-02-13T14:09:35.769 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: ceph version 19.0.0-1423-ge1b79d5a (e1b79d5a4193a484176863ab96c0ca28c00e1662) squid (dev), process ceph-mon, pid 7 2024-02-13T14:09:35.770 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: pidfile_write: ignore empty --pid-file 2024-02-13T14:09:35.770 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: load: jerasure load: lrc 2024-02-13T14:09:35.770 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: RocksDB version: 7.9.2 2024-02-13T14:09:35.770 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Git sha 0 2024-02-13T14:09:35.770 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Compile date 2024-02-12 22:40:01 2024-02-13T14:09:35.770 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: DB SUMMARY 2024-02-13T14:09:35.770 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: DB Session ID: HZHXFNG9MKL6J1YOWXU2 2024-02-13T14:09:35.770 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: CURRENT file: CURRENT 2024-02-13T14:09:35.770 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: IDENTITY file: IDENTITY 2024-02-13T14:09:35.770 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-02-13T14:09:35.770 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-02-13T14:09:35.770 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 85021 ; 2024-02-13T14:09:35.772 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.error_if_exists: 0 2024-02-13T14:09:35.772 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.create_if_missing: 0 2024-02-13T14:09:35.772 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.paranoid_checks: 1 2024-02-13T14:09:35.772 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.flush_verify_memtable_count: 1 2024-02-13T14:09:35.772 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-02-13T14:09:35.772 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-02-13T14:09:35.772 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.env: 0x55cf00d7eae0 2024-02-13T14:09:35.772 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.fs: PosixFileSystem 2024-02-13T14:09:35.772 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.info_log: 0x55cf032838c0 2024-02-13T14:09:35.773 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.max_file_opening_threads: 16 2024-02-13T14:09:35.773 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.statistics: (nil) 2024-02-13T14:09:35.773 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.use_fsync: 0 2024-02-13T14:09:35.773 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.max_log_file_size: 0 2024-02-13T14:09:35.773 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-02-13T14:09:35.773 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.log_file_time_to_roll: 0 2024-02-13T14:09:35.773 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.keep_log_file_num: 1000 2024-02-13T14:09:35.773 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.recycle_log_file_num: 0 2024-02-13T14:09:35.773 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.allow_fallocate: 1 2024-02-13T14:09:35.773 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.allow_mmap_reads: 0 2024-02-13T14:09:35.773 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.allow_mmap_writes: 0 2024-02-13T14:09:35.773 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.use_direct_reads: 0 2024-02-13T14:09:35.773 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-02-13T14:09:35.774 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.create_missing_column_families: 0 2024-02-13T14:09:35.774 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.db_log_dir: 2024-02-13T14:09:35.774 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.wal_dir: 2024-02-13T14:09:35.774 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.table_cache_numshardbits: 6 2024-02-13T14:09:35.774 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.WAL_ttl_seconds: 0 2024-02-13T14:09:35.774 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.WAL_size_limit_MB: 0 2024-02-13T14:09:35.774 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-02-13T14:09:35.774 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-02-13T14:09:35.774 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.is_fd_close_on_exec: 1 2024-02-13T14:09:35.774 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.advise_random_on_open: 1 2024-02-13T14:09:35.774 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.db_write_buffer_size: 0 2024-02-13T14:09:35.774 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.write_buffer_manager: 0x55cf03257400 2024-02-13T14:09:35.774 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-02-13T14:09:35.774 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-02-13T14:09:35.775 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.use_adaptive_mutex: 0 2024-02-13T14:09:35.775 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.rate_limiter: (nil) 2024-02-13T14:09:35.775 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-02-13T14:09:35.775 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.wal_recovery_mode: 2 2024-02-13T14:09:35.775 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.enable_thread_tracking: 0 2024-02-13T14:09:35.775 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.enable_pipelined_write: 0 2024-02-13T14:09:35.775 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.unordered_write: 0 2024-02-13T14:09:35.775 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-02-13T14:09:35.775 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-02-13T14:09:35.775 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-02-13T14:09:35.775 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-02-13T14:09:35.775 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.row_cache: None 2024-02-13T14:09:35.775 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.wal_filter: None 2024-02-13T14:09:35.776 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-02-13T14:09:35.776 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.allow_ingest_behind: 0 2024-02-13T14:09:35.776 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.two_write_queues: 0 2024-02-13T14:09:35.776 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.manual_wal_flush: 0 2024-02-13T14:09:35.776 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.wal_compression: 0 2024-02-13T14:09:35.776 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.atomic_flush: 0 2024-02-13T14:09:35.776 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-02-13T14:09:35.776 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.persist_stats_to_disk: 0 2024-02-13T14:09:35.776 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.write_dbid_to_manifest: 0 2024-02-13T14:09:35.776 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.log_readahead_size: 0 2024-02-13T14:09:35.776 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-02-13T14:09:35.776 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.best_efforts_recovery: 0 2024-02-13T14:09:35.776 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-02-13T14:09:35.777 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-02-13T14:09:35.777 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.allow_data_in_errors: 0 2024-02-13T14:09:35.777 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.db_host_id: __hostname__ 2024-02-13T14:09:35.777 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.enforce_single_del_contracts: true 2024-02-13T14:09:35.777 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.max_background_jobs: 2 2024-02-13T14:09:35.777 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.max_background_compactions: -1 2024-02-13T14:09:35.777 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.max_subcompactions: 1 2024-02-13T14:09:35.777 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-02-13T14:09:35.777 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-02-13T14:09:35.777 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.delayed_write_rate : 16777216 2024-02-13T14:09:35.777 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.max_total_wal_size: 0 2024-02-13T14:09:35.777 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-02-13T14:09:35.777 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.stats_dump_period_sec: 600 2024-02-13T14:09:35.777 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.stats_persist_period_sec: 600 2024-02-13T14:09:35.778 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-02-13T14:09:35.778 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.max_open_files: -1 2024-02-13T14:09:35.778 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.bytes_per_sync: 0 2024-02-13T14:09:35.778 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.wal_bytes_per_sync: 0 2024-02-13T14:09:35.778 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.strict_bytes_per_sync: 0 2024-02-13T14:09:35.778 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.compaction_readahead_size: 0 2024-02-13T14:09:35.778 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.max_background_flushes: -1 2024-02-13T14:09:35.778 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Compression algorithms supported: 2024-02-13T14:09:35.778 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-02-13T14:09:35.778 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: kZSTD supported: 0 2024-02-13T14:09:35.779 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: kXpressCompression supported: 0 2024-02-13T14:09:35.779 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: kLZ4HCCompression supported: 1 2024-02-13T14:09:35.779 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: kZlibCompression supported: 1 2024-02-13T14:09:35.779 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: kSnappyCompression supported: 1 2024-02-13T14:09:35.779 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: kLZ4Compression supported: 1 2024-02-13T14:09:35.779 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: kBZip2Compression supported: 0 2024-02-13T14:09:35.779 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Fast CRC32 supported: Supported on x86 2024-02-13T14:09:35.779 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: DMutex implementation: pthread_mutex_t 2024-02-13T14:09:35.779 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-02-13T14:09:35.779 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-02-13T14:09:35.779 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-02-13T14:09:35.779 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.merge_operator: 2024-02-13T14:09:35.779 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.compaction_filter: None 2024-02-13T14:09:35.780 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.compaction_filter_factory: None 2024-02-13T14:09:35.780 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.sst_partitioner_factory: None 2024-02-13T14:09:35.780 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.memtable_factory: SkipListFactory 2024-02-13T14:09:35.780 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.table_factory: BlockBasedTable 2024-02-13T14:09:35.780 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55cf032839c0) 2024-02-13T14:09:35.780 INFO:journalctl@ceph.mon.a.smithi087.stdout: cache_index_and_filter_blocks: 1 2024-02-13T14:09:35.780 INFO:journalctl@ceph.mon.a.smithi087.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-02-13T14:09:35.780 INFO:journalctl@ceph.mon.a.smithi087.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-02-13T14:09:35.780 INFO:journalctl@ceph.mon.a.smithi087.stdout: pin_top_level_index_and_filter: 1 2024-02-13T14:09:35.780 INFO:journalctl@ceph.mon.a.smithi087.stdout: index_type: 0 2024-02-13T14:09:35.780 INFO:journalctl@ceph.mon.a.smithi087.stdout: data_block_index_type: 0 2024-02-13T14:09:35.780 INFO:journalctl@ceph.mon.a.smithi087.stdout: index_shortening: 1 2024-02-13T14:09:35.781 INFO:journalctl@ceph.mon.a.smithi087.stdout: data_block_hash_table_util_ratio: 0.750000 2024-02-13T14:09:35.781 INFO:journalctl@ceph.mon.a.smithi087.stdout: checksum: 4 2024-02-13T14:09:35.781 INFO:journalctl@ceph.mon.a.smithi087.stdout: no_block_cache: 0 2024-02-13T14:09:35.781 INFO:journalctl@ceph.mon.a.smithi087.stdout: block_cache: 0x55cf03287090 2024-02-13T14:09:35.781 INFO:journalctl@ceph.mon.a.smithi087.stdout: block_cache_name: BinnedLRUCache 2024-02-13T14:09:35.781 INFO:journalctl@ceph.mon.a.smithi087.stdout: block_cache_options: 2024-02-13T14:09:35.781 INFO:journalctl@ceph.mon.a.smithi087.stdout: capacity : 536870912 2024-02-13T14:09:35.781 INFO:journalctl@ceph.mon.a.smithi087.stdout: num_shard_bits : 4 2024-02-13T14:09:35.781 INFO:journalctl@ceph.mon.a.smithi087.stdout: strict_capacity_limit : 0 2024-02-13T14:09:35.781 INFO:journalctl@ceph.mon.a.smithi087.stdout: high_pri_pool_ratio: 0.000 2024-02-13T14:09:35.782 INFO:journalctl@ceph.mon.a.smithi087.stdout: block_cache_compressed: (nil) 2024-02-13T14:09:35.782 INFO:journalctl@ceph.mon.a.smithi087.stdout: persistent_cache: (nil) 2024-02-13T14:09:35.782 INFO:journalctl@ceph.mon.a.smithi087.stdout: block_size: 4096 2024-02-13T14:09:35.782 INFO:journalctl@ceph.mon.a.smithi087.stdout: block_size_deviation: 10 2024-02-13T14:09:35.782 INFO:journalctl@ceph.mon.a.smithi087.stdout: block_restart_interval: 16 2024-02-13T14:09:35.782 INFO:journalctl@ceph.mon.a.smithi087.stdout: index_block_restart_interval: 1 2024-02-13T14:09:35.782 INFO:journalctl@ceph.mon.a.smithi087.stdout: metadata_block_size: 4096 2024-02-13T14:09:35.782 INFO:journalctl@ceph.mon.a.smithi087.stdout: partition_filters: 0 2024-02-13T14:09:35.782 INFO:journalctl@ceph.mon.a.smithi087.stdout: use_delta_encoding: 1 2024-02-13T14:09:35.782 INFO:journalctl@ceph.mon.a.smithi087.stdout: filter_policy: bloomfilter 2024-02-13T14:09:35.782 INFO:journalctl@ceph.mon.a.smithi087.stdout: whole_key_filtering: 1 2024-02-13T14:09:35.782 INFO:journalctl@ceph.mon.a.smithi087.stdout: verify_compression: 0 2024-02-13T14:09:35.782 INFO:journalctl@ceph.mon.a.smithi087.stdout: read_amp_bytes_per_bit: 0 2024-02-13T14:09:35.782 INFO:journalctl@ceph.mon.a.smithi087.stdout: format_version: 5 2024-02-13T14:09:35.783 INFO:journalctl@ceph.mon.a.smithi087.stdout: enable_index_compression: 1 2024-02-13T14:09:35.783 INFO:journalctl@ceph.mon.a.smithi087.stdout: block_align: 0 2024-02-13T14:09:35.783 INFO:journalctl@ceph.mon.a.smithi087.stdout: max_auto_readahead_size: 262144 2024-02-13T14:09:35.783 INFO:journalctl@ceph.mon.a.smithi087.stdout: prepopulate_block_cache: 0 2024-02-13T14:09:35.783 INFO:journalctl@ceph.mon.a.smithi087.stdout: initial_auto_readahead_size: 8192 2024-02-13T14:09:35.783 INFO:journalctl@ceph.mon.a.smithi087.stdout: num_file_reads_for_auto_readahead: 2 2024-02-13T14:09:35.783 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.write_buffer_size: 33554432 2024-02-13T14:09:35.783 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.max_write_buffer_number: 2 2024-02-13T14:09:35.783 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.compression: NoCompression 2024-02-13T14:09:35.783 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.bottommost_compression: Disabled 2024-02-13T14:09:35.783 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.prefix_extractor: nullptr 2024-02-13T14:09:35.783 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-02-13T14:09:35.783 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.num_levels: 7 2024-02-13T14:09:35.783 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-02-13T14:09:35.784 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-02-13T14:09:35.784 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-02-13T14:09:35.784 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-02-13T14:09:35.784 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-02-13T14:09:35.784 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-02-13T14:09:35.784 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-02-13T14:09:35.784 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-02-13T14:09:35.784 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-02-13T14:09:35.784 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-02-13T14:09:35.784 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-02-13T14:09:35.784 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-02-13T14:09:35.784 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.compression_opts.window_bits: -14 2024-02-13T14:09:35.785 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.compression_opts.level: 32767 2024-02-13T14:09:35.785 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.compression_opts.strategy: 0 2024-02-13T14:09:35.785 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-02-13T14:09:35.785 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-02-13T14:09:35.785 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-02-13T14:09:35.785 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-02-13T14:09:35.785 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.compression_opts.enabled: false 2024-02-13T14:09:35.785 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-02-13T14:09:35.786 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-02-13T14:09:35.786 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-02-13T14:09:35.786 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-02-13T14:09:35.786 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.target_file_size_base: 67108864 2024-02-13T14:09:35.786 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.target_file_size_multiplier: 1 2024-02-13T14:09:35.786 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-02-13T14:09:35.786 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-02-13T14:09:35.786 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-02-13T14:09:35.786 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-02-13T14:09:35.787 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-02-13T14:09:35.787 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-02-13T14:09:35.787 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-02-13T14:09:35.787 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-02-13T14:09:35.787 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-02-13T14:09:35.787 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-02-13T14:09:35.787 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-02-13T14:09:35.787 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-02-13T14:09:35.787 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-02-13T14:09:35.787 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.arena_block_size: 1048576 2024-02-13T14:09:35.787 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-02-13T14:09:35.787 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-02-13T14:09:35.787 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.disable_auto_compactions: 0 2024-02-13T14:09:35.787 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-02-13T14:09:35.788 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-02-13T14:09:35.788 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-02-13T14:09:35.788 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-02-13T14:09:35.788 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-02-13T14:09:35.788 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-02-13T14:09:35.788 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-02-13T14:09:35.788 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-02-13T14:09:35.788 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-02-13T14:09:35.788 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-02-13T14:09:35.789 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.table_properties_collectors: 2024-02-13T14:09:35.789 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.inplace_update_support: 0 2024-02-13T14:09:35.789 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.inplace_update_num_locks: 10000 2024-02-13T14:09:35.789 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-02-13T14:09:35.789 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-02-13T14:09:35.789 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.memtable_huge_page_size: 0 2024-02-13T14:09:35.789 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.bloom_locality: 0 2024-02-13T14:09:35.789 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.max_successive_merges: 0 2024-02-13T14:09:35.790 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.optimize_filters_for_hits: 0 2024-02-13T14:09:35.790 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.paranoid_file_checks: 0 2024-02-13T14:09:35.790 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.force_consistency_checks: 1 2024-02-13T14:09:35.790 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.report_bg_io_stats: 0 2024-02-13T14:09:35.790 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.ttl: 2592000 2024-02-13T14:09:35.790 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.periodic_compaction_seconds: 0 2024-02-13T14:09:35.790 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-02-13T14:09:35.790 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-02-13T14:09:35.790 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.enable_blob_files: false 2024-02-13T14:09:35.790 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.min_blob_size: 0 2024-02-13T14:09:35.790 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.blob_file_size: 268435456 2024-02-13T14:09:35.790 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.blob_compression_type: NoCompression 2024-02-13T14:09:35.790 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.enable_blob_garbage_collection: false 2024-02-13T14:09:35.790 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-02-13T14:09:35.790 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-02-13T14:09:35.791 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-02-13T14:09:35.791 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.blob_file_starting_level: 0 2024-02-13T14:09:35.791 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-02-13T14:09:35.791 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-02-13T14:09:35.791 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-02-13T14:09:35.791 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 9c99f6bc-4187-44a5-b83b-682d0e9d8877 2024-02-13T14:09:35.791 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: EVENT_LOG_v1 {"time_micros": 1707833375727010, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-02-13T14:09:35.791 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-02-13T14:09:35.791 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: EVENT_LOG_v1 {"time_micros": 1707833375727963, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 81646, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 254, "table_properties": {"data_size": 79833, "index_size": 223, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 581, "raw_key_size": 10126, "raw_average_key_size": 47, "raw_value_size": 74053, "raw_average_value_size": 347, "num_data_blocks": 10, "num_entries": 213, "num_filter_entries": 213, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1707833375, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "9c99f6bc-4187-44a5-b83b-682d0e9d8877", "db_session_id": "HZHXFNG9MKL6J1YOWXU2", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-02-13T14:09:35.791 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: EVENT_LOG_v1 {"time_micros": 1707833375728043, "job": 1, "event": "recovery_finished"} 2024-02-13T14:09:35.791 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-02-13T14:09:35.791 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-02-13T14:09:35.791 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55cf03378000 2024-02-13T14:09:35.792 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: DB pointer 0x55cf03362000 2024-02-13T14:09:35.792 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-02-13T14:09:35.792 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-02-13T14:09:35.792 INFO:journalctl@ceph.mon.a.smithi087.stdout: ** DB Stats ** 2024-02-13T14:09:35.792 INFO:journalctl@ceph.mon.a.smithi087.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-13T14:09:35.792 INFO:journalctl@ceph.mon.a.smithi087.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-02-13T14:09:35.792 INFO:journalctl@ceph.mon.a.smithi087.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-13T14:09:35.792 INFO:journalctl@ceph.mon.a.smithi087.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-13T14:09:35.792 INFO:journalctl@ceph.mon.a.smithi087.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-02-13T14:09:35.792 INFO:journalctl@ceph.mon.a.smithi087.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-13T14:09:35.792 INFO:journalctl@ceph.mon.a.smithi087.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-13T14:09:35.792 INFO:journalctl@ceph.mon.a.smithi087.stdout: 2024-02-13T14:09:35.793 INFO:journalctl@ceph.mon.a.smithi087.stdout: ** Compaction Stats [default] ** 2024-02-13T14:09:35.793 INFO:journalctl@ceph.mon.a.smithi087.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-13T14:09:35.793 INFO:journalctl@ceph.mon.a.smithi087.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-02-13T14:09:35.793 INFO:journalctl@ceph.mon.a.smithi087.stdout: L0 2/0 81.57 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 146.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-13T14:09:35.793 INFO:journalctl@ceph.mon.a.smithi087.stdout: Sum 2/0 81.57 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 146.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-13T14:09:35.793 INFO:journalctl@ceph.mon.a.smithi087.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 146.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-13T14:09:35.794 INFO:journalctl@ceph.mon.a.smithi087.stdout: 2024-02-13T14:09:35.794 INFO:journalctl@ceph.mon.a.smithi087.stdout: ** Compaction Stats [default] ** 2024-02-13T14:09:35.794 INFO:journalctl@ceph.mon.a.smithi087.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-13T14:09:35.794 INFO:journalctl@ceph.mon.a.smithi087.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-13T14:09:35.794 INFO:journalctl@ceph.mon.a.smithi087.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 146.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-13T14:09:35.794 INFO:journalctl@ceph.mon.a.smithi087.stdout: 2024-02-13T14:09:35.794 INFO:journalctl@ceph.mon.a.smithi087.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-02-13T14:09:35.795 INFO:journalctl@ceph.mon.a.smithi087.stdout: 2024-02-13T14:09:35.795 INFO:journalctl@ceph.mon.a.smithi087.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-13T14:09:35.795 INFO:journalctl@ceph.mon.a.smithi087.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-02-13T14:09:35.795 INFO:journalctl@ceph.mon.a.smithi087.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-13T14:09:35.795 INFO:journalctl@ceph.mon.a.smithi087.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-02-13T14:09:35.795 INFO:journalctl@ceph.mon.a.smithi087.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-02-13T14:09:35.796 INFO:journalctl@ceph.mon.a.smithi087.stdout: AddFile(Keys): cumulative 0, interval 0 2024-02-13T14:09:35.796 INFO:journalctl@ceph.mon.a.smithi087.stdout: Cumulative compaction: 0.00 GB write, 27.19 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-13T14:09:35.796 INFO:journalctl@ceph.mon.a.smithi087.stdout: Interval compaction: 0.00 GB write, 27.19 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-13T14:09:35.796 INFO:journalctl@ceph.mon.a.smithi087.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-13T14:09:35.796 INFO:journalctl@ceph.mon.a.smithi087.stdout: Block cache BinnedLRUCache@0x55cf03287090#7 capacity: 512.00 MB usage: 1.17 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.2e-05 secs_since: 0 2024-02-13T14:09:35.796 INFO:journalctl@ceph.mon.a.smithi087.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.77 KB,0.000146031%) IndexBlock(2,0.41 KB,7.7486e-05%) Misc(1,0.00 KB,0%) 2024-02-13T14:09:35.797 INFO:journalctl@ceph.mon.a.smithi087.stdout: 2024-02-13T14:09:35.797 INFO:journalctl@ceph.mon.a.smithi087.stdout: ** File Read Latency Histogram By Level [default] ** 2024-02-13T14:09:35.797 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: starting mon.a rank 0 at public addrs [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] at bind addrs [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 5c04b074-ca79-11ee-95b9-87774f69a715 2024-02-13T14:09:35.797 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: mon.a@-1(???) e1 preinit fsid 5c04b074-ca79-11ee-95b9-87774f69a715 2024-02-13T14:09:35.797 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: mon.a@-1(???).mds e1 new map 2024-02-13T14:09:35.797 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: mon.a@-1(???).mds e1 print_map 2024-02-13T14:09:35.797 INFO:journalctl@ceph.mon.a.smithi087.stdout: e1 2024-02-13T14:09:35.798 INFO:journalctl@ceph.mon.a.smithi087.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-02-13T14:09:35.798 INFO:journalctl@ceph.mon.a.smithi087.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-02-13T14:09:35.798 INFO:journalctl@ceph.mon.a.smithi087.stdout: legacy client fscid: -1 2024-02-13T14:09:35.798 INFO:journalctl@ceph.mon.a.smithi087.stdout: 2024-02-13T14:09:35.798 INFO:journalctl@ceph.mon.a.smithi087.stdout: No filesystems configured 2024-02-13T14:09:35.798 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-02-13T14:09:35.798 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-02-13T14:09:35.799 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-02-13T14:09:35.799 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-02-13T14:09:35.799 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-02-13T14:09:35.799 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 bash[27334]: 1c16bb4c95da4b569ba6767150c9c2d8fdc9b72940450e33defe962120ec75da 2024-02-13T14:09:35.799 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 systemd[1]: Started Ceph mon.a for 5c04b074-ca79-11ee-95b9-87774f69a715. 2024-02-13T14:09:35.800 INFO:teuthology.orchestra.run.smithi087.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-02-13T14:09:36.079 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: mon.a is new leader, mons a in quorum (ranks 0) 2024-02-13T14:09:36.079 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: monmap e1: 1 mons at {a=[v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-13T14:09:36.080 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: fsmap 2024-02-13T14:09:36.080 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: osdmap e1: 0 total, 0 up, 0 in 2024-02-13T14:09:36.080 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:35 smithi087 ceph-mon[27372]: mgrmap e1: no daemons active 2024-02-13T14:09:36.784 INFO:teuthology.orchestra.run.smithi087.stdout:Wrote config to /etc/ceph/ceph.conf 2024-02-13T14:09:36.806 INFO:teuthology.orchestra.run.smithi087.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-02-13T14:09:36.807 INFO:teuthology.orchestra.run.smithi087.stdout:Creating mgr... 2024-02-13T14:09:36.807 INFO:teuthology.orchestra.run.smithi087.stdout:Verifying port 0.0.0.0:9283 ... 2024-02-13T14:09:36.807 INFO:teuthology.orchestra.run.smithi087.stdout:Verifying port 0.0.0.0:8765 ... 2024-02-13T14:09:37.050 INFO:teuthology.orchestra.run.smithi087.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-5c04b074-ca79-11ee-95b9-87774f69a715@mgr.y 2024-02-13T14:09:37.051 INFO:teuthology.orchestra.run.smithi087.stdout:systemctl: stderr Failed to reset failed state of unit ceph-5c04b074-ca79-11ee-95b9-87774f69a715@mgr.y.service: Unit ceph-5c04b074-ca79-11ee-95b9-87774f69a715@mgr.y.service not loaded. 2024-02-13T14:09:37.197 INFO:teuthology.orchestra.run.smithi087.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-5c04b074-ca79-11ee-95b9-87774f69a715.target.wants/ceph-5c04b074-ca79-11ee-95b9-87774f69a715@mgr.y.service → /etc/systemd/system/ceph-5c04b074-ca79-11ee-95b9-87774f69a715@.service. 2024-02-13T14:09:37.409 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:37 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/3380228178' entity='client.admin' 2024-02-13T14:09:37.728 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:09:37 smithi087 podman[27618]: 2024-02-13 14:09:37.632500856 +0000 UTC m=+0.318299231 container start b3df15cc685cf1b1817b59ce4879a7fd631835234a3af3e093815880bf16935a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240131, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.2, GIT_CLEAN=True, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, maintainer=Guillaume Abrioux ) 2024-02-13T14:09:37.743 INFO:teuthology.orchestra.run.smithi087.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-02-13T14:09:37.743 INFO:teuthology.orchestra.run.smithi087.stdout:systemctl: stdout disabled 2024-02-13T14:09:37.748 INFO:teuthology.orchestra.run.smithi087.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-02-13T14:09:37.748 INFO:teuthology.orchestra.run.smithi087.stdout:systemctl: stdout inactive 2024-02-13T14:09:37.748 INFO:teuthology.orchestra.run.smithi087.stdout:firewalld.service is not enabled 2024-02-13T14:09:37.748 INFO:teuthology.orchestra.run.smithi087.stdout:Not possible to enable service . firewalld.service is not available 2024-02-13T14:09:37.756 INFO:teuthology.orchestra.run.smithi087.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-02-13T14:09:37.756 INFO:teuthology.orchestra.run.smithi087.stdout:systemctl: stdout disabled 2024-02-13T14:09:37.763 INFO:teuthology.orchestra.run.smithi087.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-02-13T14:09:37.763 INFO:teuthology.orchestra.run.smithi087.stdout:systemctl: stdout inactive 2024-02-13T14:09:37.763 INFO:teuthology.orchestra.run.smithi087.stdout:firewalld.service is not enabled 2024-02-13T14:09:37.763 INFO:teuthology.orchestra.run.smithi087.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-02-13T14:09:37.763 INFO:teuthology.orchestra.run.smithi087.stdout:Waiting for mgr to start... 2024-02-13T14:09:37.764 INFO:teuthology.orchestra.run.smithi087.stdout:Waiting for mgr... 2024-02-13T14:09:37.983 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:09:37 smithi087 bash[27618]: b3df15cc685cf1b1817b59ce4879a7fd631835234a3af3e093815880bf16935a 2024-02-13T14:09:37.983 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:09:37 smithi087 systemd[1]: Started Ceph mgr.y for 5c04b074-ca79-11ee-95b9-87774f69a715. 2024-02-13T14:09:38.241 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:09:38 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: 2024-02-13T14:09:38.189+0000 7f598fa65200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-13T14:09:38.591 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:38 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/1331002537' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-13T14:09:38.592 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:09:38 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: 2024-02-13T14:09:38.285+0000 7f598fa65200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-13T14:09:38.852 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout 2024-02-13T14:09:38.855 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout { 2024-02-13T14:09:38.855 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "fsid": "5c04b074-ca79-11ee-95b9-87774f69a715", 2024-02-13T14:09:38.855 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "health": { 2024-02-13T14:09:38.855 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-02-13T14:09:38.856 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-02-13T14:09:38.856 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-02-13T14:09:38.856 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout }, 2024-02-13T14:09:38.856 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-02-13T14:09:38.856 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-02-13T14:09:38.856 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout 0 2024-02-13T14:09:38.856 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout ], 2024-02-13T14:09:38.856 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-02-13T14:09:38.856 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "a" 2024-02-13T14:09:38.856 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout ], 2024-02-13T14:09:38.856 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-02-13T14:09:38.857 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "monmap": { 2024-02-13T14:09:38.857 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-13T14:09:38.857 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-02-13T14:09:38.857 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-02-13T14:09:38.857 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout }, 2024-02-13T14:09:38.857 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-02-13T14:09:38.857 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-13T14:09:38.857 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-02-13T14:09:38.857 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-02-13T14:09:38.858 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-02-13T14:09:38.858 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-02-13T14:09:38.858 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-02-13T14:09:38.858 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-02-13T14:09:38.861 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout }, 2024-02-13T14:09:38.861 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-02-13T14:09:38.862 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-02-13T14:09:38.862 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-02-13T14:09:38.862 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-02-13T14:09:38.862 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-02-13T14:09:38.862 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-02-13T14:09:38.862 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-02-13T14:09:38.862 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-02-13T14:09:38.862 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-02-13T14:09:38.862 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout }, 2024-02-13T14:09:38.862 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-02-13T14:09:38.862 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-13T14:09:38.862 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-02-13T14:09:38.862 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-02-13T14:09:38.862 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout }, 2024-02-13T14:09:38.863 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-02-13T14:09:38.863 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "available": false, 2024-02-13T14:09:38.863 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-02-13T14:09:38.863 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "modules": [ 2024-02-13T14:09:38.863 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "iostat", 2024-02-13T14:09:38.863 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "nfs", 2024-02-13T14:09:38.863 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "restful" 2024-02-13T14:09:38.863 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout ], 2024-02-13T14:09:38.863 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-13T14:09:38.863 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout }, 2024-02-13T14:09:38.863 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-02-13T14:09:38.863 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-13T14:09:38.863 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "modified": "2024-02-13T14:09:30.938724+0000", 2024-02-13T14:09:38.863 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-13T14:09:38.863 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout }, 2024-02-13T14:09:38.864 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-02-13T14:09:38.864 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout } 2024-02-13T14:09:38.864 INFO:teuthology.orchestra.run.smithi087.stdout:mgr not available, waiting (1/15)... 2024-02-13T14:09:39.177 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:09:38 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: 2024-02-13T14:09:38.946+0000 7f598fa65200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-13T14:09:39.178 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:09:39 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: 2024-02-13T14:09:39.037+0000 7f598fa65200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-13T14:09:39.178 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:09:39 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: 2024-02-13T14:09:39.176+0000 7f598fa65200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-13T14:09:39.491 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:09:39 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: 2024-02-13T14:09:39.269+0000 7f598fa65200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-13T14:09:39.492 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:09:39 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: 2024-02-13T14:09:39.352+0000 7f598fa65200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-13T14:09:39.900 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:09:39 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: 2024-02-13T14:09:39.612+0000 7f598fa65200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-13T14:09:39.900 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:09:39 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: 2024-02-13T14:09:39.720+0000 7f598fa65200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-13T14:09:39.900 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:09:39 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: 2024-02-13T14:09:39.805+0000 7f598fa65200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-13T14:09:39.900 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:09:39 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: 2024-02-13T14:09:39.898+0000 7f598fa65200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-13T14:09:40.241 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:09:40 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: 2024-02-13T14:09:40.071+0000 7f598fa65200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-13T14:09:41.241 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:09:41 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: 2024-02-13T14:09:41.038+0000 7f598fa65200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-13T14:09:41.242 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:09:41 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: 2024-02-13T14:09:41.172+0000 7f598fa65200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-13T14:09:41.514 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:41 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/3356568877' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-13T14:09:41.790 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:09:41 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: 2024-02-13T14:09:41.512+0000 7f598fa65200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-13T14:09:41.790 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:09:41 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: 2024-02-13T14:09:41.602+0000 7f598fa65200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-13T14:09:42.098 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:09:41 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: 2024-02-13T14:09:41.805+0000 7f598fa65200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-13T14:09:42.098 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:09:41 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: 2024-02-13T14:09:41.987+0000 7f598fa65200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-13T14:09:42.103 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout 2024-02-13T14:09:42.104 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout { 2024-02-13T14:09:42.104 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "fsid": "5c04b074-ca79-11ee-95b9-87774f69a715", 2024-02-13T14:09:42.104 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "health": { 2024-02-13T14:09:42.104 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-02-13T14:09:42.104 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-02-13T14:09:42.104 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-02-13T14:09:42.104 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout }, 2024-02-13T14:09:42.104 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-02-13T14:09:42.104 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-02-13T14:09:42.104 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout 0 2024-02-13T14:09:42.105 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout ], 2024-02-13T14:09:42.105 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-02-13T14:09:42.105 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "a" 2024-02-13T14:09:42.105 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout ], 2024-02-13T14:09:42.105 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-02-13T14:09:42.105 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "monmap": { 2024-02-13T14:09:42.105 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-13T14:09:42.105 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-02-13T14:09:42.105 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-02-13T14:09:42.105 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout }, 2024-02-13T14:09:42.105 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-02-13T14:09:42.105 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-13T14:09:42.105 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-02-13T14:09:42.105 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-02-13T14:09:42.106 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-02-13T14:09:42.106 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-02-13T14:09:42.106 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-02-13T14:09:42.106 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-02-13T14:09:42.109 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout }, 2024-02-13T14:09:42.110 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-02-13T14:09:42.110 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-02-13T14:09:42.110 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-02-13T14:09:42.110 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-02-13T14:09:42.110 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-02-13T14:09:42.110 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-02-13T14:09:42.110 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-02-13T14:09:42.110 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-02-13T14:09:42.110 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-02-13T14:09:42.110 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout }, 2024-02-13T14:09:42.110 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-02-13T14:09:42.110 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-13T14:09:42.110 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-02-13T14:09:42.111 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-02-13T14:09:42.111 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout }, 2024-02-13T14:09:42.111 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-02-13T14:09:42.111 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "available": false, 2024-02-13T14:09:42.111 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-02-13T14:09:42.111 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "modules": [ 2024-02-13T14:09:42.111 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "iostat", 2024-02-13T14:09:42.111 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "nfs", 2024-02-13T14:09:42.111 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "restful" 2024-02-13T14:09:42.111 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout ], 2024-02-13T14:09:42.111 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-13T14:09:42.111 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout }, 2024-02-13T14:09:42.111 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-02-13T14:09:42.111 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-13T14:09:42.111 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "modified": "2024-02-13T14:09:30.938724+0000", 2024-02-13T14:09:42.112 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-13T14:09:42.112 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout }, 2024-02-13T14:09:42.112 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-02-13T14:09:42.112 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout } 2024-02-13T14:09:42.112 INFO:teuthology.orchestra.run.smithi087.stdout:mgr not available, waiting (2/15)... 2024-02-13T14:09:42.491 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:09:42 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: 2024-02-13T14:09:42.270+0000 7f598fa65200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-13T14:09:42.951 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:09:42 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: 2024-02-13T14:09:42.537+0000 7f598fa65200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-13T14:09:42.951 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:09:42 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: 2024-02-13T14:09:42.770+0000 7f598fa65200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-13T14:09:43.991 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:09:43 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: 2024-02-13T14:09:43.572+0000 7f598fa65200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-13T14:09:44.469 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:09:44 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: 2024-02-13T14:09:44.154+0000 7f598fa65200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-13T14:09:44.469 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:09:44 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: 2024-02-13T14:09:44.250+0000 7f598fa65200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-13T14:09:44.469 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:09:44 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: 2024-02-13T14:09:44.467+0000 7f598fa65200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-13T14:09:44.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:44 smithi087 ceph-mon[27372]: Activating manager daemon y 2024-02-13T14:09:44.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:44 smithi087 ceph-mon[27372]: mgrmap e2: y(active, starting, since 0.00288711s) 2024-02-13T14:09:44.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:44 smithi087 ceph-mon[27372]: from='mgr.14100 172.21.15.87:0/2816976806' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-13T14:09:44.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:44 smithi087 ceph-mon[27372]: from='mgr.14100 172.21.15.87:0/2816976806' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-13T14:09:44.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:44 smithi087 ceph-mon[27372]: from='mgr.14100 172.21.15.87:0/2816976806' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-13T14:09:44.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:44 smithi087 ceph-mon[27372]: from='mgr.14100 172.21.15.87:0/2816976806' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-13T14:09:44.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:44 smithi087 ceph-mon[27372]: from='mgr.14100 172.21.15.87:0/2816976806' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-13T14:09:44.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:44 smithi087 ceph-mon[27372]: Manager daemon y is now available 2024-02-13T14:09:44.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:44 smithi087 ceph-mon[27372]: from='mgr.14100 172.21.15.87:0/2816976806' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-13T14:09:44.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:44 smithi087 ceph-mon[27372]: from='mgr.14100 172.21.15.87:0/2816976806' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-13T14:09:44.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:44 smithi087 ceph-mon[27372]: from='mgr.14100 172.21.15.87:0/2816976806' entity='mgr.y' 2024-02-13T14:09:44.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:44 smithi087 ceph-mon[27372]: from='mgr.14100 172.21.15.87:0/2816976806' entity='mgr.y' 2024-02-13T14:09:44.743 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:44 smithi087 ceph-mon[27372]: from='mgr.14100 172.21.15.87:0/2816976806' entity='mgr.y' 2024-02-13T14:09:46.741 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:46 smithi087 ceph-mon[27372]: mgrmap e3: y(active, since 1.00722s) 2024-02-13T14:09:47.741 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:47 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/1966512736' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-13T14:09:48.726 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:48 smithi087 ceph-mon[27372]: mgrmap e4: y(active, since 3s) 2024-02-13T14:09:53.117 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout 2024-02-13T14:09:53.117 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout { 2024-02-13T14:09:53.118 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "fsid": "5c04b074-ca79-11ee-95b9-87774f69a715", 2024-02-13T14:09:53.118 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "health": { 2024-02-13T14:09:53.118 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-02-13T14:09:53.118 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-02-13T14:09:53.118 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-02-13T14:09:53.118 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout }, 2024-02-13T14:09:53.118 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-02-13T14:09:53.118 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-02-13T14:09:53.118 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout 0 2024-02-13T14:09:53.118 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout ], 2024-02-13T14:09:53.118 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-02-13T14:09:53.118 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "a" 2024-02-13T14:09:53.118 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout ], 2024-02-13T14:09:53.119 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "quorum_age": 11, 2024-02-13T14:09:53.119 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "monmap": { 2024-02-13T14:09:53.119 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-13T14:09:53.119 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-02-13T14:09:53.119 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-02-13T14:09:53.119 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout }, 2024-02-13T14:09:53.119 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-02-13T14:09:53.119 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-13T14:09:53.119 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-02-13T14:09:53.119 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-02-13T14:09:53.119 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-02-13T14:09:53.119 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-02-13T14:09:53.119 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-02-13T14:09:53.119 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-02-13T14:09:53.119 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout }, 2024-02-13T14:09:53.119 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-02-13T14:09:53.120 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-02-13T14:09:53.120 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-02-13T14:09:53.120 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-02-13T14:09:53.120 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-02-13T14:09:53.120 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-02-13T14:09:53.120 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-02-13T14:09:53.120 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-02-13T14:09:53.120 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-02-13T14:09:53.120 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout }, 2024-02-13T14:09:53.120 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-02-13T14:09:53.120 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-13T14:09:53.120 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-02-13T14:09:53.120 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-02-13T14:09:53.120 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout }, 2024-02-13T14:09:53.121 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-02-13T14:09:53.121 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "available": true, 2024-02-13T14:09:53.121 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-02-13T14:09:53.121 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "modules": [ 2024-02-13T14:09:53.121 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "iostat", 2024-02-13T14:09:53.121 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "nfs", 2024-02-13T14:09:53.121 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "restful" 2024-02-13T14:09:53.121 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout ], 2024-02-13T14:09:53.121 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-13T14:09:53.121 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout }, 2024-02-13T14:09:53.121 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-02-13T14:09:53.122 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-13T14:09:53.122 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "modified": "2024-02-13T14:09:30.938724+0000", 2024-02-13T14:09:53.122 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-13T14:09:53.122 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout }, 2024-02-13T14:09:53.122 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-02-13T14:09:53.122 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout } 2024-02-13T14:09:53.122 INFO:teuthology.orchestra.run.smithi087.stdout:mgr is available 2024-02-13T14:09:54.046 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:53 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/106562037' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-02-13T14:09:54.326 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout 2024-02-13T14:09:54.326 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout [global] 2024-02-13T14:09:54.326 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout fsid = 5c04b074-ca79-11ee-95b9-87774f69a715 2024-02-13T14:09:54.326 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-02-13T14:09:54.326 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-02-13T14:09:54.326 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-02-13T14:09:54.326 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-02-13T14:09:54.326 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout 2024-02-13T14:09:54.326 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout [mgr] 2024-02-13T14:09:54.327 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-02-13T14:09:54.327 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout 2024-02-13T14:09:54.327 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout [osd] 2024-02-13T14:09:54.327 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-02-13T14:09:54.327 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-02-13T14:09:54.327 INFO:teuthology.orchestra.run.smithi087.stdout:Enabling cephadm module... 2024-02-13T14:09:55.378 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:55 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/3131271767' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-02-13T14:09:55.378 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:09:55 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: ignoring --setuser ceph since I am not root 2024-02-13T14:09:55.378 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:09:55 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: ignoring --setgroup ceph since I am not root 2024-02-13T14:09:55.670 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:09:55 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: 2024-02-13T14:09:55.624+0000 7fa219e1b200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-13T14:09:55.991 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:09:55 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: 2024-02-13T14:09:55.720+0000 7fa219e1b200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-13T14:09:56.376 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:56 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/3131271767' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-02-13T14:09:56.376 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:56 smithi087 ceph-mon[27372]: mgrmap e5: y(active, since 10s) 2024-02-13T14:09:56.700 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:09:56 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: 2024-02-13T14:09:56.373+0000 7fa219e1b200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-13T14:09:56.700 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:09:56 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: 2024-02-13T14:09:56.472+0000 7fa219e1b200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-13T14:09:56.700 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:09:56 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: 2024-02-13T14:09:56.612+0000 7fa219e1b200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-13T14:09:56.929 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout { 2024-02-13T14:09:56.929 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-02-13T14:09:56.929 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "available": true, 2024-02-13T14:09:56.929 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "active_name": "y", 2024-02-13T14:09:56.929 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-02-13T14:09:56.929 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout } 2024-02-13T14:09:56.929 INFO:teuthology.orchestra.run.smithi087.stdout:Waiting for the mgr to restart... 2024-02-13T14:09:56.930 INFO:teuthology.orchestra.run.smithi087.stdout:Waiting for mgr epoch 5... 2024-02-13T14:09:56.991 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:09:56 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: 2024-02-13T14:09:56.709+0000 7fa219e1b200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-13T14:09:56.991 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:09:56 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: 2024-02-13T14:09:56.791+0000 7fa219e1b200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-13T14:09:57.361 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:09:57 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/755644176' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-02-13T14:09:57.361 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:09:57 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: 2024-02-13T14:09:57.056+0000 7fa219e1b200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-13T14:09:57.362 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:09:57 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: 2024-02-13T14:09:57.170+0000 7fa219e1b200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-13T14:09:57.362 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:09:57 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: 2024-02-13T14:09:57.260+0000 7fa219e1b200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-13T14:09:57.362 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:09:57 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: 2024-02-13T14:09:57.359+0000 7fa219e1b200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-13T14:09:57.741 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:09:57 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: 2024-02-13T14:09:57.537+0000 7fa219e1b200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-13T14:09:58.949 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:09:58 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: 2024-02-13T14:09:58.490+0000 7fa219e1b200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-13T14:09:58.949 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:09:58 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: 2024-02-13T14:09:58.618+0000 7fa219e1b200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-13T14:09:58.949 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:09:58 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: 2024-02-13T14:09:58.947+0000 7fa219e1b200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-13T14:09:59.237 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:09:59 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: 2024-02-13T14:09:59.037+0000 7fa219e1b200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-13T14:09:59.237 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:09:59 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: 2024-02-13T14:09:59.235+0000 7fa219e1b200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-13T14:09:59.491 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:09:59 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: 2024-02-13T14:09:59.418+0000 7fa219e1b200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-13T14:09:59.966 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:09:59 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: 2024-02-13T14:09:59.705+0000 7fa219e1b200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-13T14:09:59.966 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:09:59 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: 2024-02-13T14:09:59.964+0000 7fa219e1b200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-13T14:10:00.241 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:10:00 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: 2024-02-13T14:10:00.199+0000 7fa219e1b200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-13T14:10:01.491 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:10:00 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: 2024-02-13T14:10:00.991+0000 7fa219e1b200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-13T14:10:01.875 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:10:01 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: 2024-02-13T14:10:01.557+0000 7fa219e1b200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-13T14:10:01.875 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:10:01 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: 2024-02-13T14:10:01.655+0000 7fa219e1b200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-13T14:10:01.875 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:10:01 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: 2024-02-13T14:10:01.873+0000 7fa219e1b200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-13T14:10:02.241 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:01 smithi087 ceph-mon[27372]: Active manager daemon y restarted 2024-02-13T14:10:02.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:01 smithi087 ceph-mon[27372]: Activating manager daemon y 2024-02-13T14:10:02.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:01 smithi087 ceph-mon[27372]: osdmap e2: 0 total, 0 up, 0 in 2024-02-13T14:10:02.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:01 smithi087 ceph-mon[27372]: mgrmap e6: y(active, starting, since 0.00330553s) 2024-02-13T14:10:02.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:01 smithi087 ceph-mon[27372]: from='mgr.14118 172.21.15.87:0/688123235' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-13T14:10:02.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:01 smithi087 ceph-mon[27372]: from='mgr.14118 172.21.15.87:0/688123235' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-13T14:10:02.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:01 smithi087 ceph-mon[27372]: from='mgr.14118 172.21.15.87:0/688123235' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-13T14:10:02.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:01 smithi087 ceph-mon[27372]: from='mgr.14118 172.21.15.87:0/688123235' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-13T14:10:02.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:01 smithi087 ceph-mon[27372]: from='mgr.14118 172.21.15.87:0/688123235' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-13T14:10:02.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:01 smithi087 ceph-mon[27372]: Manager daemon y is now available 2024-02-13T14:10:02.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:01 smithi087 ceph-mon[27372]: from='mgr.14118 172.21.15.87:0/688123235' entity='mgr.y' 2024-02-13T14:10:02.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:01 smithi087 ceph-mon[27372]: from='mgr.14118 172.21.15.87:0/688123235' entity='mgr.y' 2024-02-13T14:10:02.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:01 smithi087 ceph-mon[27372]: from='mgr.14118 172.21.15.87:0/688123235' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:10:02.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:01 smithi087 ceph-mon[27372]: from='mgr.14118 172.21.15.87:0/688123235' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:10:02.243 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:01 smithi087 ceph-mon[27372]: from='mgr.14118 172.21.15.87:0/688123235' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:10:03.224 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:02 smithi087 ceph-mon[27372]: Found migration_current of "None". Setting to last migration. 2024-02-13T14:10:03.224 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:02 smithi087 ceph-mon[27372]: from='mgr.14118 172.21.15.87:0/688123235' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-13T14:10:03.224 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:02 smithi087 ceph-mon[27372]: from='mgr.14118 172.21.15.87:0/688123235' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-13T14:10:03.224 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:02 smithi087 ceph-mon[27372]: from='mgr.14118 172.21.15.87:0/688123235' entity='mgr.y' 2024-02-13T14:10:03.224 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:02 smithi087 ceph-mon[27372]: from='mgr.14118 172.21.15.87:0/688123235' entity='mgr.y' 2024-02-13T14:10:03.224 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:02 smithi087 ceph-mon[27372]: mgrmap e7: y(active, since 1.0055s) 2024-02-13T14:10:03.460 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout { 2024-02-13T14:10:03.460 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-02-13T14:10:03.460 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "initialized": true 2024-02-13T14:10:03.461 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout } 2024-02-13T14:10:03.461 INFO:teuthology.orchestra.run.smithi087.stdout:mgr epoch 5 is available 2024-02-13T14:10:03.461 INFO:teuthology.orchestra.run.smithi087.stdout:Setting orchestrator backend to cephadm... 2024-02-13T14:10:04.121 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:03 smithi087 ceph-mon[27372]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-02-13T14:10:04.121 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:03 smithi087 ceph-mon[27372]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-02-13T14:10:04.121 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:03 smithi087 ceph-mon[27372]: from='mgr.14118 172.21.15.87:0/688123235' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:10:05.241 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:04 smithi087 ceph-mon[27372]: [13/Feb/2024:14:10:03] ENGINE Bus STARTING 2024-02-13T14:10:05.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:04 smithi087 ceph-mon[27372]: [13/Feb/2024:14:10:03] ENGINE Serving on https://172.21.15.87:7150 2024-02-13T14:10:05.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:04 smithi087 ceph-mon[27372]: [13/Feb/2024:14:10:03] ENGINE Serving on http://172.21.15.87:8765 2024-02-13T14:10:05.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:04 smithi087 ceph-mon[27372]: [13/Feb/2024:14:10:03] ENGINE Bus STARTED 2024-02-13T14:10:05.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:04 smithi087 ceph-mon[27372]: mgrmap e8: y(active, since 2s) 2024-02-13T14:10:05.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:04 smithi087 ceph-mon[27372]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T14:10:05.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:04 smithi087 ceph-mon[27372]: from='mgr.14118 172.21.15.87:0/688123235' entity='mgr.y' 2024-02-13T14:10:05.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:04 smithi087 ceph-mon[27372]: from='mgr.14118 172.21.15.87:0/688123235' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:10:05.864 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout value unchanged 2024-02-13T14:10:05.864 INFO:teuthology.orchestra.run.smithi087.stdout:Generating ssh key... 2024-02-13T14:10:06.491 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:06 smithi087 ceph-mon[27372]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T14:10:07.092 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:10:06 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: Generating public/private rsa key pair. 2024-02-13T14:10:07.092 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:10:06 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: Your identification has been saved in /tmp/tmpec3bqtip/key. 2024-02-13T14:10:07.092 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:10:06 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: Your public key has been saved in /tmp/tmpec3bqtip/key.pub. 2024-02-13T14:10:07.092 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:10:06 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: The key fingerprint is: 2024-02-13T14:10:07.093 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:10:06 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: SHA256:LgGkVIhMRD/uf2FxoIV9DrbY/W68GN9gHs9MOB+6saY ceph-5c04b074-ca79-11ee-95b9-87774f69a715 2024-02-13T14:10:07.093 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:10:06 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: The key's randomart image is: 2024-02-13T14:10:07.093 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:10:06 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: +---[RSA 3072]----+ 2024-02-13T14:10:07.093 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:10:06 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: |==.oo o | 2024-02-13T14:10:07.093 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:10:06 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: |.ooo . * . | 2024-02-13T14:10:07.093 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:10:06 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: | .o. * B | 2024-02-13T14:10:07.093 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:10:06 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: | . .+ + + | 2024-02-13T14:10:07.093 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:10:06 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: | . . S . | 2024-02-13T14:10:07.093 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:10:06 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: | . = .. | 2024-02-13T14:10:07.093 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:10:06 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: | . o o.oB o | 2024-02-13T14:10:07.093 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:10:06 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: | . o *=# . | 2024-02-13T14:10:07.093 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:10:06 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: | .. E+Bo* | 2024-02-13T14:10:07.093 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:10:06 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: +----[SHA256]-----+ 2024-02-13T14:10:07.975 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:07 smithi087 ceph-mon[27372]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T14:10:07.975 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:07 smithi087 ceph-mon[27372]: Generating ssh key... 2024-02-13T14:10:07.975 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:07 smithi087 ceph-mon[27372]: from='mgr.14118 172.21.15.87:0/688123235' entity='mgr.y' 2024-02-13T14:10:07.975 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:07 smithi087 ceph-mon[27372]: from='mgr.14118 172.21.15.87:0/688123235' entity='mgr.y' 2024-02-13T14:10:08.450 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCrci0y2aenDRwIL6mNGtcl7UkFQkH1U/HvXX4qxosZUHCJx2ei6HOepTGTjYj1o3A43vZTeYELtx1/Vdp11hYkGqjpo/EFVj9WRv+SYkFDfSWLRhG7BmubGENuH0LdOqmxq+pcpZ+rfmJcVmUUFeYPDlms45IJglVeacrUbHEoZnP1N91/QIgkwkgZlxDetrpXwQ1OScTR8LgTDmWh9/9L751+nywEe5f2kTh7RckQMS4HIz+3qJNBLARzmQpfHxBI7LmkCmL5EYsGLwuz+hvgZgdrSUc1LQXZy4LzOFEvKqmSG3L5Dg5wIXK4sK4JrTxriCbU4M700MXphxXMGaADAmO8/6IAIsfe+LgLfCH19vQ9kTIntqseZOeCqd73dzroxntn6KdeqOS5O1Wadojpo0FZbthaOFFKrAS8PAE6A6BXiCrArsFDQBtQjd59K4ZT1dYhxlFJSsqd2Dy5TP/YtGSEVqPQCwX+h+syVxK3w1fyQLJzURydXPIwyuaQ4yc= ceph-5c04b074-ca79-11ee-95b9-87774f69a715 2024-02-13T14:10:08.451 INFO:teuthology.orchestra.run.smithi087.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-02-13T14:10:08.451 INFO:teuthology.orchestra.run.smithi087.stdout:Adding key to root@localhost authorized_keys... 2024-02-13T14:10:08.451 INFO:teuthology.orchestra.run.smithi087.stdout:Adding host smithi087... 2024-02-13T14:10:08.915 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:08 smithi087 ceph-mon[27372]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T14:10:09.991 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:09 smithi087 ceph-mon[27372]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi087", "addr": "172.21.15.87", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T14:10:11.241 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:10 smithi087 ceph-mon[27372]: Deploying cephadm binary to smithi087 2024-02-13T14:10:13.992 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout Added host 'smithi087' with addr '172.21.15.87' 2024-02-13T14:10:13.992 INFO:teuthology.orchestra.run.smithi087.stdout:Deploying unmanaged mon service... 2024-02-13T14:10:14.708 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:14 smithi087 ceph-mon[27372]: from='mgr.14118 172.21.15.87:0/688123235' entity='mgr.y' 2024-02-13T14:10:14.708 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:14 smithi087 ceph-mon[27372]: Added host smithi087 2024-02-13T14:10:14.708 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:14 smithi087 ceph-mon[27372]: from='mgr.14118 172.21.15.87:0/688123235' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:10:15.991 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:15 smithi087 ceph-mon[27372]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T14:10:15.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:15 smithi087 ceph-mon[27372]: Saving service mon spec with placement count:5 2024-02-13T14:10:15.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:15 smithi087 ceph-mon[27372]: from='mgr.14118 172.21.15.87:0/688123235' entity='mgr.y' 2024-02-13T14:10:16.021 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-02-13T14:10:16.021 INFO:teuthology.orchestra.run.smithi087.stdout:Deploying unmanaged mgr service... 2024-02-13T14:10:17.618 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:17 smithi087 ceph-mon[27372]: from='mgr.14118 172.21.15.87:0/688123235' entity='mgr.y' 2024-02-13T14:10:17.618 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:17 smithi087 ceph-mon[27372]: from='mgr.14118 172.21.15.87:0/688123235' entity='mgr.y' 2024-02-13T14:10:17.618 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:17 smithi087 ceph-mon[27372]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T14:10:17.618 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:17 smithi087 ceph-mon[27372]: Saving service mgr spec with placement count:2 2024-02-13T14:10:17.618 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:17 smithi087 ceph-mon[27372]: from='mgr.14118 172.21.15.87:0/688123235' entity='mgr.y' 2024-02-13T14:10:17.620 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-02-13T14:10:19.594 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:19 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/2184862387' entity='client.admin' 2024-02-13T14:10:20.755 INFO:teuthology.orchestra.run.smithi087.stdout:Enabling the dashboard module... 2024-02-13T14:10:20.991 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:20 smithi087 ceph-mon[27372]: from='mgr.14118 172.21.15.87:0/688123235' entity='mgr.y' 2024-02-13T14:10:20.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:20 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/1803307015' entity='client.admin' 2024-02-13T14:10:22.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:22 smithi087 ceph-mon[27372]: from='mgr.14118 172.21.15.87:0/688123235' entity='mgr.y' 2024-02-13T14:10:22.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:22 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/2370854413' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-02-13T14:10:22.492 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:10:22 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: ignoring --setuser ceph since I am not root 2024-02-13T14:10:22.492 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:10:22 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: ignoring --setgroup ceph since I am not root 2024-02-13T14:10:22.972 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:10:22 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: 2024-02-13T14:10:22.644+0000 7ff35336f200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-13T14:10:22.972 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:10:22 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: 2024-02-13T14:10:22.737+0000 7ff35336f200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-13T14:10:23.241 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:23 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/2370854413' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-02-13T14:10:23.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:23 smithi087 ceph-mon[27372]: mgrmap e9: y(active, since 20s) 2024-02-13T14:10:23.533 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:10:23 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: 2024-02-13T14:10:23.381+0000 7ff35336f200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-13T14:10:23.534 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:10:23 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: 2024-02-13T14:10:23.474+0000 7ff35336f200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-13T14:10:23.805 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:10:23 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: 2024-02-13T14:10:23.620+0000 7ff35336f200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-13T14:10:23.805 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:10:23 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: 2024-02-13T14:10:23.717+0000 7ff35336f200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-13T14:10:23.805 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:10:23 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: 2024-02-13T14:10:23.803+0000 7ff35336f200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-13T14:10:24.369 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:24 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/3313244385' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-02-13T14:10:24.369 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:10:24 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: 2024-02-13T14:10:24.070+0000 7ff35336f200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-13T14:10:24.369 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:10:24 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: 2024-02-13T14:10:24.184+0000 7ff35336f200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-13T14:10:24.369 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:10:24 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: 2024-02-13T14:10:24.270+0000 7ff35336f200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-13T14:10:24.370 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:10:24 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: 2024-02-13T14:10:24.367+0000 7ff35336f200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-13T14:10:24.696 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:10:24 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: 2024-02-13T14:10:24.543+0000 7ff35336f200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-13T14:10:24.699 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout { 2024-02-13T14:10:24.699 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-02-13T14:10:24.699 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "available": true, 2024-02-13T14:10:24.699 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "active_name": "y", 2024-02-13T14:10:24.699 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-02-13T14:10:24.699 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout } 2024-02-13T14:10:24.699 INFO:teuthology.orchestra.run.smithi087.stdout:Waiting for the mgr to restart... 2024-02-13T14:10:24.699 INFO:teuthology.orchestra.run.smithi087.stdout:Waiting for mgr epoch 9... 2024-02-13T14:10:25.973 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:10:25 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: 2024-02-13T14:10:25.514+0000 7ff35336f200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-13T14:10:25.974 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:10:25 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: 2024-02-13T14:10:25.644+0000 7ff35336f200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-13T14:10:25.974 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:10:25 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: 2024-02-13T14:10:25.972+0000 7ff35336f200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-13T14:10:26.241 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:10:26 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: 2024-02-13T14:10:26.061+0000 7ff35336f200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-13T14:10:26.717 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:10:26 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: 2024-02-13T14:10:26.253+0000 7ff35336f200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-13T14:10:26.717 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:10:26 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: 2024-02-13T14:10:26.434+0000 7ff35336f200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-13T14:10:26.971 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:10:26 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: 2024-02-13T14:10:26.714+0000 7ff35336f200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-13T14:10:26.971 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:10:26 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: 2024-02-13T14:10:26.969+0000 7ff35336f200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-13T14:10:27.241 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:10:27 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: 2024-02-13T14:10:27.202+0000 7ff35336f200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-13T14:10:28.241 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:10:27 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: 2024-02-13T14:10:27.988+0000 7ff35336f200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-13T14:10:28.864 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:10:28 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: 2024-02-13T14:10:28.551+0000 7ff35336f200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-13T14:10:28.864 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:10:28 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: 2024-02-13T14:10:28.648+0000 7ff35336f200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-13T14:10:28.864 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:10:28 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: 2024-02-13T14:10:28.862+0000 7ff35336f200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-13T14:10:29.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:28 smithi087 ceph-mon[27372]: Active manager daemon y restarted 2024-02-13T14:10:29.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:28 smithi087 ceph-mon[27372]: Activating manager daemon y 2024-02-13T14:10:29.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:28 smithi087 ceph-mon[27372]: osdmap e3: 0 total, 0 up, 0 in 2024-02-13T14:10:29.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:28 smithi087 ceph-mon[27372]: mgrmap e10: y(active, starting, since 0.00320052s) 2024-02-13T14:10:29.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:28 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-13T14:10:29.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:28 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-13T14:10:29.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:28 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-13T14:10:29.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:28 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-13T14:10:29.243 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:28 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-13T14:10:29.243 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:28 smithi087 ceph-mon[27372]: Manager daemon y is now available 2024-02-13T14:10:29.243 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:28 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:10:29.243 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:28 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:10:30.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:29 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-13T14:10:30.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:29 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-13T14:10:30.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:29 smithi087 ceph-mon[27372]: mgrmap e11: y(active, since 1.00544s) 2024-02-13T14:10:30.851 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout { 2024-02-13T14:10:30.852 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-02-13T14:10:30.852 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "initialized": true 2024-02-13T14:10:30.852 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout } 2024-02-13T14:10:30.852 INFO:teuthology.orchestra.run.smithi087.stdout:mgr epoch 9 is available 2024-02-13T14:10:30.852 INFO:teuthology.orchestra.run.smithi087.stdout:Generating a dashboard self-signed certificate... 2024-02-13T14:10:31.127 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:30 smithi087 ceph-mon[27372]: [13/Feb/2024:14:10:29] ENGINE Bus STARTING 2024-02-13T14:10:31.128 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:30 smithi087 ceph-mon[27372]: [13/Feb/2024:14:10:29] ENGINE Serving on http://172.21.15.87:8765 2024-02-13T14:10:31.128 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:30 smithi087 ceph-mon[27372]: [13/Feb/2024:14:10:29] ENGINE Serving on https://172.21.15.87:7150 2024-02-13T14:10:31.128 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:30 smithi087 ceph-mon[27372]: [13/Feb/2024:14:10:29] ENGINE Bus STARTED 2024-02-13T14:10:31.128 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:30 smithi087 ceph-mon[27372]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-02-13T14:10:31.128 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:30 smithi087 ceph-mon[27372]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-02-13T14:10:31.128 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:30 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:10:32.373 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-02-13T14:10:32.374 INFO:teuthology.orchestra.run.smithi087.stdout:Creating initial admin user... 2024-02-13T14:10:32.715 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:32 smithi087 ceph-mon[27372]: mgrmap e12: y(active, since 2s) 2024-02-13T14:10:32.715 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:32 smithi087 ceph-mon[27372]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T14:10:32.715 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:32 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:10:32.715 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:32 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:10:32.715 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:32 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:10:33.980 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:33 smithi087 ceph-mon[27372]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T14:10:33.980 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:33 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:10:34.402 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$NE2QubkJeLRK3BV25j1VsuhAhJHokAUOVbW/iC5WwMmrlzjrPRt4C", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1707833433, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-02-13T14:10:34.402 INFO:teuthology.orchestra.run.smithi087.stdout:Fetching dashboard port number... 2024-02-13T14:10:35.616 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout 8443 2024-02-13T14:10:35.625 INFO:teuthology.orchestra.run.smithi087.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-02-13T14:10:35.625 INFO:teuthology.orchestra.run.smithi087.stdout:systemctl: stdout disabled 2024-02-13T14:10:35.632 INFO:teuthology.orchestra.run.smithi087.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-02-13T14:10:35.632 INFO:teuthology.orchestra.run.smithi087.stdout:systemctl: stdout inactive 2024-02-13T14:10:35.632 INFO:teuthology.orchestra.run.smithi087.stdout:firewalld.service is not enabled 2024-02-13T14:10:35.632 INFO:teuthology.orchestra.run.smithi087.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-02-13T14:10:35.634 INFO:teuthology.orchestra.run.smithi087.stdout:Ceph Dashboard is now available at: 2024-02-13T14:10:35.634 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-02-13T14:10:35.634 INFO:teuthology.orchestra.run.smithi087.stdout: URL: https://smithi087.front.sepia.ceph.com:8443/ 2024-02-13T14:10:35.634 INFO:teuthology.orchestra.run.smithi087.stdout: User: admin 2024-02-13T14:10:35.634 INFO:teuthology.orchestra.run.smithi087.stdout: Password: x53pm2a7dw 2024-02-13T14:10:35.634 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-02-13T14:10:35.635 INFO:teuthology.orchestra.run.smithi087.stdout:Saving cluster configuration to /var/lib/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/config directory 2024-02-13T14:10:35.635 INFO:teuthology.orchestra.run.smithi087.stdout:Enabling autotune for osd_memory_target 2024-02-13T14:10:36.241 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:35 smithi087 ceph-mon[27372]: mgrmap e13: y(active, since 6s) 2024-02-13T14:10:36.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:35 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/2803677208' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-02-13T14:10:38.271 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-02-13T14:10:38.271 INFO:teuthology.orchestra.run.smithi087.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-02-13T14:10:38.271 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-02-13T14:10:38.271 INFO:teuthology.orchestra.run.smithi087.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 5c04b074-ca79-11ee-95b9-87774f69a715 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-02-13T14:10:38.271 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-02-13T14:10:38.271 INFO:teuthology.orchestra.run.smithi087.stdout:Or, if you are only running a single cluster on this host: 2024-02-13T14:10:38.271 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-02-13T14:10:38.271 INFO:teuthology.orchestra.run.smithi087.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-02-13T14:10:38.271 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-02-13T14:10:38.272 INFO:teuthology.orchestra.run.smithi087.stdout:Please consider enabling telemetry to help improve Ceph: 2024-02-13T14:10:38.272 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-02-13T14:10:38.272 INFO:teuthology.orchestra.run.smithi087.stdout: ceph telemetry on 2024-02-13T14:10:38.272 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-02-13T14:10:38.272 INFO:teuthology.orchestra.run.smithi087.stdout:For more information see: 2024-02-13T14:10:38.272 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-02-13T14:10:38.272 INFO:teuthology.orchestra.run.smithi087.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-02-13T14:10:38.272 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-02-13T14:10:38.272 INFO:teuthology.orchestra.run.smithi087.stdout:Bootstrap complete. 2024-02-13T14:10:38.314 INFO:tasks.cephadm:Fetching config... 2024-02-13T14:10:38.315 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-02-13T14:10:38.315 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-02-13T14:10:38.330 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-02-13T14:10:38.330 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-02-13T14:10:38.331 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-02-13T14:10:38.385 INFO:tasks.cephadm:Fetching mon keyring... 2024-02-13T14:10:38.385 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-02-13T14:10:38.386 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd if=/var/lib/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/mon.a/keyring of=/dev/stdout 2024-02-13T14:10:38.452 INFO:tasks.cephadm:Fetching pub ssh key... 2024-02-13T14:10:38.452 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-02-13T14:10:38.452 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-02-13T14:10:38.507 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-02-13T14:10:38.507 DEBUG:teuthology.orchestra.run.smithi087:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCrci0y2aenDRwIL6mNGtcl7UkFQkH1U/HvXX4qxosZUHCJx2ei6HOepTGTjYj1o3A43vZTeYELtx1/Vdp11hYkGqjpo/EFVj9WRv+SYkFDfSWLRhG7BmubGENuH0LdOqmxq+pcpZ+rfmJcVmUUFeYPDlms45IJglVeacrUbHEoZnP1N91/QIgkwkgZlxDetrpXwQ1OScTR8LgTDmWh9/9L751+nywEe5f2kTh7RckQMS4HIz+3qJNBLARzmQpfHxBI7LmkCmL5EYsGLwuz+hvgZgdrSUc1LQXZy4LzOFEvKqmSG3L5Dg5wIXK4sK4JrTxriCbU4M700MXphxXMGaADAmO8/6IAIsfe+LgLfCH19vQ9kTIntqseZOeCqd73dzroxntn6KdeqOS5O1Wadojpo0FZbthaOFFKrAS8PAE6A6BXiCrArsFDQBtQjd59K4ZT1dYhxlFJSsqd2Dy5TP/YtGSEVqPQCwX+h+syVxK3w1fyQLJzURydXPIwyuaQ4yc= ceph-5c04b074-ca79-11ee-95b9-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-02-13T14:10:38.601 INFO:teuthology.orchestra.run.smithi087.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCrci0y2aenDRwIL6mNGtcl7UkFQkH1U/HvXX4qxosZUHCJx2ei6HOepTGTjYj1o3A43vZTeYELtx1/Vdp11hYkGqjpo/EFVj9WRv+SYkFDfSWLRhG7BmubGENuH0LdOqmxq+pcpZ+rfmJcVmUUFeYPDlms45IJglVeacrUbHEoZnP1N91/QIgkwkgZlxDetrpXwQ1OScTR8LgTDmWh9/9L751+nywEe5f2kTh7RckQMS4HIz+3qJNBLARzmQpfHxBI7LmkCmL5EYsGLwuz+hvgZgdrSUc1LQXZy4LzOFEvKqmSG3L5Dg5wIXK4sK4JrTxriCbU4M700MXphxXMGaADAmO8/6IAIsfe+LgLfCH19vQ9kTIntqseZOeCqd73dzroxntn6KdeqOS5O1Wadojpo0FZbthaOFFKrAS8PAE6A6BXiCrArsFDQBtQjd59K4ZT1dYhxlFJSsqd2Dy5TP/YtGSEVqPQCwX+h+syVxK3w1fyQLJzURydXPIwyuaQ4yc= ceph-5c04b074-ca79-11ee-95b9-87774f69a715 2024-02-13T14:10:38.616 DEBUG:teuthology.orchestra.run.smithi148:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCrci0y2aenDRwIL6mNGtcl7UkFQkH1U/HvXX4qxosZUHCJx2ei6HOepTGTjYj1o3A43vZTeYELtx1/Vdp11hYkGqjpo/EFVj9WRv+SYkFDfSWLRhG7BmubGENuH0LdOqmxq+pcpZ+rfmJcVmUUFeYPDlms45IJglVeacrUbHEoZnP1N91/QIgkwkgZlxDetrpXwQ1OScTR8LgTDmWh9/9L751+nywEe5f2kTh7RckQMS4HIz+3qJNBLARzmQpfHxBI7LmkCmL5EYsGLwuz+hvgZgdrSUc1LQXZy4LzOFEvKqmSG3L5Dg5wIXK4sK4JrTxriCbU4M700MXphxXMGaADAmO8/6IAIsfe+LgLfCH19vQ9kTIntqseZOeCqd73dzroxntn6KdeqOS5O1Wadojpo0FZbthaOFFKrAS8PAE6A6BXiCrArsFDQBtQjd59K4ZT1dYhxlFJSsqd2Dy5TP/YtGSEVqPQCwX+h+syVxK3w1fyQLJzURydXPIwyuaQ4yc= ceph-5c04b074-ca79-11ee-95b9-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-02-13T14:10:38.663 INFO:teuthology.orchestra.run.smithi148.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCrci0y2aenDRwIL6mNGtcl7UkFQkH1U/HvXX4qxosZUHCJx2ei6HOepTGTjYj1o3A43vZTeYELtx1/Vdp11hYkGqjpo/EFVj9WRv+SYkFDfSWLRhG7BmubGENuH0LdOqmxq+pcpZ+rfmJcVmUUFeYPDlms45IJglVeacrUbHEoZnP1N91/QIgkwkgZlxDetrpXwQ1OScTR8LgTDmWh9/9L751+nywEe5f2kTh7RckQMS4HIz+3qJNBLARzmQpfHxBI7LmkCmL5EYsGLwuz+hvgZgdrSUc1LQXZy4LzOFEvKqmSG3L5Dg5wIXK4sK4JrTxriCbU4M700MXphxXMGaADAmO8/6IAIsfe+LgLfCH19vQ9kTIntqseZOeCqd73dzroxntn6KdeqOS5O1Wadojpo0FZbthaOFFKrAS8PAE6A6BXiCrArsFDQBtQjd59K4ZT1dYhxlFJSsqd2Dy5TP/YtGSEVqPQCwX+h+syVxK3w1fyQLJzURydXPIwyuaQ4yc= ceph-5c04b074-ca79-11ee-95b9-87774f69a715 2024-02-13T14:10:38.678 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5c04b074-ca79-11ee-95b9-87774f69a715 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-02-13T14:10:38.917 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/mon.a/config 2024-02-13T14:10:38.991 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:38 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/2302756726' entity='client.admin' 2024-02-13T14:10:41.994 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-02-13T14:10:41.994 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5c04b074-ca79-11ee-95b9-87774f69a715 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-02-13T14:10:42.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:42 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/2904937684' entity='client.admin' 2024-02-13T14:10:42.719 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/mon.a/config 2024-02-13T14:10:43.917 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:43 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:10:43.917 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:43 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:10:43.917 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:43 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi087", "name": "osd_memory_target"}]: dispatch 2024-02-13T14:10:43.917 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:43 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:10:43.917 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:43 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:10:43.917 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:43 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:10:45.231 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi148 2024-02-13T14:10:45.231 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-02-13T14:10:45.231 DEBUG:teuthology.orchestra.run.smithi148:> dd of=/etc/ceph/ceph.conf 2024-02-13T14:10:45.249 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-02-13T14:10:45.249 DEBUG:teuthology.orchestra.run.smithi148:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-13T14:10:45.306 INFO:tasks.cephadm:Adding host smithi148 to orchestrator... 2024-02-13T14:10:45.306 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5c04b074-ca79-11ee-95b9-87774f69a715 -- ceph orch host add smithi148 2024-02-13T14:10:45.548 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/mon.a/config 2024-02-13T14:10:45.762 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:45 smithi087 ceph-mon[27372]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T14:10:45.762 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:45 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:10:45.762 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:45 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:10:45.762 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:45 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:10:45.762 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:45 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:10:45.762 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:45 smithi087 ceph-mon[27372]: Updating smithi087:/etc/ceph/ceph.conf 2024-02-13T14:10:45.762 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:45 smithi087 ceph-mon[27372]: Updating smithi087:/var/lib/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/config/ceph.conf 2024-02-13T14:10:45.762 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:45 smithi087 ceph-mon[27372]: Updating smithi087:/etc/ceph/ceph.client.admin.keyring 2024-02-13T14:10:45.762 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:45 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:10:45.762 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:45 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:10:45.763 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:45 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:10:46.625 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:46 smithi087 ceph-mon[27372]: Updating smithi087:/var/lib/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/config/ceph.client.admin.keyring 2024-02-13T14:10:48.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:48 smithi087 ceph-mon[27372]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi148", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T14:10:48.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:48 smithi087 ceph-mon[27372]: Deploying cephadm binary to smithi148 2024-02-13T14:10:49.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:49 smithi087 ceph-mon[27372]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:10:51.772 INFO:teuthology.orchestra.run.smithi087.stdout:Added host 'smithi148' with addr '172.21.15.148' 2024-02-13T14:10:52.089 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:51 smithi087 ceph-mon[27372]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:10:52.090 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:51 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:10:52.090 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:51 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:10:52.428 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5c04b074-ca79-11ee-95b9-87774f69a715 -- ceph orch host ls --format=json 2024-02-13T14:10:52.669 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/mon.a/config 2024-02-13T14:10:53.051 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:52 smithi087 ceph-mon[27372]: Added host smithi148 2024-02-13T14:10:53.051 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:52 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:10:53.051 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:52 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:10:54.241 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:53 smithi087 ceph-mon[27372]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:10:54.341 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-02-13T14:10:54.341 INFO:teuthology.orchestra.run.smithi087.stdout:[{"addr": "172.21.15.87", "hostname": "smithi087", "labels": [], "status": ""}, {"addr": "172.21.15.148", "hostname": "smithi148", "labels": [], "status": ""}] 2024-02-13T14:10:54.914 INFO:tasks.cephadm:Setting crush tunables to default 2024-02-13T14:10:54.914 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5c04b074-ca79-11ee-95b9-87774f69a715 -- ceph osd crush tunables default 2024-02-13T14:10:55.146 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/mon.a/config 2024-02-13T14:10:55.530 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:55 smithi087 ceph-mon[27372]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-13T14:10:55.530 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:55 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:10:55.530 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:55 smithi087 ceph-mon[27372]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:10:57.943 INFO:teuthology.orchestra.run.smithi087.stderr:adjusted tunables profile to default 2024-02-13T14:10:58.241 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:57 smithi087 ceph-mon[27372]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:10:58.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:57 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/2240416164' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-02-13T14:10:58.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:57 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:10:58.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:57 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:10:58.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:57 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:10:58.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:57 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:10:58.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:57 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi148", "name": "osd_memory_target"}]: dispatch 2024-02-13T14:10:58.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:57 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:10:58.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:57 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:10:58.649 INFO:tasks.cephadm:Adding mon.a on smithi087 2024-02-13T14:10:58.649 INFO:tasks.cephadm:Adding mon.c on smithi087 2024-02-13T14:10:58.649 INFO:tasks.cephadm:Adding mon.b on smithi148 2024-02-13T14:10:58.649 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5c04b074-ca79-11ee-95b9-87774f69a715 -- ceph orch apply mon '3;smithi087:172.21.15.87=a;smithi087:[v2:172.21.15.87:3301,v1:172.21.15.87:6790]=c;smithi148:172.21.15.148=b' 2024-02-13T14:10:58.878 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/config/ceph.conf 2024-02-13T14:10:59.241 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:58 smithi087 ceph-mon[27372]: Updating smithi148:/etc/ceph/ceph.conf 2024-02-13T14:10:59.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:58 smithi087 ceph-mon[27372]: Updating smithi148:/var/lib/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/config/ceph.conf 2024-02-13T14:10:59.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:58 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/2240416164' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-02-13T14:10:59.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:58 smithi087 ceph-mon[27372]: osdmap e4: 0 total, 0 up, 0 in 2024-02-13T14:10:59.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:58 smithi087 ceph-mon[27372]: Updating smithi148:/etc/ceph/ceph.client.admin.keyring 2024-02-13T14:10:59.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:58 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:10:59.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:58 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:10:59.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:58 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:00.241 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:59 smithi087 ceph-mon[27372]: Updating smithi148:/var/lib/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/config/ceph.client.admin.keyring 2024-02-13T14:11:00.241 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:10:59 smithi087 ceph-mon[27372]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:11:00.572 INFO:teuthology.orchestra.run.smithi148.stdout:Scheduled mon update... 2024-02-13T14:11:01.214 DEBUG:teuthology.orchestra.run.smithi087:mon.c> sudo journalctl -f -n 0 -u ceph-5c04b074-ca79-11ee-95b9-87774f69a715@mon.c.service 2024-02-13T14:11:01.217 DEBUG:teuthology.orchestra.run.smithi148:mon.b> sudo journalctl -f -n 0 -u ceph-5c04b074-ca79-11ee-95b9-87774f69a715@mon.b.service 2024-02-13T14:11:01.219 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-02-13T14:11:01.220 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5c04b074-ca79-11ee-95b9-87774f69a715 -- ceph mon dump -f json 2024-02-13T14:11:01.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:01 smithi087 ceph-mon[27372]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi087:172.21.15.87=a;smithi087:[v2:172.21.15.87:3301,v1:172.21.15.87:6790]=c;smithi148:172.21.15.148=b", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T14:11:01.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:01 smithi087 ceph-mon[27372]: Saving service mon spec with placement smithi087:172.21.15.87=a;smithi087:[v2:172.21.15.87:3301,v1:172.21.15.87:6790]=c;smithi148:172.21.15.148=b;count:3 2024-02-13T14:11:01.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:01 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:01.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:01 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:11:01.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:01 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:11:01.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:01 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:11:01.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:01 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:01.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:01 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-13T14:11:01.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:01 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:11:01.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:01 smithi087 ceph-mon[27372]: Deploying daemon mon.b on smithi148 2024-02-13T14:11:01.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:01 smithi087 ceph-mon[27372]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:11:02.246 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/config/ceph.conf 2024-02-13T14:11:04.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:03 smithi087 ceph-mon[27372]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:11:04.989 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:04 smithi148 systemd[1]: Starting Ceph mon.b for 5c04b074-ca79-11ee-95b9-87774f69a715... 2024-02-13T14:11:04.989 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-02-13T14:11:04.989 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":1,"fsid":"5c04b074-ca79-11ee-95b9-87774f69a715","modified":"2024-02-13T14:09:28.013951Z","created":"2024-02-13T14:09:28.013951Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:3300","nonce":0},{"type":"v1","addr":"172.21.15.87:6789","nonce":0}]},"addr":"172.21.15.87:6789/0","public_addr":"172.21.15.87:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-13T14:11:04.991 INFO:teuthology.orchestra.run.smithi148.stderr:dumped monmap epoch 1 2024-02-13T14:11:05.304 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 podman[31139]: 2024-02-13 14:11:04.996366875 +0000 UTC m=+0.018627199 image pull 28480620e5edd298d55c0e6673dcbad9ed23edd37269028a11063c1b8925676a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f 2024-02-13T14:11:05.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 podman[31139]: 2024-02-13 14:11:05.703001093 +0000 UTC m=+0.725261426 container create f226490f659a7511676a5098b1702367602d4eb76d1dae56d847f4f67b903696 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mon-b, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.2, org.label-schema.build-date=20240131, ceph=True, org.label-schema.license=GPLv2) 2024-02-13T14:11:06.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:05 smithi087 ceph-mon[27372]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:11:06.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:05 smithi087 ceph-mon[27372]: from='client.? 172.21.15.148:0/1298203649' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-13T14:11:06.315 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 podman[31139]: 2024-02-13 14:11:05.897558585 +0000 UTC m=+0.919818909 container init f226490f659a7511676a5098b1702367602d4eb76d1dae56d847f4f67b903696 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mon-b, io.buildah.version=1.33.2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240131, GIT_CLEAN=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-13T14:11:06.315 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 podman[31139]: 2024-02-13 14:11:05.903021475 +0000 UTC m=+0.925281801 container start f226490f659a7511676a5098b1702367602d4eb76d1dae56d847f4f67b903696 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mon-b, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, io.buildah.version=1.33.2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240131, GIT_CLEAN=True) 2024-02-13T14:11:06.315 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: set uid:gid to 167:167 (ceph:ceph) 2024-02-13T14:11:06.315 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: ceph version 19.0.0-1423-ge1b79d5a (e1b79d5a4193a484176863ab96c0ca28c00e1662) squid (dev), process ceph-mon, pid 7 2024-02-13T14:11:06.315 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: pidfile_write: ignore empty --pid-file 2024-02-13T14:11:06.315 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: load: jerasure load: lrc 2024-02-13T14:11:06.315 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: RocksDB version: 7.9.2 2024-02-13T14:11:06.316 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Git sha 0 2024-02-13T14:11:06.316 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Compile date 2024-02-12 22:40:01 2024-02-13T14:11:06.316 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: DB SUMMARY 2024-02-13T14:11:06.316 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: DB Session ID: FNY1YURSCWD1KBX8R10T 2024-02-13T14:11:06.316 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: CURRENT file: CURRENT 2024-02-13T14:11:06.316 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: IDENTITY file: IDENTITY 2024-02-13T14:11:06.316 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-02-13T14:11:06.316 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-02-13T14:11:06.316 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 511 ; 2024-02-13T14:11:06.316 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.error_if_exists: 0 2024-02-13T14:11:06.316 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.create_if_missing: 0 2024-02-13T14:11:06.316 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.paranoid_checks: 1 2024-02-13T14:11:06.316 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.flush_verify_memtable_count: 1 2024-02-13T14:11:06.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-02-13T14:11:06.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-02-13T14:11:06.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.env: 0x562e62127ae0 2024-02-13T14:11:06.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.fs: PosixFileSystem 2024-02-13T14:11:06.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.info_log: 0x562e648df8c0 2024-02-13T14:11:06.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.max_file_opening_threads: 16 2024-02-13T14:11:06.318 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.statistics: (nil) 2024-02-13T14:11:06.318 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.use_fsync: 0 2024-02-13T14:11:06.318 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.max_log_file_size: 0 2024-02-13T14:11:06.318 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-02-13T14:11:06.318 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.log_file_time_to_roll: 0 2024-02-13T14:11:06.318 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.keep_log_file_num: 1000 2024-02-13T14:11:06.318 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.recycle_log_file_num: 0 2024-02-13T14:11:06.318 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.allow_fallocate: 1 2024-02-13T14:11:06.318 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.allow_mmap_reads: 0 2024-02-13T14:11:06.318 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.allow_mmap_writes: 0 2024-02-13T14:11:06.319 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.use_direct_reads: 0 2024-02-13T14:11:06.319 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-02-13T14:11:06.319 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.create_missing_column_families: 0 2024-02-13T14:11:06.319 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.db_log_dir: 2024-02-13T14:11:06.319 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.wal_dir: 2024-02-13T14:11:06.319 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.table_cache_numshardbits: 6 2024-02-13T14:11:06.319 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.WAL_ttl_seconds: 0 2024-02-13T14:11:06.319 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.WAL_size_limit_MB: 0 2024-02-13T14:11:06.319 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-02-13T14:11:06.319 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-02-13T14:11:06.319 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.is_fd_close_on_exec: 1 2024-02-13T14:11:06.319 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.advise_random_on_open: 1 2024-02-13T14:11:06.320 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.db_write_buffer_size: 0 2024-02-13T14:11:06.320 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.write_buffer_manager: 0x562e648b3400 2024-02-13T14:11:06.320 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-02-13T14:11:06.320 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-02-13T14:11:06.320 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.use_adaptive_mutex: 0 2024-02-13T14:11:06.320 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.rate_limiter: (nil) 2024-02-13T14:11:06.320 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-02-13T14:11:06.320 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.wal_recovery_mode: 2 2024-02-13T14:11:06.320 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.enable_thread_tracking: 0 2024-02-13T14:11:06.320 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.enable_pipelined_write: 0 2024-02-13T14:11:06.320 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.unordered_write: 0 2024-02-13T14:11:06.320 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-02-13T14:11:06.321 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-02-13T14:11:06.321 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-02-13T14:11:06.321 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-02-13T14:11:06.321 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.row_cache: None 2024-02-13T14:11:06.321 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.wal_filter: None 2024-02-13T14:11:06.321 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-02-13T14:11:06.321 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.allow_ingest_behind: 0 2024-02-13T14:11:06.321 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.two_write_queues: 0 2024-02-13T14:11:06.321 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.manual_wal_flush: 0 2024-02-13T14:11:06.321 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.wal_compression: 0 2024-02-13T14:11:06.321 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.atomic_flush: 0 2024-02-13T14:11:06.321 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-02-13T14:11:06.321 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.persist_stats_to_disk: 0 2024-02-13T14:11:06.321 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.write_dbid_to_manifest: 0 2024-02-13T14:11:06.321 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.log_readahead_size: 0 2024-02-13T14:11:06.322 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-02-13T14:11:06.322 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.best_efforts_recovery: 0 2024-02-13T14:11:06.322 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-02-13T14:11:06.322 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-02-13T14:11:06.322 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.allow_data_in_errors: 0 2024-02-13T14:11:06.322 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.db_host_id: __hostname__ 2024-02-13T14:11:06.322 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.enforce_single_del_contracts: true 2024-02-13T14:11:06.322 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.max_background_jobs: 2 2024-02-13T14:11:06.322 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.max_background_compactions: -1 2024-02-13T14:11:06.322 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.max_subcompactions: 1 2024-02-13T14:11:06.322 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-02-13T14:11:06.322 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-02-13T14:11:06.323 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.delayed_write_rate : 16777216 2024-02-13T14:11:06.323 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.max_total_wal_size: 0 2024-02-13T14:11:06.323 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-02-13T14:11:06.323 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.stats_dump_period_sec: 600 2024-02-13T14:11:06.323 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.stats_persist_period_sec: 600 2024-02-13T14:11:06.323 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-02-13T14:11:06.323 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.max_open_files: -1 2024-02-13T14:11:06.323 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.bytes_per_sync: 0 2024-02-13T14:11:06.323 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.wal_bytes_per_sync: 0 2024-02-13T14:11:06.323 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.strict_bytes_per_sync: 0 2024-02-13T14:11:06.323 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.compaction_readahead_size: 0 2024-02-13T14:11:06.323 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.max_background_flushes: -1 2024-02-13T14:11:06.323 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Compression algorithms supported: 2024-02-13T14:11:06.323 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-02-13T14:11:06.324 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: kZSTD supported: 0 2024-02-13T14:11:06.324 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: kXpressCompression supported: 0 2024-02-13T14:11:06.324 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: kLZ4HCCompression supported: 1 2024-02-13T14:11:06.324 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: kZlibCompression supported: 1 2024-02-13T14:11:06.324 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: kSnappyCompression supported: 1 2024-02-13T14:11:06.324 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: kLZ4Compression supported: 1 2024-02-13T14:11:06.324 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: kBZip2Compression supported: 0 2024-02-13T14:11:06.324 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Fast CRC32 supported: Supported on x86 2024-02-13T14:11:06.324 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: DMutex implementation: pthread_mutex_t 2024-02-13T14:11:06.324 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-02-13T14:11:06.324 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-02-13T14:11:06.325 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-02-13T14:11:06.325 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.merge_operator: 2024-02-13T14:11:06.325 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.compaction_filter: None 2024-02-13T14:11:06.325 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.compaction_filter_factory: None 2024-02-13T14:11:06.325 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.sst_partitioner_factory: None 2024-02-13T14:11:06.325 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.memtable_factory: SkipListFactory 2024-02-13T14:11:06.325 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.table_factory: BlockBasedTable 2024-02-13T14:11:06.325 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x562e648df9c0) 2024-02-13T14:11:06.325 INFO:journalctl@ceph.mon.b.smithi148.stdout: cache_index_and_filter_blocks: 1 2024-02-13T14:11:06.325 INFO:journalctl@ceph.mon.b.smithi148.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-02-13T14:11:06.325 INFO:journalctl@ceph.mon.b.smithi148.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-02-13T14:11:06.326 INFO:journalctl@ceph.mon.b.smithi148.stdout: pin_top_level_index_and_filter: 1 2024-02-13T14:11:06.326 INFO:journalctl@ceph.mon.b.smithi148.stdout: index_type: 0 2024-02-13T14:11:06.326 INFO:journalctl@ceph.mon.b.smithi148.stdout: data_block_index_type: 0 2024-02-13T14:11:06.326 INFO:journalctl@ceph.mon.b.smithi148.stdout: index_shortening: 1 2024-02-13T14:11:06.326 INFO:journalctl@ceph.mon.b.smithi148.stdout: data_block_hash_table_util_ratio: 0.750000 2024-02-13T14:11:06.326 INFO:journalctl@ceph.mon.b.smithi148.stdout: checksum: 4 2024-02-13T14:11:06.326 INFO:journalctl@ceph.mon.b.smithi148.stdout: no_block_cache: 0 2024-02-13T14:11:06.326 INFO:journalctl@ceph.mon.b.smithi148.stdout: block_cache: 0x562e648e3090 2024-02-13T14:11:06.326 INFO:journalctl@ceph.mon.b.smithi148.stdout: block_cache_name: BinnedLRUCache 2024-02-13T14:11:06.326 INFO:journalctl@ceph.mon.b.smithi148.stdout: block_cache_options: 2024-02-13T14:11:06.326 INFO:journalctl@ceph.mon.b.smithi148.stdout: capacity : 536870912 2024-02-13T14:11:06.326 INFO:journalctl@ceph.mon.b.smithi148.stdout: num_shard_bits : 4 2024-02-13T14:11:06.326 INFO:journalctl@ceph.mon.b.smithi148.stdout: strict_capacity_limit : 0 2024-02-13T14:11:06.326 INFO:journalctl@ceph.mon.b.smithi148.stdout: high_pri_pool_ratio: 0.000 2024-02-13T14:11:06.326 INFO:journalctl@ceph.mon.b.smithi148.stdout: block_cache_compressed: (nil) 2024-02-13T14:11:06.326 INFO:journalctl@ceph.mon.b.smithi148.stdout: persistent_cache: (nil) 2024-02-13T14:11:06.327 INFO:journalctl@ceph.mon.b.smithi148.stdout: block_size: 4096 2024-02-13T14:11:06.327 INFO:journalctl@ceph.mon.b.smithi148.stdout: block_size_deviation: 10 2024-02-13T14:11:06.327 INFO:journalctl@ceph.mon.b.smithi148.stdout: block_restart_interval: 16 2024-02-13T14:11:06.327 INFO:journalctl@ceph.mon.b.smithi148.stdout: index_block_restart_interval: 1 2024-02-13T14:11:06.327 INFO:journalctl@ceph.mon.b.smithi148.stdout: metadata_block_size: 4096 2024-02-13T14:11:06.327 INFO:journalctl@ceph.mon.b.smithi148.stdout: partition_filters: 0 2024-02-13T14:11:06.327 INFO:journalctl@ceph.mon.b.smithi148.stdout: use_delta_encoding: 1 2024-02-13T14:11:06.327 INFO:journalctl@ceph.mon.b.smithi148.stdout: filter_policy: bloomfilter 2024-02-13T14:11:06.327 INFO:journalctl@ceph.mon.b.smithi148.stdout: whole_key_filtering: 1 2024-02-13T14:11:06.327 INFO:journalctl@ceph.mon.b.smithi148.stdout: verify_compression: 0 2024-02-13T14:11:06.327 INFO:journalctl@ceph.mon.b.smithi148.stdout: read_amp_bytes_per_bit: 0 2024-02-13T14:11:06.327 INFO:journalctl@ceph.mon.b.smithi148.stdout: format_version: 5 2024-02-13T14:11:06.327 INFO:journalctl@ceph.mon.b.smithi148.stdout: enable_index_compression: 1 2024-02-13T14:11:06.327 INFO:journalctl@ceph.mon.b.smithi148.stdout: block_align: 0 2024-02-13T14:11:06.327 INFO:journalctl@ceph.mon.b.smithi148.stdout: max_auto_readahead_size: 262144 2024-02-13T14:11:06.328 INFO:journalctl@ceph.mon.b.smithi148.stdout: prepopulate_block_cache: 0 2024-02-13T14:11:06.328 INFO:journalctl@ceph.mon.b.smithi148.stdout: initial_auto_readahead_size: 8192 2024-02-13T14:11:06.328 INFO:journalctl@ceph.mon.b.smithi148.stdout: num_file_reads_for_auto_readahead: 2 2024-02-13T14:11:06.328 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.write_buffer_size: 33554432 2024-02-13T14:11:06.328 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.max_write_buffer_number: 2 2024-02-13T14:11:06.328 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.compression: NoCompression 2024-02-13T14:11:06.328 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.bottommost_compression: Disabled 2024-02-13T14:11:06.328 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.prefix_extractor: nullptr 2024-02-13T14:11:06.328 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-02-13T14:11:06.328 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.num_levels: 7 2024-02-13T14:11:06.328 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-02-13T14:11:06.328 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-02-13T14:11:06.328 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-02-13T14:11:06.328 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-02-13T14:11:06.329 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-02-13T14:11:06.329 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-02-13T14:11:06.329 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-02-13T14:11:06.329 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-02-13T14:11:06.329 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-02-13T14:11:06.329 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-02-13T14:11:06.329 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-02-13T14:11:06.329 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-02-13T14:11:06.329 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.compression_opts.window_bits: -14 2024-02-13T14:11:06.329 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.compression_opts.level: 32767 2024-02-13T14:11:06.329 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.compression_opts.strategy: 0 2024-02-13T14:11:06.330 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-02-13T14:11:06.330 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-02-13T14:11:06.330 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-02-13T14:11:06.330 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-02-13T14:11:06.330 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.compression_opts.enabled: false 2024-02-13T14:11:06.330 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-02-13T14:11:06.330 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-02-13T14:11:06.330 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-02-13T14:11:06.330 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-02-13T14:11:06.330 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.target_file_size_base: 67108864 2024-02-13T14:11:06.330 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.target_file_size_multiplier: 1 2024-02-13T14:11:06.330 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-02-13T14:11:06.330 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-02-13T14:11:06.331 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-02-13T14:11:06.331 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-02-13T14:11:06.332 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-02-13T14:11:06.332 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-02-13T14:11:06.332 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-02-13T14:11:06.332 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-02-13T14:11:06.332 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-02-13T14:11:06.332 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-02-13T14:11:06.332 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-02-13T14:11:06.332 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-02-13T14:11:06.332 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-02-13T14:11:06.332 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.arena_block_size: 1048576 2024-02-13T14:11:06.332 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-02-13T14:11:06.332 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-02-13T14:11:06.333 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.disable_auto_compactions: 0 2024-02-13T14:11:06.333 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-02-13T14:11:06.333 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-02-13T14:11:06.333 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-02-13T14:11:06.333 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-02-13T14:11:06.333 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-02-13T14:11:06.333 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-02-13T14:11:06.333 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-02-13T14:11:06.333 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-02-13T14:11:06.333 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-02-13T14:11:06.333 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-02-13T14:11:06.333 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.table_properties_collectors: 2024-02-13T14:11:06.333 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.inplace_update_support: 0 2024-02-13T14:11:06.333 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.inplace_update_num_locks: 10000 2024-02-13T14:11:06.334 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-02-13T14:11:06.334 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-02-13T14:11:06.334 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.memtable_huge_page_size: 0 2024-02-13T14:11:06.334 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.bloom_locality: 0 2024-02-13T14:11:06.334 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.max_successive_merges: 0 2024-02-13T14:11:06.334 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.optimize_filters_for_hits: 0 2024-02-13T14:11:06.334 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.paranoid_file_checks: 0 2024-02-13T14:11:06.334 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.force_consistency_checks: 1 2024-02-13T14:11:06.334 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.report_bg_io_stats: 0 2024-02-13T14:11:06.334 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.ttl: 2592000 2024-02-13T14:11:06.334 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.periodic_compaction_seconds: 0 2024-02-13T14:11:06.334 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-02-13T14:11:06.334 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-02-13T14:11:06.334 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.enable_blob_files: false 2024-02-13T14:11:06.334 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.min_blob_size: 0 2024-02-13T14:11:06.335 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.blob_file_size: 268435456 2024-02-13T14:11:06.335 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.blob_compression_type: NoCompression 2024-02-13T14:11:06.335 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.enable_blob_garbage_collection: false 2024-02-13T14:11:06.335 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-02-13T14:11:06.335 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-02-13T14:11:06.335 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-02-13T14:11:06.335 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.blob_file_starting_level: 0 2024-02-13T14:11:06.335 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-02-13T14:11:06.335 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-02-13T14:11:06.335 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-02-13T14:11:06.335 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 1318dcd5-d5c3-49c6-9d3a-bea406beb46a 2024-02-13T14:11:06.335 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: EVENT_LOG_v1 {"time_micros": 1707833465932877, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-02-13T14:11:06.335 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-02-13T14:11:06.336 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: EVENT_LOG_v1 {"time_micros": 1707833465933292, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1622, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1707833465, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "1318dcd5-d5c3-49c6-9d3a-bea406beb46a", "db_session_id": "FNY1YURSCWD1KBX8R10T", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-02-13T14:11:06.336 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: EVENT_LOG_v1 {"time_micros": 1707833465933371, "job": 1, "event": "recovery_finished"} 2024-02-13T14:11:06.336 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-02-13T14:11:06.336 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-02-13T14:11:06.336 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x562e649d4000 2024-02-13T14:11:06.336 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: DB pointer 0x562e649be000 2024-02-13T14:11:06.336 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-02-13T14:11:06.336 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-02-13T14:11:06.336 INFO:journalctl@ceph.mon.b.smithi148.stdout: ** DB Stats ** 2024-02-13T14:11:06.336 INFO:journalctl@ceph.mon.b.smithi148.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-13T14:11:06.336 INFO:journalctl@ceph.mon.b.smithi148.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-02-13T14:11:06.336 INFO:journalctl@ceph.mon.b.smithi148.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-13T14:11:06.336 INFO:journalctl@ceph.mon.b.smithi148.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-13T14:11:06.337 INFO:journalctl@ceph.mon.b.smithi148.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-02-13T14:11:06.337 INFO:journalctl@ceph.mon.b.smithi148.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-13T14:11:06.337 INFO:journalctl@ceph.mon.b.smithi148.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-13T14:11:06.337 INFO:journalctl@ceph.mon.b.smithi148.stdout: 2024-02-13T14:11:06.337 INFO:journalctl@ceph.mon.b.smithi148.stdout: ** Compaction Stats [default] ** 2024-02-13T14:11:06.337 INFO:journalctl@ceph.mon.b.smithi148.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-13T14:11:06.337 INFO:journalctl@ceph.mon.b.smithi148.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-02-13T14:11:06.337 INFO:journalctl@ceph.mon.b.smithi148.stdout: L0 1/0 1.58 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-02-13T14:11:06.337 INFO:journalctl@ceph.mon.b.smithi148.stdout: Sum 1/0 1.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-02-13T14:11:06.337 INFO:journalctl@ceph.mon.b.smithi148.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-02-13T14:11:06.337 INFO:journalctl@ceph.mon.b.smithi148.stdout: 2024-02-13T14:11:06.337 INFO:journalctl@ceph.mon.b.smithi148.stdout: ** Compaction Stats [default] ** 2024-02-13T14:11:06.337 INFO:journalctl@ceph.mon.b.smithi148.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-13T14:11:06.337 INFO:journalctl@ceph.mon.b.smithi148.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-13T14:11:06.337 INFO:journalctl@ceph.mon.b.smithi148.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-02-13T14:11:06.337 INFO:journalctl@ceph.mon.b.smithi148.stdout: 2024-02-13T14:11:06.338 INFO:journalctl@ceph.mon.b.smithi148.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-02-13T14:11:06.338 INFO:journalctl@ceph.mon.b.smithi148.stdout: 2024-02-13T14:11:06.338 INFO:journalctl@ceph.mon.b.smithi148.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-13T14:11:06.338 INFO:journalctl@ceph.mon.b.smithi148.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-02-13T14:11:06.338 INFO:journalctl@ceph.mon.b.smithi148.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-13T14:11:06.338 INFO:journalctl@ceph.mon.b.smithi148.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-02-13T14:11:06.338 INFO:journalctl@ceph.mon.b.smithi148.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-02-13T14:11:06.338 INFO:journalctl@ceph.mon.b.smithi148.stdout: AddFile(Keys): cumulative 0, interval 0 2024-02-13T14:11:06.338 INFO:journalctl@ceph.mon.b.smithi148.stdout: Cumulative compaction: 0.00 GB write, 0.64 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-13T14:11:06.338 INFO:journalctl@ceph.mon.b.smithi148.stdout: Interval compaction: 0.00 GB write, 0.64 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-13T14:11:06.338 INFO:journalctl@ceph.mon.b.smithi148.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-13T14:11:06.338 INFO:journalctl@ceph.mon.b.smithi148.stdout: Block cache BinnedLRUCache@0x562e648e3090#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-02-13T14:11:06.338 INFO:journalctl@ceph.mon.b.smithi148.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-02-13T14:11:06.338 INFO:journalctl@ceph.mon.b.smithi148.stdout: 2024-02-13T14:11:06.338 INFO:journalctl@ceph.mon.b.smithi148.stdout: ** File Read Latency Histogram By Level [default] ** 2024-02-13T14:11:06.338 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-02-13T14:11:06.339 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: using public_addr v2:172.21.15.148:0/0 -> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] 2024-02-13T14:11:06.339 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: starting mon.b rank -1 at public addrs [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] at bind addrs [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 5c04b074-ca79-11ee-95b9-87774f69a715 2024-02-13T14:11:06.339 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: mon.b@-1(???) e0 preinit fsid 5c04b074-ca79-11ee-95b9-87774f69a715 2024-02-13T14:11:06.339 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: mon.b@-1(synchronizing).mds e1 new map 2024-02-13T14:11:06.339 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: mon.b@-1(synchronizing).mds e1 print_map 2024-02-13T14:11:06.339 INFO:journalctl@ceph.mon.b.smithi148.stdout: e1 2024-02-13T14:11:06.339 INFO:journalctl@ceph.mon.b.smithi148.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-02-13T14:11:06.339 INFO:journalctl@ceph.mon.b.smithi148.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-02-13T14:11:06.339 INFO:journalctl@ceph.mon.b.smithi148.stdout: legacy client fscid: -1 2024-02-13T14:11:06.339 INFO:journalctl@ceph.mon.b.smithi148.stdout: 2024-02-13T14:11:06.339 INFO:journalctl@ceph.mon.b.smithi148.stdout: No filesystems configured 2024-02-13T14:11:06.339 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-02-13T14:11:06.339 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-02-13T14:11:06.339 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-02-13T14:11:06.339 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-02-13T14:11:06.339 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-02-13T14:11:06.340 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-02-13T14:11:06.340 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-02-13T14:11:06.340 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-02-13T14:11:06.340 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-02-13T14:11:06.340 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-02-13T14:11:06.340 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:11:06.340 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/2240416164' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-02-13T14:11:06.340 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:06.340 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:06.340 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:06.340 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:06.340 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi148", "name": "osd_memory_target"}]: dispatch 2024-02-13T14:11:06.340 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:11:06.340 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:11:06.341 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: Updating smithi148:/etc/ceph/ceph.conf 2024-02-13T14:11:06.341 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: Updating smithi148:/var/lib/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/config/ceph.conf 2024-02-13T14:11:06.341 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/2240416164' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-02-13T14:11:06.341 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: osdmap e4: 0 total, 0 up, 0 in 2024-02-13T14:11:06.341 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: Updating smithi148:/etc/ceph/ceph.client.admin.keyring 2024-02-13T14:11:06.341 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:06.341 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:06.341 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:06.341 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: Updating smithi148:/var/lib/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/config/ceph.client.admin.keyring 2024-02-13T14:11:06.341 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:11:06.341 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi087:172.21.15.87=a;smithi087:[v2:172.21.15.87:3301,v1:172.21.15.87:6790]=c;smithi148:172.21.15.148=b", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T14:11:06.341 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: Saving service mon spec with placement smithi087:172.21.15.87=a;smithi087:[v2:172.21.15.87:3301,v1:172.21.15.87:6790]=c;smithi148:172.21.15.148=b;count:3 2024-02-13T14:11:06.341 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:06.341 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:11:06.341 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:11:06.341 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:11:06.342 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:06.342 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-13T14:11:06.342 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:11:06.342 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: Deploying daemon mon.b on smithi148 2024-02-13T14:11:06.342 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:11:06.342 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:11:06.342 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:11:06.342 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: from='client.? 172.21.15.148:0/1298203649' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-13T14:11:06.342 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 ceph-mon[31198]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-02-13T14:11:06.342 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 bash[31139]: f226490f659a7511676a5098b1702367602d4eb76d1dae56d847f4f67b903696 2024-02-13T14:11:06.342 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:05 smithi148 systemd[1]: Started Ceph mon.b for 5c04b074-ca79-11ee-95b9-87774f69a715. 2024-02-13T14:11:06.623 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-02-13T14:11:06.623 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5c04b074-ca79-11ee-95b9-87774f69a715 -- ceph mon dump -f json 2024-02-13T14:11:06.859 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/mon.b/config 2024-02-13T14:11:11.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:11 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-13T14:11:11.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:11 smithi148 ceph-mon[31198]: mon.a calling monitor election 2024-02-13T14:11:11.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:11 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-13T14:11:11.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:11 smithi148 ceph-mon[31198]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:11:11.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:11 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-13T14:11:11.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:11 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-13T14:11:11.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:11 smithi148 ceph-mon[31198]: mon.b calling monitor election 2024-02-13T14:11:11.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:11 smithi148 ceph-mon[31198]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:11:11.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:11 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-13T14:11:11.370 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:11 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-13T14:11:11.370 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:11 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-13T14:11:11.370 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:11 smithi148 ceph-mon[31198]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-02-13T14:11:11.370 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:11 smithi148 ceph-mon[31198]: monmap e2: 2 mons at {a=[v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0],b=[v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-13T14:11:11.370 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:11 smithi148 ceph-mon[31198]: fsmap 2024-02-13T14:11:11.370 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:11 smithi148 ceph-mon[31198]: osdmap e4: 0 total, 0 up, 0 in 2024-02-13T14:11:11.370 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:11 smithi148 ceph-mon[31198]: mgrmap e13: y(active, since 42s) 2024-02-13T14:11:11.370 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:11 smithi148 ceph-mon[31198]: overall HEALTH_OK 2024-02-13T14:11:11.370 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:11 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:11.370 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:11 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:11.370 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:11 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:11.370 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:11 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-13T14:11:11.370 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:11 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:11:11.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:11 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-13T14:11:11.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:11 smithi087 ceph-mon[27372]: mon.a calling monitor election 2024-02-13T14:11:11.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:11 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-13T14:11:11.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:11 smithi087 ceph-mon[27372]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:11:11.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:11 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-13T14:11:11.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:11 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-13T14:11:11.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:11 smithi087 ceph-mon[27372]: mon.b calling monitor election 2024-02-13T14:11:11.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:11 smithi087 ceph-mon[27372]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:11:11.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:11 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-13T14:11:11.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:11 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-13T14:11:11.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:11 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-13T14:11:11.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:11 smithi087 ceph-mon[27372]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-02-13T14:11:11.493 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:11 smithi087 ceph-mon[27372]: monmap e2: 2 mons at {a=[v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0],b=[v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-13T14:11:11.493 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:11 smithi087 ceph-mon[27372]: fsmap 2024-02-13T14:11:11.493 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:11 smithi087 ceph-mon[27372]: osdmap e4: 0 total, 0 up, 0 in 2024-02-13T14:11:11.493 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:11 smithi087 ceph-mon[27372]: mgrmap e13: y(active, since 42s) 2024-02-13T14:11:11.493 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:11 smithi087 ceph-mon[27372]: overall HEALTH_OK 2024-02-13T14:11:11.493 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:11 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:11.493 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:11 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:11.493 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:11 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:11.493 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:11 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-13T14:11:11.493 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:11 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:11:11.707 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-02-13T14:11:11.708 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":2,"fsid":"5c04b074-ca79-11ee-95b9-87774f69a715","modified":"2024-02-13T14:11:05.954613Z","created":"2024-02-13T14:09:28.013951Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:3300","nonce":0},{"type":"v1","addr":"172.21.15.87:6789","nonce":0}]},"addr":"172.21.15.87:6789/0","public_addr":"172.21.15.87:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:3300","nonce":0},{"type":"v1","addr":"172.21.15.148:6789","nonce":0}]},"addr":"172.21.15.148:6789/0","public_addr":"172.21.15.148:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-02-13T14:11:11.710 INFO:teuthology.orchestra.run.smithi148.stderr:dumped monmap epoch 2 2024-02-13T14:11:12.334 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:12 smithi148 ceph-mon[31198]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:11:12.334 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:12 smithi148 ceph-mon[31198]: Deploying daemon mon.c on smithi087 2024-02-13T14:11:12.335 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:12 smithi148 ceph-mon[31198]: from='client.? 172.21.15.148:0/1662184824' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-13T14:11:12.335 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:12 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-13T14:11:12.338 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:12 smithi087 ceph-mon[27372]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:11:12.339 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:12 smithi087 ceph-mon[27372]: Deploying daemon mon.c on smithi087 2024-02-13T14:11:12.339 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:12 smithi087 ceph-mon[27372]: from='client.? 172.21.15.148:0/1662184824' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-13T14:11:12.339 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:12 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-13T14:11:13.364 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-02-13T14:11:13.365 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5c04b074-ca79-11ee-95b9-87774f69a715 -- ceph mon dump -f json 2024-02-13T14:11:13.611 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/mon.b/config 2024-02-13T14:11:14.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[27372]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:11:14.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 systemd[1]: Starting Ceph mon.c for 5c04b074-ca79-11ee-95b9-87774f69a715... 2024-02-13T14:11:14.323 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:14 smithi148 ceph-mon[31198]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:11:14.574 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 podman[39082]: 2024-02-13 14:11:14.223730125 +0000 UTC m=+0.017695103 image pull 28480620e5edd298d55c0e6673dcbad9ed23edd37269028a11063c1b8925676a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f 2024-02-13T14:11:14.575 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 podman[39082]: 2024-02-13 14:11:14.346603079 +0000 UTC m=+0.140568088 container create b5ff86db5b9c08308bc58637575c2c304c52032b6049c3b84a37f665c46ed0b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mon-c, org.label-schema.build-date=20240131, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.2, ceph=True, org.label-schema.schema-version=1.0) 2024-02-13T14:11:14.575 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 podman[39082]: 2024-02-13 14:11:14.540350675 +0000 UTC m=+0.334315645 container init b5ff86db5b9c08308bc58637575c2c304c52032b6049c3b84a37f665c46ed0b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mon-c, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240131, io.buildah.version=1.33.2, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.vendor=CentOS, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, GIT_BRANCH=HEAD) 2024-02-13T14:11:14.575 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 podman[39082]: 2024-02-13 14:11:14.546146971 +0000 UTC m=+0.340111935 container start b5ff86db5b9c08308bc58637575c2c304c52032b6049c3b84a37f665c46ed0b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mon-c, org.label-schema.schema-version=1.0, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240131, ceph=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_BRANCH=HEAD, io.buildah.version=1.33.2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-02-13T14:11:14.575 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: set uid:gid to 167:167 (ceph:ceph) 2024-02-13T14:11:14.575 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: ceph version 19.0.0-1423-ge1b79d5a (e1b79d5a4193a484176863ab96c0ca28c00e1662) squid (dev), process ceph-mon, pid 7 2024-02-13T14:11:14.575 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: pidfile_write: ignore empty --pid-file 2024-02-13T14:11:14.575 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: load: jerasure load: lrc 2024-02-13T14:11:14.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: RocksDB version: 7.9.2 2024-02-13T14:11:14.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Git sha 0 2024-02-13T14:11:14.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Compile date 2024-02-12 22:40:01 2024-02-13T14:11:14.993 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: DB SUMMARY 2024-02-13T14:11:14.993 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: DB Session ID: 2ZU81ORSNDB6AFHVBVS4 2024-02-13T14:11:14.993 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: CURRENT file: CURRENT 2024-02-13T14:11:14.993 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: IDENTITY file: IDENTITY 2024-02-13T14:11:14.993 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-02-13T14:11:14.993 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-02-13T14:11:14.993 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 636 ; 2024-02-13T14:11:14.993 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.error_if_exists: 0 2024-02-13T14:11:14.993 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.create_if_missing: 0 2024-02-13T14:11:14.993 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.paranoid_checks: 1 2024-02-13T14:11:14.993 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.flush_verify_memtable_count: 1 2024-02-13T14:11:14.993 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-02-13T14:11:14.994 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-02-13T14:11:14.994 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.env: 0x559178c33ae0 2024-02-13T14:11:14.994 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.fs: PosixFileSystem 2024-02-13T14:11:14.994 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.info_log: 0x55917958b8c0 2024-02-13T14:11:14.994 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.max_file_opening_threads: 16 2024-02-13T14:11:14.994 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.statistics: (nil) 2024-02-13T14:11:14.994 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.use_fsync: 0 2024-02-13T14:11:14.994 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.max_log_file_size: 0 2024-02-13T14:11:14.994 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-02-13T14:11:14.994 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.log_file_time_to_roll: 0 2024-02-13T14:11:14.994 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.keep_log_file_num: 1000 2024-02-13T14:11:14.994 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.recycle_log_file_num: 0 2024-02-13T14:11:14.994 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.allow_fallocate: 1 2024-02-13T14:11:14.995 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.allow_mmap_reads: 0 2024-02-13T14:11:14.995 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.allow_mmap_writes: 0 2024-02-13T14:11:14.995 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.use_direct_reads: 0 2024-02-13T14:11:14.995 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-02-13T14:11:14.995 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.create_missing_column_families: 0 2024-02-13T14:11:14.995 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.db_log_dir: 2024-02-13T14:11:14.995 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.wal_dir: 2024-02-13T14:11:14.995 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.table_cache_numshardbits: 6 2024-02-13T14:11:14.995 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.WAL_ttl_seconds: 0 2024-02-13T14:11:14.995 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.WAL_size_limit_MB: 0 2024-02-13T14:11:14.995 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-02-13T14:11:14.996 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-02-13T14:11:14.996 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.is_fd_close_on_exec: 1 2024-02-13T14:11:14.996 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.advise_random_on_open: 1 2024-02-13T14:11:14.996 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.db_write_buffer_size: 0 2024-02-13T14:11:14.996 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.write_buffer_manager: 0x55917955f400 2024-02-13T14:11:14.996 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-02-13T14:11:14.996 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-02-13T14:11:14.996 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.use_adaptive_mutex: 0 2024-02-13T14:11:14.996 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.rate_limiter: (nil) 2024-02-13T14:11:14.996 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-02-13T14:11:14.996 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.wal_recovery_mode: 2 2024-02-13T14:11:14.996 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.enable_thread_tracking: 0 2024-02-13T14:11:14.996 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.enable_pipelined_write: 0 2024-02-13T14:11:14.996 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.unordered_write: 0 2024-02-13T14:11:14.997 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-02-13T14:11:14.997 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-02-13T14:11:14.997 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-02-13T14:11:14.997 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-02-13T14:11:14.997 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.row_cache: None 2024-02-13T14:11:14.997 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.wal_filter: None 2024-02-13T14:11:14.997 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-02-13T14:11:14.997 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.allow_ingest_behind: 0 2024-02-13T14:11:14.997 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.two_write_queues: 0 2024-02-13T14:11:14.997 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.manual_wal_flush: 0 2024-02-13T14:11:14.997 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.wal_compression: 0 2024-02-13T14:11:14.997 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.atomic_flush: 0 2024-02-13T14:11:14.997 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-02-13T14:11:14.997 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.persist_stats_to_disk: 0 2024-02-13T14:11:14.998 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.write_dbid_to_manifest: 0 2024-02-13T14:11:14.998 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.log_readahead_size: 0 2024-02-13T14:11:14.998 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-02-13T14:11:14.998 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.best_efforts_recovery: 0 2024-02-13T14:11:14.998 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-02-13T14:11:14.998 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-02-13T14:11:14.998 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.allow_data_in_errors: 0 2024-02-13T14:11:14.998 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.db_host_id: __hostname__ 2024-02-13T14:11:14.998 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.enforce_single_del_contracts: true 2024-02-13T14:11:14.998 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.max_background_jobs: 2 2024-02-13T14:11:14.998 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.max_background_compactions: -1 2024-02-13T14:11:14.998 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.max_subcompactions: 1 2024-02-13T14:11:14.998 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-02-13T14:11:15.000 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-02-13T14:11:15.000 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.delayed_write_rate : 16777216 2024-02-13T14:11:15.000 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.max_total_wal_size: 0 2024-02-13T14:11:15.000 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-02-13T14:11:15.000 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.stats_dump_period_sec: 600 2024-02-13T14:11:15.000 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.stats_persist_period_sec: 600 2024-02-13T14:11:15.000 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-02-13T14:11:15.000 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.max_open_files: -1 2024-02-13T14:11:15.000 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.bytes_per_sync: 0 2024-02-13T14:11:15.000 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.wal_bytes_per_sync: 0 2024-02-13T14:11:15.000 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.strict_bytes_per_sync: 0 2024-02-13T14:11:15.000 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.compaction_readahead_size: 0 2024-02-13T14:11:15.001 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.max_background_flushes: -1 2024-02-13T14:11:15.001 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Compression algorithms supported: 2024-02-13T14:11:15.001 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-02-13T14:11:15.001 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: kZSTD supported: 0 2024-02-13T14:11:15.001 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: kXpressCompression supported: 0 2024-02-13T14:11:15.001 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: kLZ4HCCompression supported: 1 2024-02-13T14:11:15.001 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: kZlibCompression supported: 1 2024-02-13T14:11:15.001 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: kSnappyCompression supported: 1 2024-02-13T14:11:15.001 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: kLZ4Compression supported: 1 2024-02-13T14:11:15.001 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: kBZip2Compression supported: 0 2024-02-13T14:11:15.001 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Fast CRC32 supported: Supported on x86 2024-02-13T14:11:15.001 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: DMutex implementation: pthread_mutex_t 2024-02-13T14:11:15.001 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-02-13T14:11:15.001 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-02-13T14:11:15.002 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-02-13T14:11:15.002 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.merge_operator: 2024-02-13T14:11:15.002 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.compaction_filter: None 2024-02-13T14:11:15.002 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.compaction_filter_factory: None 2024-02-13T14:11:15.002 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.sst_partitioner_factory: None 2024-02-13T14:11:15.002 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.memtable_factory: SkipListFactory 2024-02-13T14:11:15.002 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.table_factory: BlockBasedTable 2024-02-13T14:11:15.002 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55917958b9c0) 2024-02-13T14:11:15.002 INFO:journalctl@ceph.mon.c.smithi087.stdout: cache_index_and_filter_blocks: 1 2024-02-13T14:11:15.002 INFO:journalctl@ceph.mon.c.smithi087.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-02-13T14:11:15.002 INFO:journalctl@ceph.mon.c.smithi087.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-02-13T14:11:15.002 INFO:journalctl@ceph.mon.c.smithi087.stdout: pin_top_level_index_and_filter: 1 2024-02-13T14:11:15.002 INFO:journalctl@ceph.mon.c.smithi087.stdout: index_type: 0 2024-02-13T14:11:15.002 INFO:journalctl@ceph.mon.c.smithi087.stdout: data_block_index_type: 0 2024-02-13T14:11:15.002 INFO:journalctl@ceph.mon.c.smithi087.stdout: index_shortening: 1 2024-02-13T14:11:15.003 INFO:journalctl@ceph.mon.c.smithi087.stdout: data_block_hash_table_util_ratio: 0.750000 2024-02-13T14:11:15.003 INFO:journalctl@ceph.mon.c.smithi087.stdout: checksum: 4 2024-02-13T14:11:15.003 INFO:journalctl@ceph.mon.c.smithi087.stdout: no_block_cache: 0 2024-02-13T14:11:15.003 INFO:journalctl@ceph.mon.c.smithi087.stdout: block_cache: 0x55917958f090 2024-02-13T14:11:15.003 INFO:journalctl@ceph.mon.c.smithi087.stdout: block_cache_name: BinnedLRUCache 2024-02-13T14:11:15.003 INFO:journalctl@ceph.mon.c.smithi087.stdout: block_cache_options: 2024-02-13T14:11:15.003 INFO:journalctl@ceph.mon.c.smithi087.stdout: capacity : 536870912 2024-02-13T14:11:15.003 INFO:journalctl@ceph.mon.c.smithi087.stdout: num_shard_bits : 4 2024-02-13T14:11:15.003 INFO:journalctl@ceph.mon.c.smithi087.stdout: strict_capacity_limit : 0 2024-02-13T14:11:15.003 INFO:journalctl@ceph.mon.c.smithi087.stdout: high_pri_pool_ratio: 0.000 2024-02-13T14:11:15.003 INFO:journalctl@ceph.mon.c.smithi087.stdout: block_cache_compressed: (nil) 2024-02-13T14:11:15.003 INFO:journalctl@ceph.mon.c.smithi087.stdout: persistent_cache: (nil) 2024-02-13T14:11:15.003 INFO:journalctl@ceph.mon.c.smithi087.stdout: block_size: 4096 2024-02-13T14:11:15.004 INFO:journalctl@ceph.mon.c.smithi087.stdout: block_size_deviation: 10 2024-02-13T14:11:15.004 INFO:journalctl@ceph.mon.c.smithi087.stdout: block_restart_interval: 16 2024-02-13T14:11:15.004 INFO:journalctl@ceph.mon.c.smithi087.stdout: index_block_restart_interval: 1 2024-02-13T14:11:15.004 INFO:journalctl@ceph.mon.c.smithi087.stdout: metadata_block_size: 4096 2024-02-13T14:11:15.004 INFO:journalctl@ceph.mon.c.smithi087.stdout: partition_filters: 0 2024-02-13T14:11:15.004 INFO:journalctl@ceph.mon.c.smithi087.stdout: use_delta_encoding: 1 2024-02-13T14:11:15.004 INFO:journalctl@ceph.mon.c.smithi087.stdout: filter_policy: bloomfilter 2024-02-13T14:11:15.004 INFO:journalctl@ceph.mon.c.smithi087.stdout: whole_key_filtering: 1 2024-02-13T14:11:15.004 INFO:journalctl@ceph.mon.c.smithi087.stdout: verify_compression: 0 2024-02-13T14:11:15.004 INFO:journalctl@ceph.mon.c.smithi087.stdout: read_amp_bytes_per_bit: 0 2024-02-13T14:11:15.004 INFO:journalctl@ceph.mon.c.smithi087.stdout: format_version: 5 2024-02-13T14:11:15.004 INFO:journalctl@ceph.mon.c.smithi087.stdout: enable_index_compression: 1 2024-02-13T14:11:15.004 INFO:journalctl@ceph.mon.c.smithi087.stdout: block_align: 0 2024-02-13T14:11:15.004 INFO:journalctl@ceph.mon.c.smithi087.stdout: max_auto_readahead_size: 262144 2024-02-13T14:11:15.004 INFO:journalctl@ceph.mon.c.smithi087.stdout: prepopulate_block_cache: 0 2024-02-13T14:11:15.005 INFO:journalctl@ceph.mon.c.smithi087.stdout: initial_auto_readahead_size: 8192 2024-02-13T14:11:15.005 INFO:journalctl@ceph.mon.c.smithi087.stdout: num_file_reads_for_auto_readahead: 2 2024-02-13T14:11:15.005 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.write_buffer_size: 33554432 2024-02-13T14:11:15.005 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.max_write_buffer_number: 2 2024-02-13T14:11:15.005 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.compression: NoCompression 2024-02-13T14:11:15.005 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.bottommost_compression: Disabled 2024-02-13T14:11:15.005 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.prefix_extractor: nullptr 2024-02-13T14:11:15.005 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-02-13T14:11:15.005 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.num_levels: 7 2024-02-13T14:11:15.005 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-02-13T14:11:15.005 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-02-13T14:11:15.005 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-02-13T14:11:15.005 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-02-13T14:11:15.005 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-02-13T14:11:15.006 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-02-13T14:11:15.006 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-02-13T14:11:15.006 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-02-13T14:11:15.006 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-02-13T14:11:15.006 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-02-13T14:11:15.006 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-02-13T14:11:15.006 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-02-13T14:11:15.006 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.compression_opts.window_bits: -14 2024-02-13T14:11:15.006 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.compression_opts.level: 32767 2024-02-13T14:11:15.006 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.compression_opts.strategy: 0 2024-02-13T14:11:15.006 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-02-13T14:11:15.006 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-02-13T14:11:15.007 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-02-13T14:11:15.007 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-02-13T14:11:15.007 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.compression_opts.enabled: false 2024-02-13T14:11:15.007 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-02-13T14:11:15.007 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-02-13T14:11:15.007 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-02-13T14:11:15.007 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-02-13T14:11:15.007 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.target_file_size_base: 67108864 2024-02-13T14:11:15.007 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.target_file_size_multiplier: 1 2024-02-13T14:11:15.007 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-02-13T14:11:15.007 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-02-13T14:11:15.007 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-02-13T14:11:15.007 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-02-13T14:11:15.007 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-02-13T14:11:15.008 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-02-13T14:11:15.008 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-02-13T14:11:15.008 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-02-13T14:11:15.008 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-02-13T14:11:15.008 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-02-13T14:11:15.008 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-02-13T14:11:15.008 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-02-13T14:11:15.008 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-02-13T14:11:15.008 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.arena_block_size: 1048576 2024-02-13T14:11:15.008 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-02-13T14:11:15.008 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-02-13T14:11:15.008 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.disable_auto_compactions: 0 2024-02-13T14:11:15.008 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-02-13T14:11:15.008 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-02-13T14:11:15.008 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-02-13T14:11:15.009 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-02-13T14:11:15.009 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-02-13T14:11:15.009 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-02-13T14:11:15.009 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-02-13T14:11:15.009 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-02-13T14:11:15.009 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-02-13T14:11:15.009 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-02-13T14:11:15.009 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.table_properties_collectors: 2024-02-13T14:11:15.009 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.inplace_update_support: 0 2024-02-13T14:11:15.009 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.inplace_update_num_locks: 10000 2024-02-13T14:11:15.009 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-02-13T14:11:15.009 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-02-13T14:11:15.009 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.memtable_huge_page_size: 0 2024-02-13T14:11:15.009 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.bloom_locality: 0 2024-02-13T14:11:15.009 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.max_successive_merges: 0 2024-02-13T14:11:15.010 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.optimize_filters_for_hits: 0 2024-02-13T14:11:15.010 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.paranoid_file_checks: 0 2024-02-13T14:11:15.010 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.force_consistency_checks: 1 2024-02-13T14:11:15.010 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.report_bg_io_stats: 0 2024-02-13T14:11:15.010 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.ttl: 2592000 2024-02-13T14:11:15.010 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.periodic_compaction_seconds: 0 2024-02-13T14:11:15.010 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-02-13T14:11:15.010 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-02-13T14:11:15.010 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.enable_blob_files: false 2024-02-13T14:11:15.010 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.min_blob_size: 0 2024-02-13T14:11:15.010 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.blob_file_size: 268435456 2024-02-13T14:11:15.010 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.blob_compression_type: NoCompression 2024-02-13T14:11:15.010 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.enable_blob_garbage_collection: false 2024-02-13T14:11:15.010 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-02-13T14:11:15.011 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-02-13T14:11:15.011 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-02-13T14:11:15.011 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.blob_file_starting_level: 0 2024-02-13T14:11:15.011 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-02-13T14:11:15.011 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-02-13T14:11:15.011 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-02-13T14:11:15.011 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 2c6fec2c-6ee2-4684-b8ed-74e5d034fa31 2024-02-13T14:11:15.011 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: EVENT_LOG_v1 {"time_micros": 1707833474575660, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-02-13T14:11:15.011 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-02-13T14:11:15.011 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: EVENT_LOG_v1 {"time_micros": 1707833474576053, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1747, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1707833474, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "2c6fec2c-6ee2-4684-b8ed-74e5d034fa31", "db_session_id": "2ZU81ORSNDB6AFHVBVS4", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-02-13T14:11:15.011 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: EVENT_LOG_v1 {"time_micros": 1707833474576131, "job": 1, "event": "recovery_finished"} 2024-02-13T14:11:15.011 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-02-13T14:11:15.011 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-02-13T14:11:15.011 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x559179680000 2024-02-13T14:11:15.012 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: DB pointer 0x55917966a000 2024-02-13T14:11:15.012 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-02-13T14:11:15.012 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-02-13T14:11:15.012 INFO:journalctl@ceph.mon.c.smithi087.stdout: ** DB Stats ** 2024-02-13T14:11:15.012 INFO:journalctl@ceph.mon.c.smithi087.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-13T14:11:15.012 INFO:journalctl@ceph.mon.c.smithi087.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-02-13T14:11:15.012 INFO:journalctl@ceph.mon.c.smithi087.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-13T14:11:15.012 INFO:journalctl@ceph.mon.c.smithi087.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-13T14:11:15.012 INFO:journalctl@ceph.mon.c.smithi087.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-02-13T14:11:15.012 INFO:journalctl@ceph.mon.c.smithi087.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-13T14:11:15.012 INFO:journalctl@ceph.mon.c.smithi087.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-13T14:11:15.012 INFO:journalctl@ceph.mon.c.smithi087.stdout: 2024-02-13T14:11:15.012 INFO:journalctl@ceph.mon.c.smithi087.stdout: ** Compaction Stats [default] ** 2024-02-13T14:11:15.012 INFO:journalctl@ceph.mon.c.smithi087.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-13T14:11:15.013 INFO:journalctl@ceph.mon.c.smithi087.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-02-13T14:11:15.013 INFO:journalctl@ceph.mon.c.smithi087.stdout: L0 1/0 1.71 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.4 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-02-13T14:11:15.013 INFO:journalctl@ceph.mon.c.smithi087.stdout: Sum 1/0 1.71 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.4 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-02-13T14:11:15.013 INFO:journalctl@ceph.mon.c.smithi087.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.4 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-02-13T14:11:15.013 INFO:journalctl@ceph.mon.c.smithi087.stdout: 2024-02-13T14:11:15.013 INFO:journalctl@ceph.mon.c.smithi087.stdout: ** Compaction Stats [default] ** 2024-02-13T14:11:15.013 INFO:journalctl@ceph.mon.c.smithi087.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-13T14:11:15.013 INFO:journalctl@ceph.mon.c.smithi087.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-13T14:11:15.013 INFO:journalctl@ceph.mon.c.smithi087.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.4 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-02-13T14:11:15.013 INFO:journalctl@ceph.mon.c.smithi087.stdout: 2024-02-13T14:11:15.013 INFO:journalctl@ceph.mon.c.smithi087.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-02-13T14:11:15.013 INFO:journalctl@ceph.mon.c.smithi087.stdout: 2024-02-13T14:11:15.013 INFO:journalctl@ceph.mon.c.smithi087.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-13T14:11:15.013 INFO:journalctl@ceph.mon.c.smithi087.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-02-13T14:11:15.013 INFO:journalctl@ceph.mon.c.smithi087.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-13T14:11:15.013 INFO:journalctl@ceph.mon.c.smithi087.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-02-13T14:11:15.014 INFO:journalctl@ceph.mon.c.smithi087.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-02-13T14:11:15.014 INFO:journalctl@ceph.mon.c.smithi087.stdout: AddFile(Keys): cumulative 0, interval 0 2024-02-13T14:11:15.014 INFO:journalctl@ceph.mon.c.smithi087.stdout: Cumulative compaction: 0.00 GB write, 0.72 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-13T14:11:15.014 INFO:journalctl@ceph.mon.c.smithi087.stdout: Interval compaction: 0.00 GB write, 0.72 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-13T14:11:15.014 INFO:journalctl@ceph.mon.c.smithi087.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-13T14:11:15.014 INFO:journalctl@ceph.mon.c.smithi087.stdout: Block cache BinnedLRUCache@0x55917958f090#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2024-02-13T14:11:15.014 INFO:journalctl@ceph.mon.c.smithi087.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-02-13T14:11:15.014 INFO:journalctl@ceph.mon.c.smithi087.stdout: 2024-02-13T14:11:15.014 INFO:journalctl@ceph.mon.c.smithi087.stdout: ** File Read Latency Histogram By Level [default] ** 2024-02-13T14:11:15.014 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-02-13T14:11:15.014 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: using public_addrv [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] 2024-02-13T14:11:15.014 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: starting mon.c rank -1 at public addrs [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] at bind addrs [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] mon_data /var/lib/ceph/mon/ceph-c fsid 5c04b074-ca79-11ee-95b9-87774f69a715 2024-02-13T14:11:15.014 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: mon.c@-1(???) e0 preinit fsid 5c04b074-ca79-11ee-95b9-87774f69a715 2024-02-13T14:11:15.014 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: mon.c@-1(synchronizing).mds e1 new map 2024-02-13T14:11:15.014 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: mon.c@-1(synchronizing).mds e1 print_map 2024-02-13T14:11:15.015 INFO:journalctl@ceph.mon.c.smithi087.stdout: e1 2024-02-13T14:11:15.015 INFO:journalctl@ceph.mon.c.smithi087.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-02-13T14:11:15.015 INFO:journalctl@ceph.mon.c.smithi087.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-02-13T14:11:15.015 INFO:journalctl@ceph.mon.c.smithi087.stdout: legacy client fscid: -1 2024-02-13T14:11:15.015 INFO:journalctl@ceph.mon.c.smithi087.stdout: 2024-02-13T14:11:15.015 INFO:journalctl@ceph.mon.c.smithi087.stdout: No filesystems configured 2024-02-13T14:11:15.015 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-02-13T14:11:15.015 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-02-13T14:11:15.015 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-02-13T14:11:15.015 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-02-13T14:11:15.015 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-02-13T14:11:15.015 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-02-13T14:11:15.015 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-02-13T14:11:15.015 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-02-13T14:11:15.016 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-02-13T14:11:15.016 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-02-13T14:11:15.016 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:11:15.016 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/2240416164' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-02-13T14:11:15.016 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:15.016 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:15.016 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:15.016 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:15.016 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi148", "name": "osd_memory_target"}]: dispatch 2024-02-13T14:11:15.016 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:11:15.016 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:11:15.016 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: Updating smithi148:/etc/ceph/ceph.conf 2024-02-13T14:11:15.017 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: Updating smithi148:/var/lib/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/config/ceph.conf 2024-02-13T14:11:15.017 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/2240416164' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-02-13T14:11:15.017 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: osdmap e4: 0 total, 0 up, 0 in 2024-02-13T14:11:15.017 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: Updating smithi148:/etc/ceph/ceph.client.admin.keyring 2024-02-13T14:11:15.017 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:15.017 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:15.017 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:15.017 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: Updating smithi148:/var/lib/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/config/ceph.client.admin.keyring 2024-02-13T14:11:15.017 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:11:15.017 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi087:172.21.15.87=a;smithi087:[v2:172.21.15.87:3301,v1:172.21.15.87:6790]=c;smithi148:172.21.15.148=b", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T14:11:15.017 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: Saving service mon spec with placement smithi087:172.21.15.87=a;smithi087:[v2:172.21.15.87:3301,v1:172.21.15.87:6790]=c;smithi148:172.21.15.148=b;count:3 2024-02-13T14:11:15.017 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:15.017 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:11:15.018 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:11:15.018 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:11:15.018 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:15.018 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-13T14:11:15.018 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:11:15.018 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: Deploying daemon mon.b on smithi148 2024-02-13T14:11:15.018 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:11:15.018 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:11:15.018 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:11:15.018 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: from='client.? 172.21.15.148:0/1298203649' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-13T14:11:15.018 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-13T14:11:15.018 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: mon.a calling monitor election 2024-02-13T14:11:15.018 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-13T14:11:15.018 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:11:15.019 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-13T14:11:15.019 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-13T14:11:15.019 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: mon.b calling monitor election 2024-02-13T14:11:15.019 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:11:15.019 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-13T14:11:15.019 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-13T14:11:15.019 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-13T14:11:15.019 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-02-13T14:11:15.019 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: monmap e2: 2 mons at {a=[v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0],b=[v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-13T14:11:15.019 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: fsmap 2024-02-13T14:11:15.019 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: osdmap e4: 0 total, 0 up, 0 in 2024-02-13T14:11:15.019 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: mgrmap e13: y(active, since 42s) 2024-02-13T14:11:15.019 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: overall HEALTH_OK 2024-02-13T14:11:15.019 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:15.019 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:15.020 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:15.020 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-13T14:11:15.020 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:11:15.020 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:11:15.020 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: Deploying daemon mon.c on smithi087 2024-02-13T14:11:15.020 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: from='client.? 172.21.15.148:0/1662184824' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-13T14:11:15.020 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-13T14:11:15.020 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:11:15.020 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 ceph-mon[39120]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-02-13T14:11:15.020 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 bash[39082]: b5ff86db5b9c08308bc58637575c2c304c52032b6049c3b84a37f665c46ed0b8 2024-02-13T14:11:15.020 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:14 smithi087 systemd[1]: Started Ceph mon.c for 5c04b074-ca79-11ee-95b9-87774f69a715. 2024-02-13T14:11:19.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:19 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-13T14:11:19.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:19 smithi087 ceph-mon[39120]: mon.a calling monitor election 2024-02-13T14:11:19.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:19 smithi087 ceph-mon[39120]: mon.b calling monitor election 2024-02-13T14:11:19.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:19 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-13T14:11:19.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:19 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-13T14:11:19.993 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:19 smithi087 ceph-mon[39120]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:11:19.993 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:19 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-13T14:11:19.993 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:19 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-13T14:11:19.993 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:19 smithi087 ceph-mon[39120]: mon.c calling monitor election 2024-02-13T14:11:19.993 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:19 smithi087 ceph-mon[39120]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:11:19.993 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:19 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-13T14:11:19.993 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:19 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-13T14:11:19.993 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:19 smithi087 ceph-mon[39120]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:11:19.993 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:19 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-13T14:11:19.993 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:19 smithi087 ceph-mon[39120]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-02-13T14:11:19.993 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:19 smithi087 ceph-mon[39120]: monmap e3: 3 mons at {a=[v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0],b=[v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0],c=[v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-13T14:11:19.993 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:19 smithi087 ceph-mon[39120]: fsmap 2024-02-13T14:11:19.993 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:19 smithi087 ceph-mon[39120]: osdmap e4: 0 total, 0 up, 0 in 2024-02-13T14:11:19.994 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:19 smithi087 ceph-mon[39120]: mgrmap e13: y(active, since 50s) 2024-02-13T14:11:19.994 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:19 smithi087 ceph-mon[39120]: overall HEALTH_OK 2024-02-13T14:11:19.994 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:19 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:19.994 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:19 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:19.994 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:19 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:19.994 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:19 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:19.994 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:19 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-13T14:11:19.994 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:19 smithi087 ceph-mon[27372]: mon.a calling monitor election 2024-02-13T14:11:19.994 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:19 smithi087 ceph-mon[27372]: mon.b calling monitor election 2024-02-13T14:11:19.994 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:19 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-13T14:11:19.994 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:19 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-13T14:11:19.995 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:19 smithi087 ceph-mon[27372]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:11:19.995 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:19 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-13T14:11:19.995 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:19 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-13T14:11:19.995 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:19 smithi087 ceph-mon[27372]: mon.c calling monitor election 2024-02-13T14:11:19.995 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:19 smithi087 ceph-mon[27372]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:11:19.995 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:19 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-13T14:11:19.995 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:19 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-13T14:11:19.995 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:19 smithi087 ceph-mon[27372]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:11:19.995 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:19 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-13T14:11:19.995 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:19 smithi087 ceph-mon[27372]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-02-13T14:11:19.995 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:19 smithi087 ceph-mon[27372]: monmap e3: 3 mons at {a=[v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0],b=[v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0],c=[v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-13T14:11:19.995 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:19 smithi087 ceph-mon[27372]: fsmap 2024-02-13T14:11:19.996 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:19 smithi087 ceph-mon[27372]: osdmap e4: 0 total, 0 up, 0 in 2024-02-13T14:11:19.996 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:19 smithi087 ceph-mon[27372]: mgrmap e13: y(active, since 50s) 2024-02-13T14:11:19.996 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:19 smithi087 ceph-mon[27372]: overall HEALTH_OK 2024-02-13T14:11:19.996 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:19 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:19.996 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:19 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:19.996 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:19 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:19.996 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:19 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:20.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:19 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-13T14:11:20.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:19 smithi148 ceph-mon[31198]: mon.a calling monitor election 2024-02-13T14:11:20.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:19 smithi148 ceph-mon[31198]: mon.b calling monitor election 2024-02-13T14:11:20.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:19 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-13T14:11:20.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:19 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-13T14:11:20.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:19 smithi148 ceph-mon[31198]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:11:20.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:19 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-13T14:11:20.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:19 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-13T14:11:20.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:19 smithi148 ceph-mon[31198]: mon.c calling monitor election 2024-02-13T14:11:20.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:19 smithi148 ceph-mon[31198]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:11:20.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:19 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-13T14:11:20.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:19 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-13T14:11:20.120 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:19 smithi148 ceph-mon[31198]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:11:20.120 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:19 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-13T14:11:20.120 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:19 smithi148 ceph-mon[31198]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-02-13T14:11:20.120 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:19 smithi148 ceph-mon[31198]: monmap e3: 3 mons at {a=[v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0],b=[v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0],c=[v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-13T14:11:20.120 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:19 smithi148 ceph-mon[31198]: fsmap 2024-02-13T14:11:20.120 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:19 smithi148 ceph-mon[31198]: osdmap e4: 0 total, 0 up, 0 in 2024-02-13T14:11:20.120 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:19 smithi148 ceph-mon[31198]: mgrmap e13: y(active, since 50s) 2024-02-13T14:11:20.120 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:19 smithi148 ceph-mon[31198]: overall HEALTH_OK 2024-02-13T14:11:20.120 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:19 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:20.120 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:19 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:20.120 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:19 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:20.120 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:19 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:20.868 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:20 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:11:20.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:20 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-13T14:11:20.991 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:20 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:11:20.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:20 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-13T14:11:20.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:20 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:11:20.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:20 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-13T14:11:21.307 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-02-13T14:11:21.308 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":3,"fsid":"5c04b074-ca79-11ee-95b9-87774f69a715","modified":"2024-02-13T14:11:14.603495Z","created":"2024-02-13T14:09:28.013951Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:3300","nonce":0},{"type":"v1","addr":"172.21.15.87:6789","nonce":0}]},"addr":"172.21.15.87:6789/0","public_addr":"172.21.15.87:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:3300","nonce":0},{"type":"v1","addr":"172.21.15.148:6789","nonce":0}]},"addr":"172.21.15.148:6789/0","public_addr":"172.21.15.148:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:3301","nonce":0},{"type":"v1","addr":"172.21.15.87:6790","nonce":0}]},"addr":"172.21.15.87:6790/0","public_addr":"172.21.15.87:6790/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-02-13T14:11:21.310 INFO:teuthology.orchestra.run.smithi148.stderr:dumped monmap epoch 3 2024-02-13T14:11:22.159 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-02-13T14:11:22.160 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5c04b074-ca79-11ee-95b9-87774f69a715 -- ceph config generate-minimal-conf 2024-02-13T14:11:22.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:22 smithi148 ceph-mon[31198]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:11:22.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:22 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:22.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:22 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:22.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:22 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:22.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:22 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:22.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:22 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:11:22.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:22 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:11:22.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:22 smithi148 ceph-mon[31198]: from='client.? 172.21.15.148:0/2646789435' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-13T14:11:22.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:22 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:22.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:22 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:22.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:22 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:22.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:22 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:22.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:22 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:22.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:22 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:22.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:22 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:22.370 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:22 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:22.370 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:22 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:22.370 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:22 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-13T14:11:22.370 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:22 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-13T14:11:22.370 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:22 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:11:22.410 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:22 smithi087 ceph-mon[39120]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:11:22.410 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:22 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:22.410 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:22 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:22.410 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:22 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:22.410 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:22 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:22.410 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:22 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:11:22.410 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:22 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:11:22.410 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:22 smithi087 ceph-mon[39120]: from='client.? 172.21.15.148:0/2646789435' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-13T14:11:22.410 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:22 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:22.410 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:22 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:22.410 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:22 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:22.410 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:22 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:22.410 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:22 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:22.411 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:22 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:22.411 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:22 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:22.411 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:22 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:22.411 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:22 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:22.411 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:22 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-13T14:11:22.411 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:22 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-13T14:11:22.411 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:22 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:11:22.411 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:22 smithi087 ceph-mon[27372]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:11:22.411 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:22 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:22.411 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:22 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:22.411 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:22 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:22.411 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:22 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:22.411 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:22 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:11:22.411 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:22 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:11:22.411 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:22 smithi087 ceph-mon[27372]: from='client.? 172.21.15.148:0/2646789435' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-13T14:11:22.412 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:22 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:22.412 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:22 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:22.412 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:22 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:22.412 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:22 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:22.412 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:22 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:22.412 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:22 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:22.412 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:22 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:22.412 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:22 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:22.412 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:22 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:22.412 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:22 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-13T14:11:22.412 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:22 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-13T14:11:22.412 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:22 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:11:23.014 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/mon.a/config 2024-02-13T14:11:23.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:23 smithi148 ceph-mon[31198]: Updating smithi087:/etc/ceph/ceph.conf 2024-02-13T14:11:23.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:23 smithi148 ceph-mon[31198]: Updating smithi148:/etc/ceph/ceph.conf 2024-02-13T14:11:23.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:23 smithi148 ceph-mon[31198]: Updating smithi087:/var/lib/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/config/ceph.conf 2024-02-13T14:11:23.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:23 smithi148 ceph-mon[31198]: Updating smithi148:/var/lib/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/config/ceph.conf 2024-02-13T14:11:23.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:23 smithi148 ceph-mon[31198]: Reconfiguring mon.a (unknown last config time)... 2024-02-13T14:11:23.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:23 smithi148 ceph-mon[31198]: Reconfiguring daemon mon.a on smithi087 2024-02-13T14:11:23.410 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:23 smithi087 ceph-mon[39120]: Updating smithi087:/etc/ceph/ceph.conf 2024-02-13T14:11:23.411 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:23 smithi087 ceph-mon[39120]: Updating smithi148:/etc/ceph/ceph.conf 2024-02-13T14:11:23.411 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:23 smithi087 ceph-mon[39120]: Updating smithi087:/var/lib/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/config/ceph.conf 2024-02-13T14:11:23.411 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:23 smithi087 ceph-mon[39120]: Updating smithi148:/var/lib/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/config/ceph.conf 2024-02-13T14:11:23.411 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:23 smithi087 ceph-mon[39120]: Reconfiguring mon.a (unknown last config time)... 2024-02-13T14:11:23.411 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:23 smithi087 ceph-mon[39120]: Reconfiguring daemon mon.a on smithi087 2024-02-13T14:11:23.411 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:23 smithi087 ceph-mon[27372]: Updating smithi087:/etc/ceph/ceph.conf 2024-02-13T14:11:23.411 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:23 smithi087 ceph-mon[27372]: Updating smithi148:/etc/ceph/ceph.conf 2024-02-13T14:11:23.411 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:23 smithi087 ceph-mon[27372]: Updating smithi087:/var/lib/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/config/ceph.conf 2024-02-13T14:11:23.411 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:23 smithi087 ceph-mon[27372]: Updating smithi148:/var/lib/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/config/ceph.conf 2024-02-13T14:11:23.411 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:23 smithi087 ceph-mon[27372]: Reconfiguring mon.a (unknown last config time)... 2024-02-13T14:11:23.412 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:23 smithi087 ceph-mon[27372]: Reconfiguring daemon mon.a on smithi087 2024-02-13T14:11:24.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:24 smithi148 ceph-mon[31198]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:11:24.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:24 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:24.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:24 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:24.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:24 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-13T14:11:24.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:24 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-13T14:11:24.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:24 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:11:24.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:24 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:24.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:24 smithi087 ceph-mon[27372]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:11:24.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:24 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:24.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:24 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:24.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:24 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-13T14:11:24.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:24 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-13T14:11:24.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:24 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:11:24.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:24 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:24.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:24 smithi087 ceph-mon[39120]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:11:24.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:24 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:24.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:24 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:24.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:24 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-13T14:11:24.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:24 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-13T14:11:24.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:24 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:11:24.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:24 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:25.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:25 smithi148 ceph-mon[31198]: Reconfiguring mon.c (monmap changed)... 2024-02-13T14:11:25.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:25 smithi148 ceph-mon[31198]: Reconfiguring daemon mon.c on smithi087 2024-02-13T14:11:25.423 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:25 smithi087 ceph-mon[27372]: Reconfiguring mon.c (monmap changed)... 2024-02-13T14:11:25.423 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:25 smithi087 ceph-mon[27372]: Reconfiguring daemon mon.c on smithi087 2024-02-13T14:11:25.423 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:25 smithi087 ceph-mon[39120]: Reconfiguring mon.c (monmap changed)... 2024-02-13T14:11:25.423 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:25 smithi087 ceph-mon[39120]: Reconfiguring daemon mon.c on smithi087 2024-02-13T14:11:25.482 INFO:teuthology.orchestra.run.smithi087.stdout:# minimal ceph.conf for 5c04b074-ca79-11ee-95b9-87774f69a715 2024-02-13T14:11:25.482 INFO:teuthology.orchestra.run.smithi087.stdout:[global] 2024-02-13T14:11:25.482 INFO:teuthology.orchestra.run.smithi087.stdout: fsid = 5c04b074-ca79-11ee-95b9-87774f69a715 2024-02-13T14:11:25.482 INFO:teuthology.orchestra.run.smithi087.stdout: mon_host = [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] 2024-02-13T14:11:26.302 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:26 smithi087 ceph-mon[27372]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:11:26.302 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:26 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/2276395509' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:11:26.302 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:26 smithi087 ceph-mon[39120]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:11:26.303 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:26 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/2276395509' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:11:26.332 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-02-13T14:11:26.332 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-02-13T14:11:26.333 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd of=/etc/ceph/ceph.conf 2024-02-13T14:11:26.364 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-02-13T14:11:26.365 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-13T14:11:26.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:26 smithi148 ceph-mon[31198]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:11:26.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:26 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/2276395509' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:11:26.434 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-02-13T14:11:26.434 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/etc/ceph/ceph.conf 2024-02-13T14:11:26.464 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-02-13T14:11:26.464 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-13T14:11:26.532 INFO:tasks.cephadm:Adding mgr.y on smithi087 2024-02-13T14:11:26.532 INFO:tasks.cephadm:Adding mgr.x on smithi148 2024-02-13T14:11:26.532 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5c04b074-ca79-11ee-95b9-87774f69a715 -- ceph orch apply mgr '2;smithi087=y;smithi148=x' 2024-02-13T14:11:26.804 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/mon.b/config 2024-02-13T14:11:27.948 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:27 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:27.948 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:27 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:27.948 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:27 smithi148 ceph-mon[31198]: Reconfiguring mon.b (monmap changed)... 2024-02-13T14:11:27.948 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:27 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-13T14:11:27.949 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:27 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-13T14:11:27.949 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:27 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:11:27.949 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:27 smithi148 ceph-mon[31198]: Reconfiguring daemon mon.b on smithi148 2024-02-13T14:11:27.949 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:27 smithi148 ceph-mon[31198]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:11:28.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:27 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:28.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:27 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:28.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:27 smithi087 ceph-mon[39120]: Reconfiguring mon.b (monmap changed)... 2024-02-13T14:11:28.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:27 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-13T14:11:28.243 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:27 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-13T14:11:28.243 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:27 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:11:28.243 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:27 smithi087 ceph-mon[39120]: Reconfiguring daemon mon.b on smithi148 2024-02-13T14:11:28.243 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:27 smithi087 ceph-mon[39120]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:11:28.244 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:27 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:28.244 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:27 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:28.244 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:27 smithi087 ceph-mon[27372]: Reconfiguring mon.b (monmap changed)... 2024-02-13T14:11:28.244 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:27 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-13T14:11:28.244 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:27 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-13T14:11:28.244 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:27 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:11:28.245 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:27 smithi087 ceph-mon[27372]: Reconfiguring daemon mon.b on smithi148 2024-02-13T14:11:28.245 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:27 smithi087 ceph-mon[27372]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:11:28.858 INFO:teuthology.orchestra.run.smithi148.stdout:Scheduled mgr update... 2024-02-13T14:11:29.699 DEBUG:teuthology.orchestra.run.smithi148:mgr.x> sudo journalctl -f -n 0 -u ceph-5c04b074-ca79-11ee-95b9-87774f69a715@mgr.x.service 2024-02-13T14:11:29.701 INFO:tasks.cephadm:Deploying OSDs... 2024-02-13T14:11:29.702 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-02-13T14:11:29.702 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/scratch_devs of=/dev/stdout 2024-02-13T14:11:29.721 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-02-13T14:11:29.721 DEBUG:teuthology.orchestra.run.smithi087:> stat /dev/vg_nvme/lv_1 2024-02-13T14:11:29.779 INFO:teuthology.orchestra.run.smithi087.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-02-13T14:11:29.779 INFO:teuthology.orchestra.run.smithi087.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-13T14:11:29.779 INFO:teuthology.orchestra.run.smithi087.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-02-13T14:11:29.779 INFO:teuthology.orchestra.run.smithi087.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-13T14:11:29.779 INFO:teuthology.orchestra.run.smithi087.stdout:Context: system_u:object_r:device_t:s0 2024-02-13T14:11:29.779 INFO:teuthology.orchestra.run.smithi087.stdout:Access: 2024-02-13 14:11:14.048150122 +0000 2024-02-13T14:11:29.780 INFO:teuthology.orchestra.run.smithi087.stdout:Modify: 2024-02-13 14:06:17.766593576 +0000 2024-02-13T14:11:29.780 INFO:teuthology.orchestra.run.smithi087.stdout:Change: 2024-02-13 14:06:17.766593576 +0000 2024-02-13T14:11:29.780 INFO:teuthology.orchestra.run.smithi087.stdout: Birth: 2024-02-13 14:06:17.766593576 +0000 2024-02-13T14:11:29.780 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-02-13T14:11:29.848 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-02-13T14:11:29.848 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-02-13T14:11:29.848 INFO:teuthology.orchestra.run.smithi087.stderr:512 bytes copied, 0.000266948 s, 1.9 MB/s 2024-02-13T14:11:29.849 DEBUG:teuthology.orchestra.run.smithi087:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-02-13T14:11:29.906 DEBUG:teuthology.orchestra.run.smithi087:> stat /dev/vg_nvme/lv_2 2024-02-13T14:11:29.962 INFO:teuthology.orchestra.run.smithi087.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-02-13T14:11:29.963 INFO:teuthology.orchestra.run.smithi087.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-13T14:11:29.963 INFO:teuthology.orchestra.run.smithi087.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-02-13T14:11:29.963 INFO:teuthology.orchestra.run.smithi087.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-13T14:11:29.963 INFO:teuthology.orchestra.run.smithi087.stdout:Context: system_u:object_r:device_t:s0 2024-02-13T14:11:29.963 INFO:teuthology.orchestra.run.smithi087.stdout:Access: 2024-02-13 14:11:14.048150122 +0000 2024-02-13T14:11:29.963 INFO:teuthology.orchestra.run.smithi087.stdout:Modify: 2024-02-13 14:06:18.116585415 +0000 2024-02-13T14:11:29.963 INFO:teuthology.orchestra.run.smithi087.stdout:Change: 2024-02-13 14:06:18.116585415 +0000 2024-02-13T14:11:29.963 INFO:teuthology.orchestra.run.smithi087.stdout: Birth: 2024-02-13 14:06:18.116585415 +0000 2024-02-13T14:11:29.964 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-02-13T14:11:30.029 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-02-13T14:11:30.030 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-02-13T14:11:30.030 INFO:teuthology.orchestra.run.smithi087.stderr:512 bytes copied, 0.000227861 s, 2.2 MB/s 2024-02-13T14:11:30.031 DEBUG:teuthology.orchestra.run.smithi087:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-02-13T14:11:30.088 DEBUG:teuthology.orchestra.run.smithi087:> stat /dev/vg_nvme/lv_3 2024-02-13T14:11:30.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:29 smithi148 ceph-mon[31198]: from='client.24109 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi087=y;smithi148=x", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T14:11:30.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:29 smithi148 ceph-mon[31198]: Saving service mgr spec with placement smithi087=y;smithi148=x;count:2 2024-02-13T14:11:30.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:29 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:30.109 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:29 smithi148 ceph-mon[31198]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:11:30.144 INFO:teuthology.orchestra.run.smithi087.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-02-13T14:11:30.144 INFO:teuthology.orchestra.run.smithi087.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-13T14:11:30.144 INFO:teuthology.orchestra.run.smithi087.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-02-13T14:11:30.144 INFO:teuthology.orchestra.run.smithi087.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-13T14:11:30.144 INFO:teuthology.orchestra.run.smithi087.stdout:Context: system_u:object_r:device_t:s0 2024-02-13T14:11:30.144 INFO:teuthology.orchestra.run.smithi087.stdout:Access: 2024-02-13 14:11:14.049150099 +0000 2024-02-13T14:11:30.145 INFO:teuthology.orchestra.run.smithi087.stdout:Modify: 2024-02-13 14:06:18.464577300 +0000 2024-02-13T14:11:30.145 INFO:teuthology.orchestra.run.smithi087.stdout:Change: 2024-02-13 14:06:18.464577300 +0000 2024-02-13T14:11:30.145 INFO:teuthology.orchestra.run.smithi087.stdout: Birth: 2024-02-13 14:06:18.464577300 +0000 2024-02-13T14:11:30.145 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-02-13T14:11:30.210 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:29 smithi087 ceph-mon[27372]: from='client.24109 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi087=y;smithi148=x", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T14:11:30.210 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:29 smithi087 ceph-mon[27372]: Saving service mgr spec with placement smithi087=y;smithi148=x;count:2 2024-02-13T14:11:30.210 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:29 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:30.211 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:29 smithi087 ceph-mon[27372]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:11:30.211 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:29 smithi087 ceph-mon[39120]: from='client.24109 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi087=y;smithi148=x", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T14:11:30.211 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:29 smithi087 ceph-mon[39120]: Saving service mgr spec with placement smithi087=y;smithi148=x;count:2 2024-02-13T14:11:30.211 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:29 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:30.211 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:29 smithi087 ceph-mon[39120]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:11:30.213 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-02-13T14:11:30.213 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-02-13T14:11:30.213 INFO:teuthology.orchestra.run.smithi087.stderr:512 bytes copied, 0.000173059 s, 3.0 MB/s 2024-02-13T14:11:30.214 DEBUG:teuthology.orchestra.run.smithi087:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-02-13T14:11:30.270 DEBUG:teuthology.orchestra.run.smithi087:> stat /dev/vg_nvme/lv_4 2024-02-13T14:11:30.326 INFO:teuthology.orchestra.run.smithi087.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-02-13T14:11:30.326 INFO:teuthology.orchestra.run.smithi087.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-13T14:11:30.326 INFO:teuthology.orchestra.run.smithi087.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-02-13T14:11:30.326 INFO:teuthology.orchestra.run.smithi087.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-13T14:11:30.326 INFO:teuthology.orchestra.run.smithi087.stdout:Context: system_u:object_r:device_t:s0 2024-02-13T14:11:30.326 INFO:teuthology.orchestra.run.smithi087.stdout:Access: 2024-02-13 14:11:14.049150099 +0000 2024-02-13T14:11:30.326 INFO:teuthology.orchestra.run.smithi087.stdout:Modify: 2024-02-13 14:06:18.808569279 +0000 2024-02-13T14:11:30.326 INFO:teuthology.orchestra.run.smithi087.stdout:Change: 2024-02-13 14:06:18.808569279 +0000 2024-02-13T14:11:30.326 INFO:teuthology.orchestra.run.smithi087.stdout: Birth: 2024-02-13 14:06:18.808569279 +0000 2024-02-13T14:11:30.327 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-02-13T14:11:30.392 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-02-13T14:11:30.392 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-02-13T14:11:30.392 INFO:teuthology.orchestra.run.smithi087.stderr:512 bytes copied, 0.000237976 s, 2.2 MB/s 2024-02-13T14:11:30.394 DEBUG:teuthology.orchestra.run.smithi087:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-02-13T14:11:30.450 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-02-13T14:11:30.450 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/scratch_devs of=/dev/stdout 2024-02-13T14:11:30.465 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-02-13T14:11:30.465 DEBUG:teuthology.orchestra.run.smithi148:> stat /dev/vg_nvme/lv_1 2024-02-13T14:11:30.520 INFO:teuthology.orchestra.run.smithi148.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-02-13T14:11:30.521 INFO:teuthology.orchestra.run.smithi148.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-13T14:11:30.521 INFO:teuthology.orchestra.run.smithi148.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-02-13T14:11:30.521 INFO:teuthology.orchestra.run.smithi148.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-13T14:11:30.521 INFO:teuthology.orchestra.run.smithi148.stdout:Context: system_u:object_r:device_t:s0 2024-02-13T14:11:30.521 INFO:teuthology.orchestra.run.smithi148.stdout:Access: 2024-02-13 14:11:04.791257627 +0000 2024-02-13T14:11:30.521 INFO:teuthology.orchestra.run.smithi148.stdout:Modify: 2024-02-13 14:06:17.617170873 +0000 2024-02-13T14:11:30.521 INFO:teuthology.orchestra.run.smithi148.stdout:Change: 2024-02-13 14:06:17.617170873 +0000 2024-02-13T14:11:30.521 INFO:teuthology.orchestra.run.smithi148.stdout: Birth: 2024-02-13 14:06:17.617170873 +0000 2024-02-13T14:11:30.521 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-02-13T14:11:30.587 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-02-13T14:11:30.587 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-02-13T14:11:30.587 INFO:teuthology.orchestra.run.smithi148.stderr:512 bytes copied, 0.000199638 s, 2.6 MB/s 2024-02-13T14:11:30.588 DEBUG:teuthology.orchestra.run.smithi148:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-02-13T14:11:30.644 DEBUG:teuthology.orchestra.run.smithi148:> stat /dev/vg_nvme/lv_2 2024-02-13T14:11:30.699 INFO:teuthology.orchestra.run.smithi148.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-02-13T14:11:30.700 INFO:teuthology.orchestra.run.smithi148.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-13T14:11:30.700 INFO:teuthology.orchestra.run.smithi148.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-02-13T14:11:30.700 INFO:teuthology.orchestra.run.smithi148.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-13T14:11:30.700 INFO:teuthology.orchestra.run.smithi148.stdout:Context: system_u:object_r:device_t:s0 2024-02-13T14:11:30.700 INFO:teuthology.orchestra.run.smithi148.stdout:Access: 2024-02-13 14:11:04.792257604 +0000 2024-02-13T14:11:30.700 INFO:teuthology.orchestra.run.smithi148.stdout:Modify: 2024-02-13 14:06:17.955162903 +0000 2024-02-13T14:11:30.700 INFO:teuthology.orchestra.run.smithi148.stdout:Change: 2024-02-13 14:06:17.955162903 +0000 2024-02-13T14:11:30.700 INFO:teuthology.orchestra.run.smithi148.stdout: Birth: 2024-02-13 14:06:17.955162903 +0000 2024-02-13T14:11:30.700 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-02-13T14:11:30.767 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-02-13T14:11:30.767 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-02-13T14:11:30.767 INFO:teuthology.orchestra.run.smithi148.stderr:512 bytes copied, 0.000230054 s, 2.2 MB/s 2024-02-13T14:11:30.768 DEBUG:teuthology.orchestra.run.smithi148:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-02-13T14:11:30.823 DEBUG:teuthology.orchestra.run.smithi148:> stat /dev/vg_nvme/lv_3 2024-02-13T14:11:30.879 INFO:teuthology.orchestra.run.smithi148.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-02-13T14:11:30.879 INFO:teuthology.orchestra.run.smithi148.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-13T14:11:30.879 INFO:teuthology.orchestra.run.smithi148.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-02-13T14:11:30.879 INFO:teuthology.orchestra.run.smithi148.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-13T14:11:30.879 INFO:teuthology.orchestra.run.smithi148.stdout:Context: system_u:object_r:device_t:s0 2024-02-13T14:11:30.879 INFO:teuthology.orchestra.run.smithi148.stdout:Access: 2024-02-13 14:11:04.792257604 +0000 2024-02-13T14:11:30.879 INFO:teuthology.orchestra.run.smithi148.stdout:Modify: 2024-02-13 14:06:18.301154745 +0000 2024-02-13T14:11:30.879 INFO:teuthology.orchestra.run.smithi148.stdout:Change: 2024-02-13 14:06:18.301154745 +0000 2024-02-13T14:11:30.879 INFO:teuthology.orchestra.run.smithi148.stdout: Birth: 2024-02-13 14:06:18.301154745 +0000 2024-02-13T14:11:30.879 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-02-13T14:11:30.945 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-02-13T14:11:30.945 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-02-13T14:11:30.945 INFO:teuthology.orchestra.run.smithi148.stderr:512 bytes copied, 0.000206655 s, 2.5 MB/s 2024-02-13T14:11:30.947 DEBUG:teuthology.orchestra.run.smithi148:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-02-13T14:11:31.003 DEBUG:teuthology.orchestra.run.smithi148:> stat /dev/vg_nvme/lv_4 2024-02-13T14:11:31.058 INFO:teuthology.orchestra.run.smithi148.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-02-13T14:11:31.058 INFO:teuthology.orchestra.run.smithi148.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-13T14:11:31.058 INFO:teuthology.orchestra.run.smithi148.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-02-13T14:11:31.058 INFO:teuthology.orchestra.run.smithi148.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-13T14:11:31.058 INFO:teuthology.orchestra.run.smithi148.stdout:Context: system_u:object_r:device_t:s0 2024-02-13T14:11:31.058 INFO:teuthology.orchestra.run.smithi148.stdout:Access: 2024-02-13 14:11:04.793257581 +0000 2024-02-13T14:11:31.059 INFO:teuthology.orchestra.run.smithi148.stdout:Modify: 2024-02-13 14:06:18.655146398 +0000 2024-02-13T14:11:31.059 INFO:teuthology.orchestra.run.smithi148.stdout:Change: 2024-02-13 14:06:18.655146398 +0000 2024-02-13T14:11:31.059 INFO:teuthology.orchestra.run.smithi148.stdout: Birth: 2024-02-13 14:06:18.655146398 +0000 2024-02-13T14:11:31.059 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-02-13T14:11:31.126 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-02-13T14:11:31.126 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-02-13T14:11:31.126 INFO:teuthology.orchestra.run.smithi148.stderr:512 bytes copied, 0.000334838 s, 1.5 MB/s 2024-02-13T14:11:31.127 DEBUG:teuthology.orchestra.run.smithi148:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-02-13T14:11:31.184 INFO:tasks.cephadm:Deploying osd.0 on smithi087 with /dev/vg_nvme/lv_4... 2024-02-13T14:11:31.184 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5c04b074-ca79-11ee-95b9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-02-13T14:11:31.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:31 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:31.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:31 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:31.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:31 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:11:31.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:31 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:11:31.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:31 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:11:31.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:31 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:31.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:31 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-13T14:11:31.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:31 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-13T14:11:31.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:31 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-13T14:11:31.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:31 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:11:31.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:31 smithi148 ceph-mon[31198]: Deploying daemon mgr.x on smithi148 2024-02-13T14:11:31.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:31 smithi148 ceph-mon[31198]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:11:31.430 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/mon.a/config 2024-02-13T14:11:31.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:31 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:31.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:31 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:31.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:31 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:11:31.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:31 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:11:31.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:31 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:11:31.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:31 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:31.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:31 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-13T14:11:31.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:31 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-13T14:11:31.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:31 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-13T14:11:31.493 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:31 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:11:31.493 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:31 smithi087 ceph-mon[27372]: Deploying daemon mgr.x on smithi148 2024-02-13T14:11:31.493 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:31 smithi087 ceph-mon[27372]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:11:31.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:31 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:31.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:31 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:31.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:31 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:11:31.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:31 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:11:31.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:31 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:11:31.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:31 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:31.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:31 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-13T14:11:31.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:31 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-13T14:11:31.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:31 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-13T14:11:31.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:31 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:11:31.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:31 smithi087 ceph-mon[39120]: Deploying daemon mgr.x on smithi148 2024-02-13T14:11:31.494 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:31 smithi087 ceph-mon[39120]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:11:32.368 INFO:journalctl@ceph.mgr.x.smithi148.stdout:Feb 13 14:11:32 smithi148 systemd[1]: Starting Ceph mgr.x for 5c04b074-ca79-11ee-95b9-87774f69a715... 2024-02-13T14:11:32.753 INFO:journalctl@ceph.mgr.x.smithi148.stdout:Feb 13 14:11:32 smithi148 podman[33263]: 2024-02-13 14:11:32.474951137 +0000 UTC m=+0.106579701 container create a4c89d7bac084a3590523e174031c318270badf7857a44b265729b0e5d1bb273 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-x, org.label-schema.build-date=20240131, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-13T14:11:32.753 INFO:journalctl@ceph.mgr.x.smithi148.stdout:Feb 13 14:11:32 smithi148 podman[33263]: 2024-02-13 14:11:32.386282343 +0000 UTC m=+0.017910885 image pull 28480620e5edd298d55c0e6673dcbad9ed23edd37269028a11063c1b8925676a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f 2024-02-13T14:11:32.753 INFO:journalctl@ceph.mgr.x.smithi148.stdout:Feb 13 14:11:32 smithi148 podman[33263]: 2024-02-13 14:11:32.685502024 +0000 UTC m=+0.317130556 container init a4c89d7bac084a3590523e174031c318270badf7857a44b265729b0e5d1bb273 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-x, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240131, org.label-schema.schema-version=1.0, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-02-13T14:11:32.753 INFO:journalctl@ceph.mgr.x.smithi148.stdout:Feb 13 14:11:32 smithi148 podman[33263]: 2024-02-13 14:11:32.690961356 +0000 UTC m=+0.322589881 container start a4c89d7bac084a3590523e174031c318270badf7857a44b265729b0e5d1bb273 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-x, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240131, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-13T14:11:32.753 INFO:journalctl@ceph.mgr.x.smithi148.stdout:Feb 13 14:11:32 smithi148 ceph-mgr[33301]: -- 172.21.15.148:0/2263931013 <== mon.1 v2:172.21.15.148:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55e53fcbe1a0 con 0x55e53fc94c00 2024-02-13T14:11:32.753 INFO:journalctl@ceph.mgr.x.smithi148.stdout:Feb 13 14:11:32 smithi148 bash[33263]: a4c89d7bac084a3590523e174031c318270badf7857a44b265729b0e5d1bb273 2024-02-13T14:11:33.118 INFO:journalctl@ceph.mgr.x.smithi148.stdout:Feb 13 14:11:32 smithi148 systemd[1]: Started Ceph mgr.x for 5c04b074-ca79-11ee-95b9-87774f69a715. 2024-02-13T14:11:33.427 INFO:teuthology.orchestra.run.smithi087.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-02-13T14:11:33.427 INFO:teuthology.orchestra.run.smithi087.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-02-13T14:11:33.427 INFO:teuthology.orchestra.run.smithi087.stderr: stderr: 10+0 records in 2024-02-13T14:11:33.427 INFO:teuthology.orchestra.run.smithi087.stderr:10+0 records out 2024-02-13T14:11:33.428 INFO:teuthology.orchestra.run.smithi087.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0119478 s, 878 MB/s 2024-02-13T14:11:33.428 INFO:teuthology.orchestra.run.smithi087.stderr:--> Zapping successful for: 2024-02-13T14:11:33.618 INFO:journalctl@ceph.mgr.x.smithi148.stdout:Feb 13 14:11:33 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-x[33278]: 2024-02-13T14:11:33.253+0000 7f827df36200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-13T14:11:33.619 INFO:journalctl@ceph.mgr.x.smithi148.stdout:Feb 13 14:11:33 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-x[33278]: 2024-02-13T14:11:33.353+0000 7f827df36200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-13T14:11:33.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:33 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:33.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:33 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:33.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:33 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:33.993 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:33 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:33.993 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:33 smithi087 ceph-mon[27372]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:11:33.993 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:33 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:11:33.993 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:33 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:33.993 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:33 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:33.993 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:33 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:33.993 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:33 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:33.993 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:33 smithi087 ceph-mon[39120]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:11:33.993 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:33 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:11:34.018 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5c04b074-ca79-11ee-95b9-87774f69a715 -- ceph orch daemon add osd smithi087:vg_nvme/lv_4 2024-02-13T14:11:34.047 INFO:journalctl@ceph.mgr.x.smithi148.stdout:Feb 13 14:11:34 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-x[33278]: 2024-02-13T14:11:34.045+0000 7f827df36200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-13T14:11:34.047 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:33 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:34.047 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:33 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:34.047 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:33 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:34.047 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:33 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:34.047 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:33 smithi148 ceph-mon[31198]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:11:34.047 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:33 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:11:34.255 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/mon.a/config 2024-02-13T14:11:34.368 INFO:journalctl@ceph.mgr.x.smithi148.stdout:Feb 13 14:11:34 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-x[33278]: 2024-02-13T14:11:34.138+0000 7f827df36200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-13T14:11:34.368 INFO:journalctl@ceph.mgr.x.smithi148.stdout:Feb 13 14:11:34 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-x[33278]: 2024-02-13T14:11:34.286+0000 7f827df36200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-13T14:11:34.730 INFO:journalctl@ceph.mgr.x.smithi148.stdout:Feb 13 14:11:34 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-x[33278]: 2024-02-13T14:11:34.384+0000 7f827df36200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-13T14:11:34.730 INFO:journalctl@ceph.mgr.x.smithi148.stdout:Feb 13 14:11:34 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-x[33278]: 2024-02-13T14:11:34.466+0000 7f827df36200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-13T14:11:34.731 INFO:journalctl@ceph.mgr.x.smithi148.stdout:Feb 13 14:11:34 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-x[33278]: 2024-02-13T14:11:34.728+0000 7f827df36200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-13T14:11:35.021 INFO:journalctl@ceph.mgr.x.smithi148.stdout:Feb 13 14:11:34 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-x[33278]: 2024-02-13T14:11:34.838+0000 7f827df36200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-13T14:11:35.021 INFO:journalctl@ceph.mgr.x.smithi148.stdout:Feb 13 14:11:34 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-x[33278]: 2024-02-13T14:11:34.924+0000 7f827df36200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-13T14:11:35.022 INFO:journalctl@ceph.mgr.x.smithi148.stdout:Feb 13 14:11:35 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-x[33278]: 2024-02-13T14:11:35.019+0000 7f827df36200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-13T14:11:35.022 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:34 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:35.022 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:34 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:35.022 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:34 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:35.022 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:34 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:11:35.022 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:34 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:11:35.022 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:34 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:35.022 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:34 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-13T14:11:35.023 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:34 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-13T14:11:35.023 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:34 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:11:35.204 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:34 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:35.205 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:34 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:35.205 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:34 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:35.205 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:34 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:11:35.205 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:34 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:11:35.205 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:34 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:35.205 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:34 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-13T14:11:35.205 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:34 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-13T14:11:35.205 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:34 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:11:35.205 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:34 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:35.205 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:34 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:35.205 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:34 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:35.206 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:34 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:11:35.206 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:34 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:11:35.206 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:34 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:35.206 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:34 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-13T14:11:35.206 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:34 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-13T14:11:35.206 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:34 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:11:35.368 INFO:journalctl@ceph.mgr.x.smithi148.stdout:Feb 13 14:11:35 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-x[33278]: 2024-02-13T14:11:35.196+0000 7f827df36200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-13T14:11:36.201 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:35 smithi087 ceph-mon[27372]: Reconfiguring mgr.y (unknown last config time)... 2024-02-13T14:11:36.201 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:35 smithi087 ceph-mon[27372]: Reconfiguring daemon mgr.y on smithi087 2024-02-13T14:11:36.201 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:35 smithi087 ceph-mon[27372]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:11:36.201 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:35 smithi087 ceph-mon[39120]: Reconfiguring mgr.y (unknown last config time)... 2024-02-13T14:11:36.201 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:35 smithi087 ceph-mon[39120]: Reconfiguring daemon mgr.y on smithi087 2024-02-13T14:11:36.202 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:35 smithi087 ceph-mon[39120]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:11:36.301 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:35 smithi148 ceph-mon[31198]: Reconfiguring mgr.y (unknown last config time)... 2024-02-13T14:11:36.302 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:35 smithi148 ceph-mon[31198]: Reconfiguring daemon mgr.y on smithi087 2024-02-13T14:11:36.302 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:35 smithi148 ceph-mon[31198]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:11:36.302 INFO:journalctl@ceph.mgr.x.smithi148.stdout:Feb 13 14:11:36 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-x[33278]: 2024-02-13T14:11:36.169+0000 7f827df36200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-13T14:11:36.302 INFO:journalctl@ceph.mgr.x.smithi148.stdout:Feb 13 14:11:36 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-x[33278]: 2024-02-13T14:11:36.299+0000 7f827df36200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-13T14:11:36.868 INFO:journalctl@ceph.mgr.x.smithi148.stdout:Feb 13 14:11:36 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-x[33278]: 2024-02-13T14:11:36.651+0000 7f827df36200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-13T14:11:36.868 INFO:journalctl@ceph.mgr.x.smithi148.stdout:Feb 13 14:11:36 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-x[33278]: 2024-02-13T14:11:36.742+0000 7f827df36200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-13T14:11:37.368 INFO:journalctl@ceph.mgr.x.smithi148.stdout:Feb 13 14:11:36 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-x[33278]: 2024-02-13T14:11:36.941+0000 7f827df36200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-13T14:11:37.368 INFO:journalctl@ceph.mgr.x.smithi148.stdout:Feb 13 14:11:37 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-x[33278]: 2024-02-13T14:11:37.125+0000 7f827df36200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-13T14:11:37.676 INFO:journalctl@ceph.mgr.x.smithi148.stdout:Feb 13 14:11:37 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-x[33278]: 2024-02-13T14:11:37.412+0000 7f827df36200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-13T14:11:37.676 INFO:journalctl@ceph.mgr.x.smithi148.stdout:Feb 13 14:11:37 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-x[33278]: 2024-02-13T14:11:37.674+0000 7f827df36200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-13T14:11:37.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:37 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:37.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:37 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:37.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:37 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:11:37.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:37 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:11:37.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:37 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:11:37.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:37 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:37.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:37 smithi087 ceph-mon[27372]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:11:37.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:37 smithi087 ceph-mon[27372]: from='client.14214 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi087:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T14:11:37.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:37 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-13T14:11:37.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:37 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-13T14:11:37.993 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:37 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:11:37.993 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:37 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:37.993 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:37 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:37.993 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:37 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:11:37.993 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:37 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:11:37.993 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:37 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:11:37.993 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:37 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:37.993 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:37 smithi087 ceph-mon[39120]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:11:37.993 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:37 smithi087 ceph-mon[39120]: from='client.14214 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi087:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T14:11:37.994 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:37 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-13T14:11:37.994 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:37 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-13T14:11:37.994 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:37 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:11:38.119 INFO:journalctl@ceph.mgr.x.smithi148.stdout:Feb 13 14:11:37 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-x[33278]: 2024-02-13T14:11:37.914+0000 7f827df36200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-13T14:11:38.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:37 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:38.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:37 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:38.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:37 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:11:38.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:37 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:11:38.120 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:37 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:11:38.120 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:37 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:38.120 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:37 smithi148 ceph-mon[31198]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:11:38.120 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:37 smithi148 ceph-mon[31198]: from='client.14214 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi087:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T14:11:38.120 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:37 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-13T14:11:38.120 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:37 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-13T14:11:38.120 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:37 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:11:39.118 INFO:journalctl@ceph.mgr.x.smithi148.stdout:Feb 13 14:11:38 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-x[33278]: 2024-02-13T14:11:38.724+0000 7f827df36200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-13T14:11:39.613 INFO:journalctl@ceph.mgr.x.smithi148.stdout:Feb 13 14:11:39 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-x[33278]: 2024-02-13T14:11:39.296+0000 7f827df36200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-13T14:11:39.613 INFO:journalctl@ceph.mgr.x.smithi148.stdout:Feb 13 14:11:39 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-x[33278]: 2024-02-13T14:11:39.394+0000 7f827df36200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-13T14:11:39.613 INFO:journalctl@ceph.mgr.x.smithi148.stdout:Feb 13 14:11:39 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-x[33278]: 2024-02-13T14:11:39.611+0000 7f827df36200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-13T14:11:40.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:39 smithi087 ceph-mon[27372]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:11:40.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:39 smithi087 ceph-mon[27372]: Standby manager daemon x started 2024-02-13T14:11:40.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:39 smithi087 ceph-mon[27372]: from='mgr.? 172.21.15.148:0/1990112717' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-02-13T14:11:40.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:39 smithi087 ceph-mon[27372]: from='mgr.? 172.21.15.148:0/1990112717' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-13T14:11:40.243 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:39 smithi087 ceph-mon[27372]: from='mgr.? 172.21.15.148:0/1990112717' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-02-13T14:11:40.243 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:39 smithi087 ceph-mon[27372]: from='mgr.? 172.21.15.148:0/1990112717' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-13T14:11:40.243 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:39 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/107628776' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "efef6830-b1ff-4496-a434-dfd495a55a6e"}]: dispatch 2024-02-13T14:11:40.243 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:39 smithi087 ceph-mon[27372]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "efef6830-b1ff-4496-a434-dfd495a55a6e"}]: dispatch 2024-02-13T14:11:40.243 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:39 smithi087 ceph-mon[27372]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "efef6830-b1ff-4496-a434-dfd495a55a6e"}]': finished 2024-02-13T14:11:40.243 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:39 smithi087 ceph-mon[27372]: osdmap e5: 1 total, 0 up, 1 in 2024-02-13T14:11:40.243 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:39 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-13T14:11:40.244 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:39 smithi087 ceph-mon[39120]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:11:40.244 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:39 smithi087 ceph-mon[39120]: Standby manager daemon x started 2024-02-13T14:11:40.244 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:39 smithi087 ceph-mon[39120]: from='mgr.? 172.21.15.148:0/1990112717' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-02-13T14:11:40.244 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:39 smithi087 ceph-mon[39120]: from='mgr.? 172.21.15.148:0/1990112717' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-13T14:11:40.245 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:39 smithi087 ceph-mon[39120]: from='mgr.? 172.21.15.148:0/1990112717' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-02-13T14:11:40.245 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:39 smithi087 ceph-mon[39120]: from='mgr.? 172.21.15.148:0/1990112717' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-13T14:11:40.245 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:39 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/107628776' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "efef6830-b1ff-4496-a434-dfd495a55a6e"}]: dispatch 2024-02-13T14:11:40.245 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:39 smithi087 ceph-mon[39120]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "efef6830-b1ff-4496-a434-dfd495a55a6e"}]: dispatch 2024-02-13T14:11:40.245 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:39 smithi087 ceph-mon[39120]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "efef6830-b1ff-4496-a434-dfd495a55a6e"}]': finished 2024-02-13T14:11:40.245 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:39 smithi087 ceph-mon[39120]: osdmap e5: 1 total, 0 up, 1 in 2024-02-13T14:11:40.245 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:39 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-13T14:11:40.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:39 smithi148 ceph-mon[31198]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:11:40.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:39 smithi148 ceph-mon[31198]: Standby manager daemon x started 2024-02-13T14:11:40.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:39 smithi148 ceph-mon[31198]: from='mgr.? 172.21.15.148:0/1990112717' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-02-13T14:11:40.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:39 smithi148 ceph-mon[31198]: from='mgr.? 172.21.15.148:0/1990112717' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-13T14:11:40.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:39 smithi148 ceph-mon[31198]: from='mgr.? 172.21.15.148:0/1990112717' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-02-13T14:11:40.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:39 smithi148 ceph-mon[31198]: from='mgr.? 172.21.15.148:0/1990112717' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-13T14:11:40.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:39 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/107628776' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "efef6830-b1ff-4496-a434-dfd495a55a6e"}]: dispatch 2024-02-13T14:11:40.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:39 smithi148 ceph-mon[31198]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "efef6830-b1ff-4496-a434-dfd495a55a6e"}]: dispatch 2024-02-13T14:11:40.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:39 smithi148 ceph-mon[31198]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "efef6830-b1ff-4496-a434-dfd495a55a6e"}]': finished 2024-02-13T14:11:40.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:39 smithi148 ceph-mon[31198]: osdmap e5: 1 total, 0 up, 1 in 2024-02-13T14:11:40.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:39 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-13T14:11:41.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:40 smithi087 ceph-mon[27372]: mgrmap e14: y(active, since 71s), standbys: x 2024-02-13T14:11:41.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:40 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-13T14:11:41.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:40 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/2606640132' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-13T14:11:41.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:40 smithi087 ceph-mon[39120]: mgrmap e14: y(active, since 71s), standbys: x 2024-02-13T14:11:41.243 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:40 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-13T14:11:41.243 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:40 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/2606640132' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-13T14:11:41.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:40 smithi148 ceph-mon[31198]: mgrmap e14: y(active, since 71s), standbys: x 2024-02-13T14:11:41.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:40 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-13T14:11:41.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:40 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/2606640132' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-13T14:11:42.241 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:41 smithi087 ceph-mon[27372]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:11:42.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:41 smithi087 ceph-mon[39120]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:11:42.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:41 smithi148 ceph-mon[31198]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:11:44.209 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:43 smithi087 ceph-mon[27372]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:11:44.210 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:43 smithi087 ceph-mon[39120]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:11:44.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:43 smithi148 ceph-mon[31198]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:11:46.119 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:45 smithi087 ceph-mon[27372]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:11:46.120 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:45 smithi087 ceph-mon[39120]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:11:46.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:45 smithi148 ceph-mon[31198]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:11:47.089 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:46 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-13T14:11:47.089 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:46 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:11:47.089 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:46 smithi087 ceph-mon[27372]: Deploying daemon osd.0 on smithi087 2024-02-13T14:11:47.089 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:46 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-13T14:11:47.089 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:46 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:11:47.089 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:46 smithi087 ceph-mon[39120]: Deploying daemon osd.0 on smithi087 2024-02-13T14:11:47.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:46 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-13T14:11:47.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:46 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:11:47.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:46 smithi148 ceph-mon[31198]: Deploying daemon osd.0 on smithi087 2024-02-13T14:11:48.228 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:47 smithi087 ceph-mon[27372]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:11:48.228 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:47 smithi087 ceph-mon[39120]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:11:48.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:47 smithi148 ceph-mon[31198]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:11:49.984 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:49 smithi087 ceph-mon[27372]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:11:49.985 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:49 smithi087 ceph-mon[39120]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:11:50.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:49 smithi148 ceph-mon[31198]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:11:52.241 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:51 smithi087 ceph-mon[27372]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:11:52.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:51 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:52.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:51 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:52.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:51 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:11:52.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:51 smithi087 ceph-mon[39120]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:11:52.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:51 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:52.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:51 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:52.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:51 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:11:52.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:51 smithi148 ceph-mon[31198]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:11:52.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:51 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:52.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:51 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:52.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:51 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:11:54.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:53 smithi087 ceph-mon[27372]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:11:54.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:53 smithi087 ceph-mon[39120]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:11:54.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:53 smithi148 ceph-mon[31198]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:11:55.595 INFO:teuthology.orchestra.run.smithi087.stdout:Created osd(s) 0 on host 'smithi087' 2024-02-13T14:11:55.864 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:55 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:55.864 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:55 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:55.864 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:55 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:11:55.864 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:55 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:11:55.864 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:55 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:55.864 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:55 smithi087 ceph-mon[39120]: from='osd.0 [v2:172.21.15.87:6802/2357055010,v1:172.21.15.87:6803/2357055010]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-13T14:11:55.864 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:55 smithi087 ceph-mon[39120]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:11:55.864 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:55 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:55.864 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:55 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:55.864 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:55 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:11:55.865 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:55 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:55.865 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:55 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:55.865 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:55 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:11:55.865 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:55 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:11:55.865 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:55 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:55.865 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:55 smithi087 ceph-mon[27372]: from='osd.0 [v2:172.21.15.87:6802/2357055010,v1:172.21.15.87:6803/2357055010]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-13T14:11:55.865 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:55 smithi087 ceph-mon[27372]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:11:55.865 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:55 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:55.865 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:55 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:55.865 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:55 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:11:56.118 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:55 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:56.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:55 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:56.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:55 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:11:56.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:55 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:11:56.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:55 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:56.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:55 smithi148 ceph-mon[31198]: from='osd.0 [v2:172.21.15.87:6802/2357055010,v1:172.21.15.87:6803/2357055010]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-13T14:11:56.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:55 smithi148 ceph-mon[31198]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:11:56.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:55 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:56.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:55 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:11:56.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:55 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:11:56.190 DEBUG:teuthology.orchestra.run.smithi087:osd.0> sudo journalctl -f -n 0 -u ceph-5c04b074-ca79-11ee-95b9-87774f69a715@osd.0.service 2024-02-13T14:11:56.193 INFO:tasks.cephadm:Deploying osd.1 on smithi087 with /dev/vg_nvme/lv_3... 2024-02-13T14:11:56.193 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5c04b074-ca79-11ee-95b9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-02-13T14:11:56.951 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/mon.a/config 2024-02-13T14:11:56.991 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:56 smithi087 ceph-mon[27372]: from='osd.0 [v2:172.21.15.87:6802/2357055010,v1:172.21.15.87:6803/2357055010]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-02-13T14:11:56.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:56 smithi087 ceph-mon[27372]: osdmap e6: 1 total, 0 up, 1 in 2024-02-13T14:11:56.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:56 smithi087 ceph-mon[27372]: from='osd.0 [v2:172.21.15.87:6802/2357055010,v1:172.21.15.87:6803/2357055010]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]: dispatch 2024-02-13T14:11:56.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:56 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-13T14:11:56.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:56 smithi087 ceph-mon[39120]: from='osd.0 [v2:172.21.15.87:6802/2357055010,v1:172.21.15.87:6803/2357055010]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-02-13T14:11:56.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:56 smithi087 ceph-mon[39120]: osdmap e6: 1 total, 0 up, 1 in 2024-02-13T14:11:56.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:56 smithi087 ceph-mon[39120]: from='osd.0 [v2:172.21.15.87:6802/2357055010,v1:172.21.15.87:6803/2357055010]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]: dispatch 2024-02-13T14:11:56.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:56 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-13T14:11:57.118 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:56 smithi148 ceph-mon[31198]: from='osd.0 [v2:172.21.15.87:6802/2357055010,v1:172.21.15.87:6803/2357055010]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-02-13T14:11:57.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:56 smithi148 ceph-mon[31198]: osdmap e6: 1 total, 0 up, 1 in 2024-02-13T14:11:57.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:56 smithi148 ceph-mon[31198]: from='osd.0 [v2:172.21.15.87:6802/2357055010,v1:172.21.15.87:6803/2357055010]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]: dispatch 2024-02-13T14:11:57.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:56 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-13T14:11:57.371 INFO:journalctl@ceph.osd.0.smithi087.stdout:Feb 13 14:11:57 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-0[43500]: 2024-02-13T14:11:57.100+0000 7f0a960f0700 -1 osd.0 0 waiting for initial osdmap 2024-02-13T14:11:57.371 INFO:journalctl@ceph.osd.0.smithi087.stdout:Feb 13 14:11:57 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-0[43500]: 2024-02-13T14:11:57.102+0000 7f0a8dedb700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-13T14:11:57.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:57 smithi087 ceph-mon[27372]: from='osd.0 [v2:172.21.15.87:6802/2357055010,v1:172.21.15.87:6803/2357055010]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]': finished 2024-02-13T14:11:57.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:57 smithi087 ceph-mon[27372]: osdmap e7: 1 total, 0 up, 1 in 2024-02-13T14:11:57.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:57 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-13T14:11:57.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:57 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-13T14:11:57.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:57 smithi087 ceph-mon[27372]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:11:57.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:57 smithi087 ceph-mon[27372]: from='osd.0 [v2:172.21.15.87:6802/2357055010,v1:172.21.15.87:6803/2357055010]' entity='osd.0' 2024-02-13T14:11:57.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:57 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-13T14:11:57.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:57 smithi087 ceph-mon[39120]: from='osd.0 [v2:172.21.15.87:6802/2357055010,v1:172.21.15.87:6803/2357055010]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]': finished 2024-02-13T14:11:57.993 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:57 smithi087 ceph-mon[39120]: osdmap e7: 1 total, 0 up, 1 in 2024-02-13T14:11:57.993 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:57 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-13T14:11:57.993 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:57 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-13T14:11:57.993 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:57 smithi087 ceph-mon[39120]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:11:57.993 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:57 smithi087 ceph-mon[39120]: from='osd.0 [v2:172.21.15.87:6802/2357055010,v1:172.21.15.87:6803/2357055010]' entity='osd.0' 2024-02-13T14:11:57.993 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:57 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-13T14:11:58.118 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:57 smithi148 ceph-mon[31198]: from='osd.0 [v2:172.21.15.87:6802/2357055010,v1:172.21.15.87:6803/2357055010]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]': finished 2024-02-13T14:11:58.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:57 smithi148 ceph-mon[31198]: osdmap e7: 1 total, 0 up, 1 in 2024-02-13T14:11:58.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:57 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-13T14:11:58.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:57 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-13T14:11:58.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:57 smithi148 ceph-mon[31198]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:11:58.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:57 smithi148 ceph-mon[31198]: from='osd.0 [v2:172.21.15.87:6802/2357055010,v1:172.21.15.87:6803/2357055010]' entity='osd.0' 2024-02-13T14:11:58.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:57 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-13T14:11:58.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:58 smithi087 ceph-mon[27372]: purged_snaps scrub starts 2024-02-13T14:11:58.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:58 smithi087 ceph-mon[27372]: purged_snaps scrub ok 2024-02-13T14:11:58.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:58 smithi087 ceph-mon[27372]: osd.0 [v2:172.21.15.87:6802/2357055010,v1:172.21.15.87:6803/2357055010] boot 2024-02-13T14:11:58.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:58 smithi087 ceph-mon[27372]: osdmap e8: 1 total, 1 up, 1 in 2024-02-13T14:11:58.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:58 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-13T14:11:58.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:58 smithi087 ceph-mon[39120]: purged_snaps scrub starts 2024-02-13T14:11:58.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:58 smithi087 ceph-mon[39120]: purged_snaps scrub ok 2024-02-13T14:11:58.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:58 smithi087 ceph-mon[39120]: osd.0 [v2:172.21.15.87:6802/2357055010,v1:172.21.15.87:6803/2357055010] boot 2024-02-13T14:11:58.743 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:58 smithi087 ceph-mon[39120]: osdmap e8: 1 total, 1 up, 1 in 2024-02-13T14:11:58.743 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:58 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-13T14:11:59.118 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:58 smithi148 ceph-mon[31198]: purged_snaps scrub starts 2024-02-13T14:11:59.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:58 smithi148 ceph-mon[31198]: purged_snaps scrub ok 2024-02-13T14:11:59.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:58 smithi148 ceph-mon[31198]: osd.0 [v2:172.21.15.87:6802/2357055010,v1:172.21.15.87:6803/2357055010] boot 2024-02-13T14:11:59.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:58 smithi148 ceph-mon[31198]: osdmap e8: 1 total, 1 up, 1 in 2024-02-13T14:11:59.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:58 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-13T14:11:59.713 INFO:teuthology.orchestra.run.smithi087.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-02-13T14:11:59.713 INFO:teuthology.orchestra.run.smithi087.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-02-13T14:11:59.713 INFO:teuthology.orchestra.run.smithi087.stderr: stderr: 10+0 records in 2024-02-13T14:11:59.713 INFO:teuthology.orchestra.run.smithi087.stderr:10+0 records out 2024-02-13T14:11:59.713 INFO:teuthology.orchestra.run.smithi087.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0143545 s, 730 MB/s 2024-02-13T14:11:59.713 INFO:teuthology.orchestra.run.smithi087.stderr: stderr: 2024-02-13T14:11:59.713 INFO:teuthology.orchestra.run.smithi087.stderr:--> Zapping successful for: 2024-02-13T14:11:59.991 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:11:59 smithi087 ceph-mon[39120]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-13T14:11:59.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:11:59 smithi087 ceph-mon[27372]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-13T14:12:00.118 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:11:59 smithi148 ceph-mon[31198]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-13T14:12:00.991 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:00 smithi087 ceph-mon[27372]: osdmap e9: 1 total, 1 up, 1 in 2024-02-13T14:12:00.991 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:00 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:12:00.991 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:00 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:12:00.991 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:00 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-13T14:12:00.991 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:00 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:12:00.991 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:00 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:12:00.991 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:00 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:12:00.991 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:00 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:12:00.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:00 smithi087 ceph-mon[39120]: osdmap e9: 1 total, 1 up, 1 in 2024-02-13T14:12:00.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:00 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:12:00.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:00 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:12:00.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:00 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-13T14:12:00.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:00 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:12:00.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:00 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:12:00.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:00 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:12:00.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:00 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:12:01.017 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5c04b074-ca79-11ee-95b9-87774f69a715 -- ceph orch daemon add osd smithi087:vg_nvme/lv_3 2024-02-13T14:12:01.118 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:00 smithi148 ceph-mon[31198]: osdmap e9: 1 total, 1 up, 1 in 2024-02-13T14:12:01.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:00 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:12:01.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:00 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:12:01.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:00 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-13T14:12:01.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:00 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:12:01.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:00 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:12:01.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:00 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:12:01.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:00 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:12:01.251 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/mon.a/config 2024-02-13T14:12:01.991 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:01 smithi087 ceph-mon[39120]: Detected new or changed devices on smithi087 2024-02-13T14:12:01.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:01 smithi087 ceph-mon[39120]: Adjusting osd_memory_target on smithi087 to 16064M 2024-02-13T14:12:01.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:01 smithi087 ceph-mon[39120]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-13T14:12:01.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:01 smithi087 ceph-mon[27372]: Detected new or changed devices on smithi087 2024-02-13T14:12:01.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:01 smithi087 ceph-mon[27372]: Adjusting osd_memory_target on smithi087 to 16064M 2024-02-13T14:12:01.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:01 smithi087 ceph-mon[27372]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-13T14:12:02.118 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:01 smithi148 ceph-mon[31198]: Detected new or changed devices on smithi087 2024-02-13T14:12:02.118 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:01 smithi148 ceph-mon[31198]: Adjusting osd_memory_target on smithi087 to 16064M 2024-02-13T14:12:02.118 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:01 smithi148 ceph-mon[31198]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-13T14:12:04.237 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:03 smithi087 ceph-mon[27372]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-13T14:12:04.237 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:03 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-13T14:12:04.237 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:03 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-13T14:12:04.237 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:03 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:12:04.237 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:03 smithi087 ceph-mon[39120]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-13T14:12:04.237 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:03 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-13T14:12:04.237 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:03 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-13T14:12:04.237 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:03 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:12:04.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:03 smithi148 ceph-mon[31198]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-13T14:12:04.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:03 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-13T14:12:04.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:03 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-13T14:12:04.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:03 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:12:05.241 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:04 smithi087 ceph-mon[27372]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi087:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T14:12:05.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:04 smithi087 ceph-mon[39120]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi087:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T14:12:05.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:04 smithi148 ceph-mon[31198]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi087:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T14:12:06.241 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:05 smithi087 ceph-mon[39120]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-13T14:12:06.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:05 smithi087 ceph-mon[27372]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-13T14:12:06.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:05 smithi148 ceph-mon[31198]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-13T14:12:07.118 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:06 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/522847015' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "399d507f-ed8f-4f3f-83c3-a3f5fdbaeefd"}]: dispatch 2024-02-13T14:12:07.118 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:06 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/522847015' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "399d507f-ed8f-4f3f-83c3-a3f5fdbaeefd"}]': finished 2024-02-13T14:12:07.118 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:06 smithi148 ceph-mon[31198]: osdmap e10: 2 total, 1 up, 2 in 2024-02-13T14:12:07.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:06 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-13T14:12:07.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:06 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/2215084559' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-13T14:12:07.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:06 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/522847015' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "399d507f-ed8f-4f3f-83c3-a3f5fdbaeefd"}]: dispatch 2024-02-13T14:12:07.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:06 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/522847015' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "399d507f-ed8f-4f3f-83c3-a3f5fdbaeefd"}]': finished 2024-02-13T14:12:07.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:06 smithi087 ceph-mon[27372]: osdmap e10: 2 total, 1 up, 2 in 2024-02-13T14:12:07.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:06 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-13T14:12:07.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:06 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/2215084559' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-13T14:12:07.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:06 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/522847015' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "399d507f-ed8f-4f3f-83c3-a3f5fdbaeefd"}]: dispatch 2024-02-13T14:12:07.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:06 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/522847015' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "399d507f-ed8f-4f3f-83c3-a3f5fdbaeefd"}]': finished 2024-02-13T14:12:07.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:06 smithi087 ceph-mon[39120]: osdmap e10: 2 total, 1 up, 2 in 2024-02-13T14:12:07.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:06 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-13T14:12:07.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:06 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/2215084559' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-13T14:12:08.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:08 smithi148 ceph-mon[31198]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-13T14:12:08.491 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:08 smithi087 ceph-mon[27372]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-13T14:12:08.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:08 smithi087 ceph-mon[39120]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-13T14:12:10.472 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:10 smithi087 ceph-mon[27372]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-13T14:12:10.472 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:10 smithi087 ceph-mon[39120]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-13T14:12:10.618 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:10 smithi148 ceph-mon[31198]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-13T14:12:12.361 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:12 smithi087 ceph-mon[39120]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-13T14:12:12.361 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:12 smithi087 ceph-mon[27372]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-13T14:12:12.618 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:12 smithi148 ceph-mon[31198]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-13T14:12:13.430 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:13 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-13T14:12:13.431 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:13 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:12:13.431 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:13 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-13T14:12:13.431 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:13 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:12:13.618 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:13 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-13T14:12:13.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:13 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:12:14.318 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:14 smithi087 ceph-mon[39120]: Deploying daemon osd.1 on smithi087 2024-02-13T14:12:14.318 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:14 smithi087 ceph-mon[39120]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-13T14:12:14.318 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:14 smithi087 ceph-mon[27372]: Deploying daemon osd.1 on smithi087 2024-02-13T14:12:14.318 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:14 smithi087 ceph-mon[27372]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-13T14:12:14.618 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:14 smithi148 ceph-mon[31198]: Deploying daemon osd.1 on smithi087 2024-02-13T14:12:14.618 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:14 smithi148 ceph-mon[31198]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-13T14:12:16.450 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:16 smithi087 ceph-mon[27372]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-13T14:12:16.450 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:16 smithi087 ceph-mon[39120]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-13T14:12:16.618 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:16 smithi148 ceph-mon[31198]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-13T14:12:18.491 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:18 smithi087 ceph-mon[27372]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-13T14:12:18.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:18 smithi087 ceph-mon[39120]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-13T14:12:18.618 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:18 smithi148 ceph-mon[31198]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-13T14:12:19.618 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:19 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:12:19.618 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:19 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:12:19.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:19 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:12:19.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:19 smithi148 ceph-mon[31198]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-13T14:12:19.741 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:19 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:12:19.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:19 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:12:19.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:19 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:12:19.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:19 smithi087 ceph-mon[27372]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-13T14:12:19.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:19 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:12:19.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:19 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:12:19.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:19 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:12:19.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:19 smithi087 ceph-mon[39120]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-13T14:12:22.121 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:21 smithi087 ceph-mon[39120]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-13T14:12:22.121 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:21 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:12:22.121 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:21 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:12:22.121 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:21 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:12:22.121 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:21 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:12:22.122 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:21 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:12:22.122 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:21 smithi087 ceph-mon[39120]: from='osd.1 [v2:172.21.15.87:6810/1433765078,v1:172.21.15.87:6811/1433765078]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-13T14:12:22.122 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:21 smithi087 ceph-mon[27372]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-13T14:12:22.122 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:21 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:12:22.122 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:21 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:12:22.122 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:21 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:12:22.122 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:21 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:12:22.122 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:21 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:12:22.122 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:21 smithi087 ceph-mon[27372]: from='osd.1 [v2:172.21.15.87:6810/1433765078,v1:172.21.15.87:6811/1433765078]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-13T14:12:22.162 INFO:teuthology.orchestra.run.smithi087.stdout:Created osd(s) 1 on host 'smithi087' 2024-02-13T14:12:22.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:21 smithi148 ceph-mon[31198]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-13T14:12:22.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:21 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:12:22.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:21 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:12:22.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:21 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:12:22.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:21 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:12:22.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:21 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:12:22.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:21 smithi148 ceph-mon[31198]: from='osd.1 [v2:172.21.15.87:6810/1433765078,v1:172.21.15.87:6811/1433765078]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-13T14:12:22.797 DEBUG:teuthology.orchestra.run.smithi087:osd.1> sudo journalctl -f -n 0 -u ceph-5c04b074-ca79-11ee-95b9-87774f69a715@osd.1.service 2024-02-13T14:12:22.799 INFO:tasks.cephadm:Deploying osd.2 on smithi087 with /dev/vg_nvme/lv_2... 2024-02-13T14:12:22.799 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5c04b074-ca79-11ee-95b9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-02-13T14:12:23.370 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:23 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:12:23.370 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:23 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:12:23.371 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:23 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:12:23.371 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:23 smithi087 ceph-mon[39120]: from='osd.1 [v2:172.21.15.87:6810/1433765078,v1:172.21.15.87:6811/1433765078]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-02-13T14:12:23.371 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:23 smithi087 ceph-mon[39120]: osdmap e11: 2 total, 1 up, 2 in 2024-02-13T14:12:23.371 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:23 smithi087 ceph-mon[39120]: from='osd.1 [v2:172.21.15.87:6810/1433765078,v1:172.21.15.87:6811/1433765078]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]: dispatch 2024-02-13T14:12:23.371 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:23 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-13T14:12:23.371 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:23 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:12:23.371 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:23 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:12:23.371 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:23 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:12:23.371 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:23 smithi087 ceph-mon[27372]: from='osd.1 [v2:172.21.15.87:6810/1433765078,v1:172.21.15.87:6811/1433765078]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-02-13T14:12:23.371 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:23 smithi087 ceph-mon[27372]: osdmap e11: 2 total, 1 up, 2 in 2024-02-13T14:12:23.372 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:23 smithi087 ceph-mon[27372]: from='osd.1 [v2:172.21.15.87:6810/1433765078,v1:172.21.15.87:6811/1433765078]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]: dispatch 2024-02-13T14:12:23.372 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:23 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-13T14:12:23.372 INFO:journalctl@ceph.osd.1.smithi087.stdout:Feb 13 14:12:23 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-1[49621]: 2024-02-13T14:12:23.263+0000 7f213befd700 -1 osd.1 0 waiting for initial osdmap 2024-02-13T14:12:23.372 INFO:journalctl@ceph.osd.1.smithi087.stdout:Feb 13 14:12:23 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-1[49621]: 2024-02-13T14:12:23.266+0000 7f2137cf0700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-13T14:12:23.605 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/mon.a/config 2024-02-13T14:12:23.618 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:23 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:12:23.618 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:23 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:12:23.618 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:23 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:12:23.618 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:23 smithi148 ceph-mon[31198]: from='osd.1 [v2:172.21.15.87:6810/1433765078,v1:172.21.15.87:6811/1433765078]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-02-13T14:12:23.618 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:23 smithi148 ceph-mon[31198]: osdmap e11: 2 total, 1 up, 2 in 2024-02-13T14:12:23.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:23 smithi148 ceph-mon[31198]: from='osd.1 [v2:172.21.15.87:6810/1433765078,v1:172.21.15.87:6811/1433765078]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]: dispatch 2024-02-13T14:12:23.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:23 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-13T14:12:24.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:24 smithi087 ceph-mon[39120]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-13T14:12:24.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:24 smithi087 ceph-mon[39120]: from='osd.1 [v2:172.21.15.87:6810/1433765078,v1:172.21.15.87:6811/1433765078]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]': finished 2024-02-13T14:12:24.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:24 smithi087 ceph-mon[39120]: osdmap e12: 2 total, 1 up, 2 in 2024-02-13T14:12:24.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:24 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-13T14:12:24.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:24 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-13T14:12:24.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:24 smithi087 ceph-mon[27372]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-13T14:12:24.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:24 smithi087 ceph-mon[27372]: from='osd.1 [v2:172.21.15.87:6810/1433765078,v1:172.21.15.87:6811/1433765078]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]': finished 2024-02-13T14:12:24.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:24 smithi087 ceph-mon[27372]: osdmap e12: 2 total, 1 up, 2 in 2024-02-13T14:12:24.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:24 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-13T14:12:24.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:24 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-13T14:12:24.618 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:24 smithi148 ceph-mon[31198]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-13T14:12:24.618 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:24 smithi148 ceph-mon[31198]: from='osd.1 [v2:172.21.15.87:6810/1433765078,v1:172.21.15.87:6811/1433765078]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]': finished 2024-02-13T14:12:24.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:24 smithi148 ceph-mon[31198]: osdmap e12: 2 total, 1 up, 2 in 2024-02-13T14:12:24.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:24 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-13T14:12:24.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:24 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-13T14:12:25.374 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:25 smithi087 ceph-mon[39120]: purged_snaps scrub starts 2024-02-13T14:12:25.374 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:25 smithi087 ceph-mon[39120]: purged_snaps scrub ok 2024-02-13T14:12:25.374 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:25 smithi087 ceph-mon[39120]: osd.1 [v2:172.21.15.87:6810/1433765078,v1:172.21.15.87:6811/1433765078] boot 2024-02-13T14:12:25.374 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:25 smithi087 ceph-mon[39120]: osdmap e13: 2 total, 2 up, 2 in 2024-02-13T14:12:25.374 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:25 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-13T14:12:25.374 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:25 smithi087 ceph-mon[39120]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-13T14:12:25.375 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:25 smithi087 ceph-mon[27372]: purged_snaps scrub starts 2024-02-13T14:12:25.375 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:25 smithi087 ceph-mon[27372]: purged_snaps scrub ok 2024-02-13T14:12:25.375 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:25 smithi087 ceph-mon[27372]: osd.1 [v2:172.21.15.87:6810/1433765078,v1:172.21.15.87:6811/1433765078] boot 2024-02-13T14:12:25.375 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:25 smithi087 ceph-mon[27372]: osdmap e13: 2 total, 2 up, 2 in 2024-02-13T14:12:25.375 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:25 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-13T14:12:25.375 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:25 smithi087 ceph-mon[27372]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-13T14:12:25.618 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:25 smithi148 ceph-mon[31198]: purged_snaps scrub starts 2024-02-13T14:12:25.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:25 smithi148 ceph-mon[31198]: purged_snaps scrub ok 2024-02-13T14:12:25.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:25 smithi148 ceph-mon[31198]: osd.1 [v2:172.21.15.87:6810/1433765078,v1:172.21.15.87:6811/1433765078] boot 2024-02-13T14:12:25.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:25 smithi148 ceph-mon[31198]: osdmap e13: 2 total, 2 up, 2 in 2024-02-13T14:12:25.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:25 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-13T14:12:25.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:25 smithi148 ceph-mon[31198]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-13T14:12:26.618 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:26 smithi148 ceph-mon[31198]: osdmap e14: 2 total, 2 up, 2 in 2024-02-13T14:12:26.715 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:26 smithi087 ceph-mon[27372]: osdmap e14: 2 total, 2 up, 2 in 2024-02-13T14:12:26.715 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:26 smithi087 ceph-mon[39120]: osdmap e14: 2 total, 2 up, 2 in 2024-02-13T14:12:26.857 INFO:teuthology.orchestra.run.smithi087.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-02-13T14:12:26.857 INFO:teuthology.orchestra.run.smithi087.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-02-13T14:12:26.858 INFO:teuthology.orchestra.run.smithi087.stderr: stderr: 10+0 records in 2024-02-13T14:12:26.858 INFO:teuthology.orchestra.run.smithi087.stderr:10+0 records out 2024-02-13T14:12:26.858 INFO:teuthology.orchestra.run.smithi087.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0116537 s, 900 MB/s 2024-02-13T14:12:26.858 INFO:teuthology.orchestra.run.smithi087.stderr:--> Zapping successful for: 2024-02-13T14:12:27.679 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5c04b074-ca79-11ee-95b9-87774f69a715 -- ceph orch daemon add osd smithi087:vg_nvme/lv_2 2024-02-13T14:12:27.915 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/mon.a/config 2024-02-13T14:12:28.285 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:28 smithi148 ceph-mon[31198]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-13T14:12:28.285 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:28 smithi148 ceph-mon[31198]: Detected new or changed devices on smithi087 2024-02-13T14:12:28.285 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:28 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:12:28.285 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:28 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:12:28.285 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:28 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-13T14:12:28.286 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:28 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-13T14:12:28.286 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:28 smithi148 ceph-mon[31198]: Adjusting osd_memory_target on smithi087 to 8032M 2024-02-13T14:12:28.286 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:28 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:12:28.286 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:28 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:12:28.286 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:28 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:12:28.286 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:28 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:12:28.306 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:28 smithi087 ceph-mon[39120]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-13T14:12:28.306 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:28 smithi087 ceph-mon[39120]: Detected new or changed devices on smithi087 2024-02-13T14:12:28.306 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:28 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:12:28.306 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:28 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:12:28.306 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:28 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-13T14:12:28.306 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:28 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-13T14:12:28.306 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:28 smithi087 ceph-mon[39120]: Adjusting osd_memory_target on smithi087 to 8032M 2024-02-13T14:12:28.306 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:28 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:12:28.306 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:28 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:12:28.306 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:28 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:12:28.306 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:28 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:12:28.307 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:28 smithi087 ceph-mon[27372]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-13T14:12:28.307 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:28 smithi087 ceph-mon[27372]: Detected new or changed devices on smithi087 2024-02-13T14:12:28.307 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:28 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:12:28.307 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:28 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:12:28.307 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:28 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-13T14:12:28.307 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:28 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-13T14:12:28.307 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:28 smithi087 ceph-mon[27372]: Adjusting osd_memory_target on smithi087 to 8032M 2024-02-13T14:12:28.307 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:28 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:12:28.307 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:28 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:12:28.307 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:28 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:12:28.307 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:28 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:12:30.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:30 smithi148 ceph-mon[31198]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-13T14:12:30.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:30 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-13T14:12:30.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:30 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-13T14:12:30.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:30 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:12:30.413 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:30 smithi087 ceph-mon[27372]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-13T14:12:30.413 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:30 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-13T14:12:30.413 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:30 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-13T14:12:30.413 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:30 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:12:30.414 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:30 smithi087 ceph-mon[39120]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-13T14:12:30.414 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:30 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-13T14:12:30.414 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:30 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-13T14:12:30.414 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:30 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:12:31.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:31 smithi148 ceph-mon[31198]: from='client.14268 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi087:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T14:12:31.438 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:31 smithi087 ceph-mon[27372]: from='client.14268 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi087:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T14:12:31.438 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:31 smithi087 ceph-mon[39120]: from='client.14268 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi087:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T14:12:32.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:32 smithi148 ceph-mon[31198]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-13T14:12:32.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:32 smithi087 ceph-mon[27372]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-13T14:12:32.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:32 smithi087 ceph-mon[39120]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-13T14:12:33.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:33 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/3851386310' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9517393e-da9c-46b8-90ab-9937cf20659c"}]: dispatch 2024-02-13T14:12:33.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:33 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/3851386310' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9517393e-da9c-46b8-90ab-9937cf20659c"}]': finished 2024-02-13T14:12:33.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:33 smithi148 ceph-mon[31198]: osdmap e15: 3 total, 2 up, 3 in 2024-02-13T14:12:33.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:33 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-13T14:12:33.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:33 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/3851386310' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9517393e-da9c-46b8-90ab-9937cf20659c"}]: dispatch 2024-02-13T14:12:33.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:33 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/3851386310' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9517393e-da9c-46b8-90ab-9937cf20659c"}]': finished 2024-02-13T14:12:33.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:33 smithi087 ceph-mon[27372]: osdmap e15: 3 total, 2 up, 3 in 2024-02-13T14:12:33.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:33 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-13T14:12:33.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:33 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/3851386310' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9517393e-da9c-46b8-90ab-9937cf20659c"}]: dispatch 2024-02-13T14:12:33.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:33 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/3851386310' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9517393e-da9c-46b8-90ab-9937cf20659c"}]': finished 2024-02-13T14:12:33.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:33 smithi087 ceph-mon[39120]: osdmap e15: 3 total, 2 up, 3 in 2024-02-13T14:12:33.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:33 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-13T14:12:34.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:34 smithi148 ceph-mon[31198]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-13T14:12:34.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:34 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/301486749' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-13T14:12:34.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:34 smithi087 ceph-mon[27372]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-13T14:12:34.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:34 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/301486749' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-13T14:12:34.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:34 smithi087 ceph-mon[39120]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-13T14:12:34.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:34 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/301486749' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-13T14:12:36.126 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:36 smithi087 ceph-mon[27372]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-13T14:12:36.126 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:36 smithi087 ceph-mon[39120]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-13T14:12:36.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:36 smithi148 ceph-mon[31198]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-13T14:12:38.103 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:38 smithi087 ceph-mon[27372]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-13T14:12:38.103 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:38 smithi087 ceph-mon[39120]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-13T14:12:38.343 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:38 smithi148 ceph-mon[31198]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-13T14:12:40.159 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:40 smithi087 ceph-mon[27372]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-13T14:12:40.159 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:40 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-13T14:12:40.159 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:40 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:12:40.159 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:40 smithi087 ceph-mon[39120]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-13T14:12:40.159 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:40 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-13T14:12:40.160 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:40 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:12:40.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:40 smithi148 ceph-mon[31198]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-13T14:12:40.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:40 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-13T14:12:40.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:40 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:12:41.214 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:41 smithi087 ceph-mon[27372]: Deploying daemon osd.2 on smithi087 2024-02-13T14:12:41.214 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:41 smithi087 ceph-mon[39120]: Deploying daemon osd.2 on smithi087 2024-02-13T14:12:41.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:41 smithi148 ceph-mon[31198]: Deploying daemon osd.2 on smithi087 2024-02-13T14:12:42.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:42 smithi087 ceph-mon[27372]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-13T14:12:42.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:42 smithi087 ceph-mon[39120]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-13T14:12:42.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:42 smithi148 ceph-mon[31198]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-13T14:12:44.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:44 smithi148 ceph-mon[31198]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-13T14:12:44.419 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:44 smithi087 ceph-mon[27372]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-13T14:12:44.419 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:44 smithi087 ceph-mon[39120]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-13T14:12:46.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:46 smithi087 ceph-mon[39120]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-13T14:12:46.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:46 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:12:46.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:46 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:12:46.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:46 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:12:46.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:46 smithi087 ceph-mon[27372]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-13T14:12:46.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:46 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:12:46.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:46 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:12:46.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:46 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:12:46.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:46 smithi148 ceph-mon[31198]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-13T14:12:46.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:46 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:12:46.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:46 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:12:46.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:46 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:12:48.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:48 smithi148 ceph-mon[31198]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-13T14:12:48.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:48 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:12:48.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:48 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:12:48.388 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:48 smithi087 ceph-mon[27372]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-13T14:12:48.388 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:48 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:12:48.388 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:48 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:12:48.388 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:48 smithi087 ceph-mon[39120]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-13T14:12:48.388 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:48 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:12:48.389 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:48 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:12:48.808 INFO:teuthology.orchestra.run.smithi087.stdout:Created osd(s) 2 on host 'smithi087' 2024-02-13T14:12:49.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:49 smithi148 ceph-mon[31198]: from='osd.2 [v2:172.21.15.87:6818/1764493594,v1:172.21.15.87:6819/1764493594]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-13T14:12:49.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:49 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:12:49.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:49 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:12:49.400 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:49 smithi087 ceph-mon[39120]: from='osd.2 [v2:172.21.15.87:6818/1764493594,v1:172.21.15.87:6819/1764493594]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-13T14:12:49.400 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:49 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:12:49.400 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:49 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:12:49.400 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:49 smithi087 ceph-mon[27372]: from='osd.2 [v2:172.21.15.87:6818/1764493594,v1:172.21.15.87:6819/1764493594]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-13T14:12:49.400 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:49 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:12:49.400 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:49 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:12:49.428 DEBUG:teuthology.orchestra.run.smithi087:osd.2> sudo journalctl -f -n 0 -u ceph-5c04b074-ca79-11ee-95b9-87774f69a715@osd.2.service 2024-02-13T14:12:49.431 INFO:tasks.cephadm:Deploying osd.3 on smithi087 with /dev/vg_nvme/lv_1... 2024-02-13T14:12:49.431 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5c04b074-ca79-11ee-95b9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-02-13T14:12:50.173 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/mon.a/config 2024-02-13T14:12:50.295 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:50 smithi087 ceph-mon[27372]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-13T14:12:50.295 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:50 smithi087 ceph-mon[27372]: from='osd.2 [v2:172.21.15.87:6818/1764493594,v1:172.21.15.87:6819/1764493594]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-02-13T14:12:50.295 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:50 smithi087 ceph-mon[27372]: osdmap e16: 3 total, 2 up, 3 in 2024-02-13T14:12:50.295 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:50 smithi087 ceph-mon[27372]: from='osd.2 [v2:172.21.15.87:6818/1764493594,v1:172.21.15.87:6819/1764493594]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]: dispatch 2024-02-13T14:12:50.295 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:50 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-13T14:12:50.295 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:50 smithi087 ceph-mon[39120]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-13T14:12:50.295 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:50 smithi087 ceph-mon[39120]: from='osd.2 [v2:172.21.15.87:6818/1764493594,v1:172.21.15.87:6819/1764493594]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-02-13T14:12:50.296 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:50 smithi087 ceph-mon[39120]: osdmap e16: 3 total, 2 up, 3 in 2024-02-13T14:12:50.296 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:50 smithi087 ceph-mon[39120]: from='osd.2 [v2:172.21.15.87:6818/1764493594,v1:172.21.15.87:6819/1764493594]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]: dispatch 2024-02-13T14:12:50.296 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:50 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-13T14:12:50.296 INFO:journalctl@ceph.osd.2.smithi087.stdout:Feb 13 14:12:50 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-2[55599]: 2024-02-13T14:12:50.028+0000 7f77c2897700 -1 osd.2 0 waiting for initial osdmap 2024-02-13T14:12:50.296 INFO:journalctl@ceph.osd.2.smithi087.stdout:Feb 13 14:12:50 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-2[55599]: 2024-02-13T14:12:50.032+0000 7f77be68a700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-13T14:12:50.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:50 smithi148 ceph-mon[31198]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-13T14:12:50.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:50 smithi148 ceph-mon[31198]: from='osd.2 [v2:172.21.15.87:6818/1764493594,v1:172.21.15.87:6819/1764493594]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-02-13T14:12:50.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:50 smithi148 ceph-mon[31198]: osdmap e16: 3 total, 2 up, 3 in 2024-02-13T14:12:50.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:50 smithi148 ceph-mon[31198]: from='osd.2 [v2:172.21.15.87:6818/1764493594,v1:172.21.15.87:6819/1764493594]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]: dispatch 2024-02-13T14:12:50.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:50 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-13T14:12:51.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:51 smithi087 ceph-mon[27372]: from='osd.2 [v2:172.21.15.87:6818/1764493594,v1:172.21.15.87:6819/1764493594]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]': finished 2024-02-13T14:12:51.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:51 smithi087 ceph-mon[27372]: osdmap e17: 3 total, 2 up, 3 in 2024-02-13T14:12:51.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:51 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-13T14:12:51.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:51 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-13T14:12:51.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:51 smithi087 ceph-mon[39120]: from='osd.2 [v2:172.21.15.87:6818/1764493594,v1:172.21.15.87:6819/1764493594]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]': finished 2024-02-13T14:12:51.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:51 smithi087 ceph-mon[39120]: osdmap e17: 3 total, 2 up, 3 in 2024-02-13T14:12:51.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:51 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-13T14:12:51.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:51 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-13T14:12:51.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:51 smithi148 ceph-mon[31198]: from='osd.2 [v2:172.21.15.87:6818/1764493594,v1:172.21.15.87:6819/1764493594]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]': finished 2024-02-13T14:12:51.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:51 smithi148 ceph-mon[31198]: osdmap e17: 3 total, 2 up, 3 in 2024-02-13T14:12:51.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:51 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-13T14:12:51.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:51 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-13T14:12:52.308 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:52 smithi087 ceph-mon[27372]: purged_snaps scrub starts 2024-02-13T14:12:52.309 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:52 smithi087 ceph-mon[27372]: purged_snaps scrub ok 2024-02-13T14:12:52.309 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:52 smithi087 ceph-mon[27372]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-13T14:12:52.309 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:52 smithi087 ceph-mon[27372]: osd.2 [v2:172.21.15.87:6818/1764493594,v1:172.21.15.87:6819/1764493594] boot 2024-02-13T14:12:52.309 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:52 smithi087 ceph-mon[27372]: osdmap e18: 3 total, 3 up, 3 in 2024-02-13T14:12:52.309 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:52 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-13T14:12:52.309 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:52 smithi087 ceph-mon[39120]: purged_snaps scrub starts 2024-02-13T14:12:52.309 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:52 smithi087 ceph-mon[39120]: purged_snaps scrub ok 2024-02-13T14:12:52.309 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:52 smithi087 ceph-mon[39120]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-13T14:12:52.309 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:52 smithi087 ceph-mon[39120]: osd.2 [v2:172.21.15.87:6818/1764493594,v1:172.21.15.87:6819/1764493594] boot 2024-02-13T14:12:52.309 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:52 smithi087 ceph-mon[39120]: osdmap e18: 3 total, 3 up, 3 in 2024-02-13T14:12:52.310 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:52 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-13T14:12:52.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:52 smithi148 ceph-mon[31198]: purged_snaps scrub starts 2024-02-13T14:12:52.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:52 smithi148 ceph-mon[31198]: purged_snaps scrub ok 2024-02-13T14:12:52.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:52 smithi148 ceph-mon[31198]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-13T14:12:52.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:52 smithi148 ceph-mon[31198]: osd.2 [v2:172.21.15.87:6818/1764493594,v1:172.21.15.87:6819/1764493594] boot 2024-02-13T14:12:52.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:52 smithi148 ceph-mon[31198]: osdmap e18: 3 total, 3 up, 3 in 2024-02-13T14:12:52.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:52 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-13T14:12:53.165 INFO:teuthology.orchestra.run.smithi087.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-02-13T14:12:53.165 INFO:teuthology.orchestra.run.smithi087.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-02-13T14:12:53.165 INFO:teuthology.orchestra.run.smithi087.stderr: stderr: 10+0 records in 2024-02-13T14:12:53.165 INFO:teuthology.orchestra.run.smithi087.stderr:10+0 records out 2024-02-13T14:12:53.165 INFO:teuthology.orchestra.run.smithi087.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0147522 s, 711 MB/s 2024-02-13T14:12:53.165 INFO:teuthology.orchestra.run.smithi087.stderr:--> Zapping successful for: 2024-02-13T14:12:53.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:53 smithi087 ceph-mon[27372]: osdmap e19: 3 total, 3 up, 3 in 2024-02-13T14:12:53.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:53 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-02-13T14:12:53.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:53 smithi087 ceph-mon[39120]: osdmap e19: 3 total, 3 up, 3 in 2024-02-13T14:12:53.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:53 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-02-13T14:12:53.618 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:53 smithi148 ceph-mon[31198]: osdmap e19: 3 total, 3 up, 3 in 2024-02-13T14:12:53.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:53 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-02-13T14:12:53.938 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5c04b074-ca79-11ee-95b9-87774f69a715 -- ceph orch daemon add osd smithi087:vg_nvme/lv_1 2024-02-13T14:12:54.173 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/mon.a/config 2024-02-13T14:12:54.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:54 smithi087 ceph-mon[27372]: pgmap v81: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-02-13T14:12:54.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:54 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-02-13T14:12:54.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:54 smithi087 ceph-mon[27372]: osdmap e20: 3 total, 3 up, 3 in 2024-02-13T14:12:54.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:54 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-02-13T14:12:54.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:54 smithi087 ceph-mon[27372]: Detected new or changed devices on smithi087 2024-02-13T14:12:54.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:54 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:12:54.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:54 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:12:54.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:54 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-13T14:12:54.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:54 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-13T14:12:54.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:54 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-13T14:12:54.493 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:54 smithi087 ceph-mon[27372]: Adjusting osd_memory_target on smithi087 to 5354M 2024-02-13T14:12:54.493 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:54 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:12:54.493 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:54 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:12:54.493 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:54 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:12:54.493 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:54 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:12:54.493 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:54 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:12:54.493 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:54 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:12:54.493 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:54 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:12:54.493 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:54 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:12:54.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:54 smithi087 ceph-mon[39120]: pgmap v81: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-02-13T14:12:54.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:54 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-02-13T14:12:54.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:54 smithi087 ceph-mon[39120]: osdmap e20: 3 total, 3 up, 3 in 2024-02-13T14:12:54.494 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:54 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-02-13T14:12:54.494 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:54 smithi087 ceph-mon[39120]: Detected new or changed devices on smithi087 2024-02-13T14:12:54.494 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:54 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:12:54.494 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:54 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:12:54.494 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:54 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-13T14:12:54.494 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:54 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-13T14:12:54.494 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:54 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-13T14:12:54.494 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:54 smithi087 ceph-mon[39120]: Adjusting osd_memory_target on smithi087 to 5354M 2024-02-13T14:12:54.494 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:54 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:12:54.494 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:54 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:12:54.494 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:54 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:12:54.494 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:54 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:12:54.494 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:54 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:12:54.494 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:54 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:12:54.494 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:54 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:12:54.494 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:54 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:12:54.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:54 smithi148 ceph-mon[31198]: pgmap v81: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-02-13T14:12:54.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:54 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-02-13T14:12:54.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:54 smithi148 ceph-mon[31198]: osdmap e20: 3 total, 3 up, 3 in 2024-02-13T14:12:54.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:54 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-02-13T14:12:54.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:54 smithi148 ceph-mon[31198]: Detected new or changed devices on smithi087 2024-02-13T14:12:54.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:54 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:12:54.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:54 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:12:54.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:54 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-13T14:12:54.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:54 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-13T14:12:54.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:54 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-13T14:12:54.620 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:54 smithi148 ceph-mon[31198]: Adjusting osd_memory_target on smithi087 to 5354M 2024-02-13T14:12:54.620 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:54 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:12:54.620 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:54 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:12:54.620 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:54 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:12:54.620 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:54 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:12:54.620 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:54 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:12:54.620 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:54 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:12:54.620 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:54 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:12:54.620 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:54 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:12:55.328 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:55 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-02-13T14:12:55.328 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:55 smithi087 ceph-mon[39120]: osdmap e21: 3 total, 3 up, 3 in 2024-02-13T14:12:55.328 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:55 smithi087 ceph-mon[39120]: pgmap v84: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-02-13T14:12:55.329 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:55 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-02-13T14:12:55.329 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:55 smithi087 ceph-mon[27372]: osdmap e21: 3 total, 3 up, 3 in 2024-02-13T14:12:55.329 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:55 smithi087 ceph-mon[27372]: pgmap v84: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-02-13T14:12:55.329 INFO:journalctl@ceph.osd.0.smithi087.stdout:Feb 13 14:12:55 smithi087 sudo[59615]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-13T14:12:55.329 INFO:journalctl@ceph.osd.0.smithi087.stdout:Feb 13 14:12:55 smithi087 sudo[59615]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-13T14:12:55.329 INFO:journalctl@ceph.osd.0.smithi087.stdout:Feb 13 14:12:55 smithi087 sudo[59615]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-13T14:12:55.329 INFO:journalctl@ceph.osd.0.smithi087.stdout:Feb 13 14:12:55 smithi087 sudo[59615]: pam_unix(sudo:session): session closed for user root 2024-02-13T14:12:55.603 INFO:journalctl@ceph.osd.0.smithi087.stdout:Feb 13 14:12:55 smithi087 sudo[59618]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-13T14:12:55.603 INFO:journalctl@ceph.osd.0.smithi087.stdout:Feb 13 14:12:55 smithi087 sudo[59618]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-13T14:12:55.603 INFO:journalctl@ceph.osd.0.smithi087.stdout:Feb 13 14:12:55 smithi087 sudo[59618]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-13T14:12:55.603 INFO:journalctl@ceph.osd.0.smithi087.stdout:Feb 13 14:12:55 smithi087 sudo[59618]: pam_unix(sudo:session): session closed for user root 2024-02-13T14:12:55.618 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:55 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-02-13T14:12:55.618 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:55 smithi148 ceph-mon[31198]: osdmap e21: 3 total, 3 up, 3 in 2024-02-13T14:12:55.618 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:55 smithi148 ceph-mon[31198]: pgmap v84: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-02-13T14:12:55.867 INFO:journalctl@ceph.osd.1.smithi087.stdout:Feb 13 14:12:55 smithi087 sudo[59641]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-13T14:12:55.868 INFO:journalctl@ceph.osd.1.smithi087.stdout:Feb 13 14:12:55 smithi087 sudo[59641]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-13T14:12:55.868 INFO:journalctl@ceph.osd.1.smithi087.stdout:Feb 13 14:12:55 smithi087 sudo[59641]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-13T14:12:55.868 INFO:journalctl@ceph.osd.1.smithi087.stdout:Feb 13 14:12:55 smithi087 sudo[59641]: pam_unix(sudo:session): session closed for user root 2024-02-13T14:12:55.868 INFO:journalctl@ceph.osd.1.smithi087.stdout:Feb 13 14:12:55 smithi087 sudo[59671]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-13T14:12:56.120 INFO:journalctl@ceph.osd.1.smithi087.stdout:Feb 13 14:12:55 smithi087 sudo[59671]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-13T14:12:56.121 INFO:journalctl@ceph.osd.1.smithi087.stdout:Feb 13 14:12:55 smithi087 sudo[59671]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-13T14:12:56.121 INFO:journalctl@ceph.osd.1.smithi087.stdout:Feb 13 14:12:55 smithi087 sudo[59671]: pam_unix(sudo:session): session closed for user root 2024-02-13T14:12:56.388 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:56 smithi087 ceph-mon[39120]: osdmap e22: 3 total, 3 up, 3 in 2024-02-13T14:12:56.388 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:56 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:12:56.388 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:56 smithi087 ceph-mon[39120]: from='client.24206 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi087:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T14:12:56.388 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:56 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-13T14:12:56.388 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:56 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-13T14:12:56.388 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:56 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:12:56.389 INFO:journalctl@ceph.osd.2.smithi087.stdout:Feb 13 14:12:56 smithi087 sudo[59694]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-13T14:12:56.389 INFO:journalctl@ceph.osd.2.smithi087.stdout:Feb 13 14:12:56 smithi087 sudo[59694]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-13T14:12:56.389 INFO:journalctl@ceph.osd.2.smithi087.stdout:Feb 13 14:12:56 smithi087 sudo[59694]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-13T14:12:56.389 INFO:journalctl@ceph.osd.2.smithi087.stdout:Feb 13 14:12:56 smithi087 sudo[59694]: pam_unix(sudo:session): session closed for user root 2024-02-13T14:12:56.389 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:56 smithi087 ceph-mon[27372]: osdmap e22: 3 total, 3 up, 3 in 2024-02-13T14:12:56.389 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:56 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:12:56.389 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:56 smithi087 ceph-mon[27372]: from='client.24206 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi087:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T14:12:56.389 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:56 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-13T14:12:56.389 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:56 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-13T14:12:56.389 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:56 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:12:56.618 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:56 smithi148 ceph-mon[31198]: osdmap e22: 3 total, 3 up, 3 in 2024-02-13T14:12:56.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:56 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:12:56.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:56 smithi148 ceph-mon[31198]: from='client.24206 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi087:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T14:12:56.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:56 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-13T14:12:56.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:56 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-13T14:12:56.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:56 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:12:56.646 INFO:journalctl@ceph.osd.2.smithi087.stdout:Feb 13 14:12:56 smithi087 sudo[59764]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-13T14:12:56.646 INFO:journalctl@ceph.osd.2.smithi087.stdout:Feb 13 14:12:56 smithi087 sudo[59764]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-13T14:12:56.646 INFO:journalctl@ceph.osd.2.smithi087.stdout:Feb 13 14:12:56 smithi087 sudo[59764]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-13T14:12:56.646 INFO:journalctl@ceph.osd.2.smithi087.stdout:Feb 13 14:12:56 smithi087 sudo[59764]: pam_unix(sudo:session): session closed for user root 2024-02-13T14:12:56.905 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:56 smithi087 sudo[59814]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-13T14:12:56.905 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:56 smithi087 sudo[59814]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-13T14:12:56.905 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:56 smithi087 sudo[59814]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-13T14:12:56.905 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:56 smithi087 sudo[59814]: pam_unix(sudo:session): session closed for user root 2024-02-13T14:12:57.241 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:56 smithi087 sudo[59844]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-13T14:12:57.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:56 smithi087 sudo[59844]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-13T14:12:57.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:56 smithi087 sudo[59844]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-13T14:12:57.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:56 smithi087 sudo[59844]: pam_unix(sudo:session): session closed for user root 2024-02-13T14:12:57.405 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:57 smithi148 sudo[34031]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-13T14:12:57.405 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:57 smithi148 sudo[34031]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-13T14:12:57.405 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:57 smithi148 sudo[34031]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-13T14:12:57.405 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:57 smithi148 sudo[34031]: pam_unix(sudo:session): session closed for user root 2024-02-13T14:12:57.671 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:57 smithi087 ceph-mon[27372]: mgrmap e15: y(active, since 2m), standbys: x 2024-02-13T14:12:57.672 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:57 smithi087 ceph-mon[27372]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-13T14:12:57.672 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:57 smithi087 ceph-mon[27372]: pgmap v86: 1 pgs: 1 active+clean; 449 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-02-13T14:12:57.672 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:57 smithi087 ceph-mon[27372]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-13T14:12:57.672 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:57 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-13T14:12:57.672 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:57 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-13T14:12:57.672 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:57 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-13T14:12:57.672 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:57 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-13T14:12:57.672 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:57 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-13T14:12:57.672 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:57 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-13T14:12:57.673 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:57 smithi087 ceph-mon[39120]: mgrmap e15: y(active, since 2m), standbys: x 2024-02-13T14:12:57.673 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:57 smithi087 ceph-mon[39120]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-13T14:12:57.673 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:57 smithi087 ceph-mon[39120]: pgmap v86: 1 pgs: 1 active+clean; 449 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-02-13T14:12:57.673 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:57 smithi087 ceph-mon[39120]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-13T14:12:57.673 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:57 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-13T14:12:57.673 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:57 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-13T14:12:57.673 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:57 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-13T14:12:57.673 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:57 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-13T14:12:57.673 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:57 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-13T14:12:57.673 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:57 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-13T14:12:57.868 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:57 smithi148 sudo[34034]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-13T14:12:57.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:57 smithi148 sudo[34034]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-13T14:12:57.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:57 smithi148 sudo[34034]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-13T14:12:57.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:57 smithi148 sudo[34034]: pam_unix(sudo:session): session closed for user root 2024-02-13T14:12:57.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:57 smithi148 ceph-mon[31198]: mgrmap e15: y(active, since 2m), standbys: x 2024-02-13T14:12:57.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:57 smithi148 ceph-mon[31198]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-13T14:12:57.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:57 smithi148 ceph-mon[31198]: pgmap v86: 1 pgs: 1 active+clean; 449 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-02-13T14:12:57.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:57 smithi148 ceph-mon[31198]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-13T14:12:57.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:57 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-13T14:12:57.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:57 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-13T14:12:57.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:57 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-13T14:12:57.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:57 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-13T14:12:57.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:57 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-13T14:12:57.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:57 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-13T14:12:57.934 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:57 smithi087 sudo[59873]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-13T14:12:57.934 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:57 smithi087 sudo[59873]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-13T14:12:57.934 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:57 smithi087 sudo[59873]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-13T14:12:57.934 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:57 smithi087 sudo[59873]: pam_unix(sudo:session): session closed for user root 2024-02-13T14:12:58.189 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:57 smithi087 sudo[59897]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-13T14:12:58.190 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:57 smithi087 sudo[59897]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-13T14:12:58.190 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:57 smithi087 sudo[59897]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-13T14:12:58.190 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:57 smithi087 sudo[59897]: pam_unix(sudo:session): session closed for user root 2024-02-13T14:12:58.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:58 smithi087 ceph-mon[27372]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-13T14:12:58.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:58 smithi087 ceph-mon[27372]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-13T14:12:58.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:58 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:12:58.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:58 smithi087 ceph-mon[27372]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-13T14:12:58.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:58 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-13T14:12:58.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:58 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-13T14:12:58.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:58 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-13T14:12:58.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:58 smithi087 ceph-mon[27372]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-13T14:12:58.743 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:58 smithi087 ceph-mon[39120]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-13T14:12:58.743 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:58 smithi087 ceph-mon[39120]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-13T14:12:58.743 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:58 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:12:58.743 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:58 smithi087 ceph-mon[39120]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-13T14:12:58.743 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:58 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-13T14:12:58.743 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:58 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-13T14:12:58.743 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:58 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-13T14:12:58.743 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:58 smithi087 ceph-mon[39120]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-13T14:12:58.868 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:58 smithi148 ceph-mon[31198]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-13T14:12:58.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:58 smithi148 ceph-mon[31198]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-13T14:12:58.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:58 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:12:58.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:58 smithi148 ceph-mon[31198]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-13T14:12:58.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:58 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-13T14:12:58.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:58 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-13T14:12:58.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:58 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-13T14:12:58.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:58 smithi148 ceph-mon[31198]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-13T14:12:59.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:59 smithi087 ceph-mon[27372]: pgmap v87: 1 pgs: 1 active+clean; 449 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-02-13T14:12:59.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:59 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/3629031891' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dfa2f275-b84b-4227-8135-e009b2cb461e"}]: dispatch 2024-02-13T14:12:59.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:59 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/3629031891' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dfa2f275-b84b-4227-8135-e009b2cb461e"}]': finished 2024-02-13T14:12:59.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:59 smithi087 ceph-mon[27372]: osdmap e23: 4 total, 3 up, 4 in 2024-02-13T14:12:59.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:12:59 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-13T14:12:59.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:59 smithi087 ceph-mon[39120]: pgmap v87: 1 pgs: 1 active+clean; 449 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-02-13T14:12:59.743 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:59 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/3629031891' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dfa2f275-b84b-4227-8135-e009b2cb461e"}]: dispatch 2024-02-13T14:12:59.743 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:59 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/3629031891' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dfa2f275-b84b-4227-8135-e009b2cb461e"}]': finished 2024-02-13T14:12:59.743 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:59 smithi087 ceph-mon[39120]: osdmap e23: 4 total, 3 up, 4 in 2024-02-13T14:12:59.743 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:12:59 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-13T14:12:59.868 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:59 smithi148 ceph-mon[31198]: pgmap v87: 1 pgs: 1 active+clean; 449 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-02-13T14:12:59.868 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:59 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/3629031891' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dfa2f275-b84b-4227-8135-e009b2cb461e"}]: dispatch 2024-02-13T14:12:59.868 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:59 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/3629031891' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dfa2f275-b84b-4227-8135-e009b2cb461e"}]': finished 2024-02-13T14:12:59.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:59 smithi148 ceph-mon[31198]: osdmap e23: 4 total, 3 up, 4 in 2024-02-13T14:12:59.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:12:59 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-13T14:13:00.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:00 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/3315625680' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-13T14:13:00.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:00 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/3315625680' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-13T14:13:00.868 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:00 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/3315625680' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-13T14:13:01.714 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:01 smithi087 ceph-mon[39120]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 4.8 KiB/s rd, 232 KiB/s wr, 15 op/s 2024-02-13T14:13:01.715 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:01 smithi087 ceph-mon[27372]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 4.8 KiB/s rd, 232 KiB/s wr, 15 op/s 2024-02-13T14:13:01.868 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:01 smithi148 ceph-mon[31198]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 4.8 KiB/s rd, 232 KiB/s wr, 15 op/s 2024-02-13T14:13:04.241 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:03 smithi087 ceph-mon[39120]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 3.5 KiB/s rd, 171 KiB/s wr, 11 op/s 2024-02-13T14:13:04.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:03 smithi087 ceph-mon[27372]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 3.5 KiB/s rd, 171 KiB/s wr, 11 op/s 2024-02-13T14:13:04.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:03 smithi148 ceph-mon[31198]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 3.5 KiB/s rd, 171 KiB/s wr, 11 op/s 2024-02-13T14:13:06.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:05 smithi087 ceph-mon[27372]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 2.8 KiB/s rd, 136 KiB/s wr, 9 op/s 2024-02-13T14:13:06.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:05 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-02-13T14:13:06.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:05 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:13:06.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:05 smithi087 ceph-mon[39120]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 2.8 KiB/s rd, 136 KiB/s wr, 9 op/s 2024-02-13T14:13:06.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:05 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-02-13T14:13:06.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:05 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:13:06.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:05 smithi148 ceph-mon[31198]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 2.8 KiB/s rd, 136 KiB/s wr, 9 op/s 2024-02-13T14:13:06.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:05 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-02-13T14:13:06.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:05 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:13:07.241 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:06 smithi087 ceph-mon[27372]: Deploying daemon osd.3 on smithi087 2024-02-13T14:13:07.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:06 smithi087 ceph-mon[39120]: Deploying daemon osd.3 on smithi087 2024-02-13T14:13:07.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:06 smithi148 ceph-mon[31198]: Deploying daemon osd.3 on smithi087 2024-02-13T14:13:08.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:07 smithi087 ceph-mon[39120]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 2.7 KiB/s rd, 131 KiB/s wr, 8 op/s 2024-02-13T14:13:08.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:07 smithi087 ceph-mon[27372]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 2.7 KiB/s rd, 131 KiB/s wr, 8 op/s 2024-02-13T14:13:08.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:07 smithi148 ceph-mon[31198]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 2.7 KiB/s rd, 131 KiB/s wr, 8 op/s 2024-02-13T14:13:10.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:09 smithi087 ceph-mon[27372]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 2.7 KiB/s rd, 131 KiB/s wr, 8 op/s 2024-02-13T14:13:10.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:09 smithi087 ceph-mon[39120]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 2.7 KiB/s rd, 131 KiB/s wr, 8 op/s 2024-02-13T14:13:10.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:09 smithi148 ceph-mon[31198]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 2.7 KiB/s rd, 131 KiB/s wr, 8 op/s 2024-02-13T14:13:12.005 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:11 smithi087 ceph-mon[39120]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 2.3 KiB/s rd, 110 KiB/s wr, 7 op/s 2024-02-13T14:13:12.005 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:11 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:13:12.005 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:11 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:13:12.005 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:11 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:13:12.006 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:11 smithi087 ceph-mon[27372]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 2.3 KiB/s rd, 110 KiB/s wr, 7 op/s 2024-02-13T14:13:12.006 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:11 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:13:12.006 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:11 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:13:12.006 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:11 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:13:12.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:11 smithi148 ceph-mon[31198]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 2.3 KiB/s rd, 110 KiB/s wr, 7 op/s 2024-02-13T14:13:12.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:11 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:13:12.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:11 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:13:12.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:11 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:13:14.129 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:13 smithi087 ceph-mon[27372]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-02-13T14:13:14.129 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:13 smithi087 ceph-mon[39120]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-02-13T14:13:14.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:13 smithi148 ceph-mon[31198]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-02-13T14:13:15.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:14 smithi087 ceph-mon[27372]: from='osd.3 [v2:172.21.15.87:6826/1162405572,v1:172.21.15.87:6827/1162405572]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-13T14:13:15.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:14 smithi087 ceph-mon[27372]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-13T14:13:15.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:14 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:13:15.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:14 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:13:15.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:14 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:13:15.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:14 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:13:15.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:14 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:13:15.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:14 smithi087 ceph-mon[39120]: from='osd.3 [v2:172.21.15.87:6826/1162405572,v1:172.21.15.87:6827/1162405572]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-13T14:13:15.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:14 smithi087 ceph-mon[39120]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-13T14:13:15.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:14 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:13:15.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:14 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:13:15.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:14 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:13:15.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:14 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:13:15.243 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:14 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:13:15.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:14 smithi148 ceph-mon[31198]: from='osd.3 [v2:172.21.15.87:6826/1162405572,v1:172.21.15.87:6827/1162405572]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-13T14:13:15.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:14 smithi148 ceph-mon[31198]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-13T14:13:15.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:14 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:13:15.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:14 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:13:15.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:14 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:13:15.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:14 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:13:15.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:14 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:13:15.431 INFO:teuthology.orchestra.run.smithi087.stdout:Created osd(s) 3 on host 'smithi087' 2024-02-13T14:13:15.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:15 smithi087 ceph-mon[27372]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-02-13T14:13:15.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:15 smithi087 ceph-mon[27372]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-02-13T14:13:15.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:15 smithi087 ceph-mon[27372]: osdmap e24: 4 total, 3 up, 4 in 2024-02-13T14:13:15.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:15 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-13T14:13:15.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:15 smithi087 ceph-mon[27372]: from='osd.3 [v2:172.21.15.87:6826/1162405572,v1:172.21.15.87:6827/1162405572]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]: dispatch 2024-02-13T14:13:15.993 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:15 smithi087 ceph-mon[27372]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]: dispatch 2024-02-13T14:13:15.993 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:15 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:13:15.993 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:15 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:13:15.993 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:15 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:13:15.993 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:15 smithi087 ceph-mon[39120]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-02-13T14:13:15.994 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:15 smithi087 ceph-mon[39120]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-02-13T14:13:15.994 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:15 smithi087 ceph-mon[39120]: osdmap e24: 4 total, 3 up, 4 in 2024-02-13T14:13:15.994 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:15 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-13T14:13:15.994 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:15 smithi087 ceph-mon[39120]: from='osd.3 [v2:172.21.15.87:6826/1162405572,v1:172.21.15.87:6827/1162405572]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]: dispatch 2024-02-13T14:13:15.994 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:15 smithi087 ceph-mon[39120]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]: dispatch 2024-02-13T14:13:15.994 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:15 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:13:15.994 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:15 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:13:15.995 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:15 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:13:16.058 DEBUG:teuthology.orchestra.run.smithi087:osd.3> sudo journalctl -f -n 0 -u ceph-5c04b074-ca79-11ee-95b9-87774f69a715@osd.3.service 2024-02-13T14:13:16.061 INFO:tasks.cephadm:Deploying osd.4 on smithi148 with /dev/vg_nvme/lv_4... 2024-02-13T14:13:16.061 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5c04b074-ca79-11ee-95b9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-02-13T14:13:16.308 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/mon.b/config 2024-02-13T14:13:16.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:15 smithi148 ceph-mon[31198]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-02-13T14:13:16.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:15 smithi148 ceph-mon[31198]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-02-13T14:13:16.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:15 smithi148 ceph-mon[31198]: osdmap e24: 4 total, 3 up, 4 in 2024-02-13T14:13:16.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:15 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-13T14:13:16.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:15 smithi148 ceph-mon[31198]: from='osd.3 [v2:172.21.15.87:6826/1162405572,v1:172.21.15.87:6827/1162405572]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]: dispatch 2024-02-13T14:13:16.370 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:15 smithi148 ceph-mon[31198]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]: dispatch 2024-02-13T14:13:16.370 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:15 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:13:16.370 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:15 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:13:16.370 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:15 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:13:16.626 INFO:journalctl@ceph.osd.3.smithi087.stdout:Feb 13 14:13:16 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-3[61781]: 2024-02-13T14:13:16.417+0000 7fe4682a2700 -1 osd.3 0 waiting for initial osdmap 2024-02-13T14:13:16.626 INFO:journalctl@ceph.osd.3.smithi087.stdout:Feb 13 14:13:16 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-3[61781]: 2024-02-13T14:13:16.420+0000 7fe46008d700 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-13T14:13:17.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:16 smithi087 ceph-mon[27372]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]': finished 2024-02-13T14:13:17.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:16 smithi087 ceph-mon[27372]: osdmap e25: 4 total, 3 up, 4 in 2024-02-13T14:13:17.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:16 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-13T14:13:17.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:16 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-13T14:13:17.243 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:16 smithi087 ceph-mon[27372]: from='osd.3 ' entity='osd.3' 2024-02-13T14:13:17.243 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:16 smithi087 ceph-mon[39120]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]': finished 2024-02-13T14:13:17.243 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:16 smithi087 ceph-mon[39120]: osdmap e25: 4 total, 3 up, 4 in 2024-02-13T14:13:17.243 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:16 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-13T14:13:17.243 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:16 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-13T14:13:17.244 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:16 smithi087 ceph-mon[39120]: from='osd.3 ' entity='osd.3' 2024-02-13T14:13:17.312 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:16 smithi148 ceph-mon[31198]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]': finished 2024-02-13T14:13:17.312 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:16 smithi148 ceph-mon[31198]: osdmap e25: 4 total, 3 up, 4 in 2024-02-13T14:13:17.312 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:16 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-13T14:13:17.312 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:16 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-13T14:13:17.312 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:16 smithi148 ceph-mon[31198]: from='osd.3 ' entity='osd.3' 2024-02-13T14:13:18.163 INFO:teuthology.orchestra.run.smithi148.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-02-13T14:13:18.163 INFO:teuthology.orchestra.run.smithi148.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-02-13T14:13:18.163 INFO:teuthology.orchestra.run.smithi148.stderr: stderr: 10+0 records in 2024-02-13T14:13:18.163 INFO:teuthology.orchestra.run.smithi148.stderr:10+0 records out 2024-02-13T14:13:18.164 INFO:teuthology.orchestra.run.smithi148.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0128266 s, 817 MB/s 2024-02-13T14:13:18.164 INFO:teuthology.orchestra.run.smithi148.stderr:--> Zapping successful for: 2024-02-13T14:13:18.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:17 smithi087 ceph-mon[27372]: purged_snaps scrub starts 2024-02-13T14:13:18.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:17 smithi087 ceph-mon[27372]: purged_snaps scrub ok 2024-02-13T14:13:18.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:17 smithi087 ceph-mon[27372]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-02-13T14:13:18.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:17 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-13T14:13:18.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:17 smithi087 ceph-mon[27372]: osd.3 [v2:172.21.15.87:6826/1162405572,v1:172.21.15.87:6827/1162405572] boot 2024-02-13T14:13:18.243 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:17 smithi087 ceph-mon[27372]: osdmap e26: 4 total, 4 up, 4 in 2024-02-13T14:13:18.243 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:17 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-13T14:13:18.243 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:17 smithi087 ceph-mon[39120]: purged_snaps scrub starts 2024-02-13T14:13:18.243 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:17 smithi087 ceph-mon[39120]: purged_snaps scrub ok 2024-02-13T14:13:18.243 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:17 smithi087 ceph-mon[39120]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-02-13T14:13:18.243 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:17 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-13T14:13:18.243 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:17 smithi087 ceph-mon[39120]: osd.3 [v2:172.21.15.87:6826/1162405572,v1:172.21.15.87:6827/1162405572] boot 2024-02-13T14:13:18.243 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:17 smithi087 ceph-mon[39120]: osdmap e26: 4 total, 4 up, 4 in 2024-02-13T14:13:18.243 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:17 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-13T14:13:18.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:17 smithi148 ceph-mon[31198]: purged_snaps scrub starts 2024-02-13T14:13:18.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:17 smithi148 ceph-mon[31198]: purged_snaps scrub ok 2024-02-13T14:13:18.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:17 smithi148 ceph-mon[31198]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-02-13T14:13:18.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:17 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-13T14:13:18.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:17 smithi148 ceph-mon[31198]: osd.3 [v2:172.21.15.87:6826/1162405572,v1:172.21.15.87:6827/1162405572] boot 2024-02-13T14:13:18.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:17 smithi148 ceph-mon[31198]: osdmap e26: 4 total, 4 up, 4 in 2024-02-13T14:13:18.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:17 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-13T14:13:18.724 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5c04b074-ca79-11ee-95b9-87774f69a715 -- ceph orch daemon add osd smithi148:vg_nvme/lv_4 2024-02-13T14:13:18.959 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/mon.b/config 2024-02-13T14:13:19.618 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:19 smithi148 ceph-mon[31198]: osdmap e27: 4 total, 4 up, 4 in 2024-02-13T14:13:19.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:19 smithi148 ceph-mon[31198]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-13T14:13:19.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:19 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:13:19.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:19 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:13:19.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:19 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-13T14:13:19.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:19 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-13T14:13:19.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:19 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-13T14:13:19.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:19 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-13T14:13:19.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:19 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:13:19.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:19 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:13:19.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:19 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:13:19.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:19 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:13:19.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:19 smithi087 ceph-mon[27372]: osdmap e27: 4 total, 4 up, 4 in 2024-02-13T14:13:19.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:19 smithi087 ceph-mon[27372]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-13T14:13:19.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:19 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:13:19.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:19 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:13:19.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:19 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-13T14:13:19.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:19 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-13T14:13:19.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:19 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-13T14:13:19.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:19 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-13T14:13:19.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:19 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:13:19.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:19 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:13:19.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:19 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:13:19.743 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:19 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:13:19.743 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:19 smithi087 ceph-mon[39120]: osdmap e27: 4 total, 4 up, 4 in 2024-02-13T14:13:19.743 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:19 smithi087 ceph-mon[39120]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-13T14:13:19.743 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:19 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:13:19.743 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:19 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:13:19.743 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:19 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-13T14:13:19.743 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:19 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-13T14:13:19.743 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:19 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-13T14:13:19.743 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:19 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-13T14:13:19.743 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:19 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:13:19.743 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:19 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:13:19.743 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:19 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:13:19.744 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:19 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:13:20.572 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:20 smithi087 ceph-mon[39120]: Detected new or changed devices on smithi087 2024-02-13T14:13:20.573 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:20 smithi087 ceph-mon[39120]: Adjusting osd_memory_target on smithi087 to 4016M 2024-02-13T14:13:20.573 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:20 smithi087 ceph-mon[27372]: Detected new or changed devices on smithi087 2024-02-13T14:13:20.573 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:20 smithi087 ceph-mon[27372]: Adjusting osd_memory_target on smithi087 to 4016M 2024-02-13T14:13:20.868 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:20 smithi148 ceph-mon[31198]: Detected new or changed devices on smithi087 2024-02-13T14:13:20.868 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:20 smithi148 ceph-mon[31198]: Adjusting osd_memory_target on smithi087 to 4016M 2024-02-13T14:13:21.503 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:21 smithi148 ceph-mon[31198]: from='client.24193 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T14:13:21.503 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:21 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-13T14:13:21.504 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:21 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-13T14:13:21.504 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:21 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:13:21.504 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:21 smithi148 ceph-mon[31198]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-13T14:13:21.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:21 smithi087 ceph-mon[39120]: from='client.24193 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T14:13:21.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:21 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-13T14:13:21.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:21 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-13T14:13:21.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:21 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:13:21.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:21 smithi087 ceph-mon[39120]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-13T14:13:21.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:21 smithi087 ceph-mon[27372]: from='client.24193 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T14:13:21.743 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:21 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-13T14:13:21.743 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:21 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-13T14:13:21.743 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:21 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:13:21.743 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:21 smithi087 ceph-mon[27372]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-13T14:13:24.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:23 smithi087 ceph-mon[27372]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-13T14:13:24.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:23 smithi087 ceph-mon[27372]: from='client.? 172.21.15.148:0/1533279967' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9b07d44a-ddd1-4b31-8492-3e1459c0ad1c"}]: dispatch 2024-02-13T14:13:24.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:23 smithi087 ceph-mon[27372]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9b07d44a-ddd1-4b31-8492-3e1459c0ad1c"}]: dispatch 2024-02-13T14:13:24.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:23 smithi087 ceph-mon[27372]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9b07d44a-ddd1-4b31-8492-3e1459c0ad1c"}]': finished 2024-02-13T14:13:24.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:23 smithi087 ceph-mon[27372]: osdmap e28: 5 total, 4 up, 5 in 2024-02-13T14:13:24.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:23 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-13T14:13:24.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:23 smithi087 ceph-mon[27372]: from='client.? 172.21.15.148:0/3872576451' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-13T14:13:24.243 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:23 smithi087 ceph-mon[39120]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-13T14:13:24.243 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:23 smithi087 ceph-mon[39120]: from='client.? 172.21.15.148:0/1533279967' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9b07d44a-ddd1-4b31-8492-3e1459c0ad1c"}]: dispatch 2024-02-13T14:13:24.243 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:23 smithi087 ceph-mon[39120]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9b07d44a-ddd1-4b31-8492-3e1459c0ad1c"}]: dispatch 2024-02-13T14:13:24.243 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:23 smithi087 ceph-mon[39120]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9b07d44a-ddd1-4b31-8492-3e1459c0ad1c"}]': finished 2024-02-13T14:13:24.243 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:23 smithi087 ceph-mon[39120]: osdmap e28: 5 total, 4 up, 5 in 2024-02-13T14:13:24.243 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:23 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-13T14:13:24.243 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:23 smithi087 ceph-mon[39120]: from='client.? 172.21.15.148:0/3872576451' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-13T14:13:24.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:23 smithi148 ceph-mon[31198]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-13T14:13:24.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:23 smithi148 ceph-mon[31198]: from='client.? 172.21.15.148:0/1533279967' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9b07d44a-ddd1-4b31-8492-3e1459c0ad1c"}]: dispatch 2024-02-13T14:13:24.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:23 smithi148 ceph-mon[31198]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9b07d44a-ddd1-4b31-8492-3e1459c0ad1c"}]: dispatch 2024-02-13T14:13:24.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:23 smithi148 ceph-mon[31198]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9b07d44a-ddd1-4b31-8492-3e1459c0ad1c"}]': finished 2024-02-13T14:13:24.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:23 smithi148 ceph-mon[31198]: osdmap e28: 5 total, 4 up, 5 in 2024-02-13T14:13:24.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:23 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-13T14:13:24.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:23 smithi148 ceph-mon[31198]: from='client.? 172.21.15.148:0/3872576451' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-13T14:13:26.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:25 smithi087 ceph-mon[39120]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-13T14:13:26.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:25 smithi087 ceph-mon[27372]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-13T14:13:26.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:25 smithi148 ceph-mon[31198]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-13T14:13:28.118 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:27 smithi148 ceph-mon[31198]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-13T14:13:28.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:27 smithi087 ceph-mon[27372]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-13T14:13:28.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:27 smithi087 ceph-mon[39120]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-13T14:13:30.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:29 smithi087 ceph-mon[27372]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-13T14:13:30.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:29 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-02-13T14:13:30.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:29 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:13:30.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:29 smithi087 ceph-mon[39120]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-13T14:13:30.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:29 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-02-13T14:13:30.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:29 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:13:30.291 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:29 smithi148 ceph-mon[31198]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-13T14:13:30.292 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:29 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-02-13T14:13:30.292 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:29 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:13:31.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:30 smithi148 ceph-mon[31198]: Deploying daemon osd.4 on smithi148 2024-02-13T14:13:31.241 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:30 smithi087 ceph-mon[27372]: Deploying daemon osd.4 on smithi148 2024-02-13T14:13:31.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:30 smithi087 ceph-mon[39120]: Deploying daemon osd.4 on smithi148 2024-02-13T14:13:32.244 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:31 smithi087 ceph-mon[39120]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-13T14:13:32.244 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:31 smithi087 ceph-mon[27372]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-13T14:13:32.257 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:31 smithi148 ceph-mon[31198]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-13T14:13:34.215 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:33 smithi148 ceph-mon[31198]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-13T14:13:34.241 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:33 smithi087 ceph-mon[27372]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-13T14:13:34.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:33 smithi087 ceph-mon[39120]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-13T14:13:36.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:35 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:13:36.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:35 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:13:36.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:35 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:13:36.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:35 smithi148 ceph-mon[31198]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-13T14:13:36.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:35 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:13:36.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:35 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:13:36.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:35 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:13:36.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:35 smithi087 ceph-mon[27372]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-13T14:13:36.243 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:35 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:13:36.243 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:35 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:13:36.243 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:35 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:13:36.243 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:35 smithi087 ceph-mon[39120]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-13T14:13:38.341 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:38 smithi148 ceph-mon[31198]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-13T14:13:38.341 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:38 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:13:38.341 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:38 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:13:38.342 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:38 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:13:38.342 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:38 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:13:38.342 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:38 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:13:38.342 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:38 smithi148 ceph-mon[31198]: from='osd.4 [v2:172.21.15.148:6800/1403587816,v1:172.21.15.148:6801/1403587816]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-13T14:13:38.342 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:38 smithi148 ceph-mon[31198]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-13T14:13:38.409 INFO:teuthology.orchestra.run.smithi148.stdout:Created osd(s) 4 on host 'smithi148' 2024-02-13T14:13:38.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:38 smithi087 ceph-mon[27372]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-13T14:13:38.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:38 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:13:38.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:38 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:13:38.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:38 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:13:38.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:38 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:13:38.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:38 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:13:38.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:38 smithi087 ceph-mon[27372]: from='osd.4 [v2:172.21.15.148:6800/1403587816,v1:172.21.15.148:6801/1403587816]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-13T14:13:38.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:38 smithi087 ceph-mon[27372]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-13T14:13:38.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:38 smithi087 ceph-mon[39120]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-13T14:13:38.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:38 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:13:38.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:38 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:13:38.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:38 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:13:38.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:38 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:13:38.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:38 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:13:38.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:38 smithi087 ceph-mon[39120]: from='osd.4 [v2:172.21.15.148:6800/1403587816,v1:172.21.15.148:6801/1403587816]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-13T14:13:38.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:38 smithi087 ceph-mon[39120]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-13T14:13:39.029 DEBUG:teuthology.orchestra.run.smithi148:osd.4> sudo journalctl -f -n 0 -u ceph-5c04b074-ca79-11ee-95b9-87774f69a715@osd.4.service 2024-02-13T14:13:39.031 INFO:tasks.cephadm:Deploying osd.5 on smithi148 with /dev/vg_nvme/lv_3... 2024-02-13T14:13:39.032 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5c04b074-ca79-11ee-95b9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-02-13T14:13:39.308 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:39 smithi148 ceph-mon[31198]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-02-13T14:13:39.308 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:39 smithi148 ceph-mon[31198]: from='osd.4 [v2:172.21.15.148:6800/1403587816,v1:172.21.15.148:6801/1403587816]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-02-13T14:13:39.308 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:39 smithi148 ceph-mon[31198]: osdmap e29: 5 total, 4 up, 5 in 2024-02-13T14:13:39.308 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:39 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-13T14:13:39.308 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:39 smithi148 ceph-mon[31198]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-02-13T14:13:39.308 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:39 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:13:39.308 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:39 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:13:39.309 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:39 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:13:39.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:39 smithi087 ceph-mon[27372]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-02-13T14:13:39.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:39 smithi087 ceph-mon[27372]: from='osd.4 [v2:172.21.15.148:6800/1403587816,v1:172.21.15.148:6801/1403587816]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-02-13T14:13:39.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:39 smithi087 ceph-mon[27372]: osdmap e29: 5 total, 4 up, 5 in 2024-02-13T14:13:39.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:39 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-13T14:13:39.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:39 smithi087 ceph-mon[27372]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-02-13T14:13:39.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:39 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:13:39.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:39 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:13:39.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:39 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:13:39.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:39 smithi087 ceph-mon[39120]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-02-13T14:13:39.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:39 smithi087 ceph-mon[39120]: from='osd.4 [v2:172.21.15.148:6800/1403587816,v1:172.21.15.148:6801/1403587816]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-02-13T14:13:39.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:39 smithi087 ceph-mon[39120]: osdmap e29: 5 total, 4 up, 5 in 2024-02-13T14:13:39.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:39 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-13T14:13:39.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:39 smithi087 ceph-mon[39120]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-02-13T14:13:39.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:39 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:13:39.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:39 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:13:39.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:39 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:13:39.864 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/mon.b/config 2024-02-13T14:13:39.868 INFO:journalctl@ceph.osd.4.smithi148.stdout:Feb 13 14:13:39 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-4[36733]: 2024-02-13T14:13:39.545+0000 7f4aef799700 -1 osd.4 0 waiting for initial osdmap 2024-02-13T14:13:39.868 INFO:journalctl@ceph.osd.4.smithi148.stdout:Feb 13 14:13:39 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-4[36733]: 2024-02-13T14:13:39.548+0000 7f4ae7584700 -1 osd.4 30 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-13T14:13:40.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:40 smithi148 ceph-mon[31198]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-13T14:13:40.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:40 smithi148 ceph-mon[31198]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]': finished 2024-02-13T14:13:40.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:40 smithi148 ceph-mon[31198]: osdmap e30: 5 total, 4 up, 5 in 2024-02-13T14:13:40.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:40 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-13T14:13:40.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:40 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-13T14:13:40.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:40 smithi148 ceph-mon[31198]: from='osd.4 ' entity='osd.4' 2024-02-13T14:13:40.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:40 smithi087 ceph-mon[27372]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-13T14:13:40.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:40 smithi087 ceph-mon[27372]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]': finished 2024-02-13T14:13:40.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:40 smithi087 ceph-mon[27372]: osdmap e30: 5 total, 4 up, 5 in 2024-02-13T14:13:40.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:40 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-13T14:13:40.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:40 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-13T14:13:40.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:40 smithi087 ceph-mon[27372]: from='osd.4 ' entity='osd.4' 2024-02-13T14:13:40.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:40 smithi087 ceph-mon[39120]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-13T14:13:40.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:40 smithi087 ceph-mon[39120]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]': finished 2024-02-13T14:13:40.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:40 smithi087 ceph-mon[39120]: osdmap e30: 5 total, 4 up, 5 in 2024-02-13T14:13:40.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:40 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-13T14:13:40.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:40 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-13T14:13:40.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:40 smithi087 ceph-mon[39120]: from='osd.4 ' entity='osd.4' 2024-02-13T14:13:41.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:41 smithi148 ceph-mon[31198]: purged_snaps scrub starts 2024-02-13T14:13:41.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:41 smithi148 ceph-mon[31198]: purged_snaps scrub ok 2024-02-13T14:13:41.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:41 smithi148 ceph-mon[31198]: osd.4 [v2:172.21.15.148:6800/1403587816,v1:172.21.15.148:6801/1403587816] boot 2024-02-13T14:13:41.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:41 smithi148 ceph-mon[31198]: osdmap e31: 5 total, 5 up, 5 in 2024-02-13T14:13:41.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:41 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-13T14:13:41.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:41 smithi148 ceph-mon[31198]: osdmap e32: 5 total, 5 up, 5 in 2024-02-13T14:13:41.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:41 smithi087 ceph-mon[27372]: purged_snaps scrub starts 2024-02-13T14:13:41.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:41 smithi087 ceph-mon[27372]: purged_snaps scrub ok 2024-02-13T14:13:41.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:41 smithi087 ceph-mon[27372]: osd.4 [v2:172.21.15.148:6800/1403587816,v1:172.21.15.148:6801/1403587816] boot 2024-02-13T14:13:41.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:41 smithi087 ceph-mon[27372]: osdmap e31: 5 total, 5 up, 5 in 2024-02-13T14:13:41.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:41 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-13T14:13:41.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:41 smithi087 ceph-mon[27372]: osdmap e32: 5 total, 5 up, 5 in 2024-02-13T14:13:41.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:41 smithi087 ceph-mon[39120]: purged_snaps scrub starts 2024-02-13T14:13:41.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:41 smithi087 ceph-mon[39120]: purged_snaps scrub ok 2024-02-13T14:13:41.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:41 smithi087 ceph-mon[39120]: osd.4 [v2:172.21.15.148:6800/1403587816,v1:172.21.15.148:6801/1403587816] boot 2024-02-13T14:13:41.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:41 smithi087 ceph-mon[39120]: osdmap e31: 5 total, 5 up, 5 in 2024-02-13T14:13:41.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:41 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-13T14:13:41.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:41 smithi087 ceph-mon[39120]: osdmap e32: 5 total, 5 up, 5 in 2024-02-13T14:13:42.339 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:42 smithi148 ceph-mon[31198]: pgmap v118: 1 pgs: 1 remapped+peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-13T14:13:42.339 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:42 smithi148 ceph-mon[31198]: osdmap e33: 5 total, 5 up, 5 in 2024-02-13T14:13:42.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:42 smithi087 ceph-mon[27372]: pgmap v118: 1 pgs: 1 remapped+peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-13T14:13:42.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:42 smithi087 ceph-mon[27372]: osdmap e33: 5 total, 5 up, 5 in 2024-02-13T14:13:42.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:42 smithi087 ceph-mon[39120]: pgmap v118: 1 pgs: 1 remapped+peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-13T14:13:42.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:42 smithi087 ceph-mon[39120]: osdmap e33: 5 total, 5 up, 5 in 2024-02-13T14:13:42.558 INFO:teuthology.orchestra.run.smithi148.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-02-13T14:13:42.558 INFO:teuthology.orchestra.run.smithi148.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-02-13T14:13:42.558 INFO:teuthology.orchestra.run.smithi148.stderr: stderr: 10+0 records in 2024-02-13T14:13:42.558 INFO:teuthology.orchestra.run.smithi148.stderr:10+0 records out 2024-02-13T14:13:42.558 INFO:teuthology.orchestra.run.smithi148.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0122154 s, 858 MB/s 2024-02-13T14:13:42.558 INFO:teuthology.orchestra.run.smithi148.stderr: stderr: 2024-02-13T14:13:42.558 INFO:teuthology.orchestra.run.smithi148.stderr:--> Zapping successful for: 2024-02-13T14:13:43.177 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5c04b074-ca79-11ee-95b9-87774f69a715 -- ceph orch daemon add osd smithi148:vg_nvme/lv_3 2024-02-13T14:13:43.749 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/mon.b/config 2024-02-13T14:13:43.868 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:43 smithi148 ceph-mon[31198]: osdmap e34: 5 total, 5 up, 5 in 2024-02-13T14:13:43.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:43 smithi148 ceph-mon[31198]: pgmap v121: 1 pgs: 1 remapped+peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-13T14:13:43.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:43 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:13:43.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:43 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:13:43.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:43 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-13T14:13:43.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:43 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:13:43.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:43 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:13:43.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:43 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:13:43.870 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:43 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:13:44.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:43 smithi087 ceph-mon[27372]: osdmap e34: 5 total, 5 up, 5 in 2024-02-13T14:13:44.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:43 smithi087 ceph-mon[27372]: pgmap v121: 1 pgs: 1 remapped+peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-13T14:13:44.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:43 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:13:44.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:43 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:13:44.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:43 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-13T14:13:44.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:43 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:13:44.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:43 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:13:44.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:43 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:13:44.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:43 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:13:44.243 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:43 smithi087 ceph-mon[39120]: osdmap e34: 5 total, 5 up, 5 in 2024-02-13T14:13:44.243 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:43 smithi087 ceph-mon[39120]: pgmap v121: 1 pgs: 1 remapped+peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-13T14:13:44.243 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:43 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:13:44.243 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:43 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:13:44.243 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:43 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-13T14:13:44.243 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:43 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:13:44.243 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:43 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:13:44.243 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:43 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:13:44.243 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:43 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:13:44.859 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:44 smithi148 ceph-mon[31198]: Detected new or changed devices on smithi148 2024-02-13T14:13:44.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:44 smithi148 ceph-mon[31198]: Adjusting osd_memory_target on smithi148 to 17088M 2024-02-13T14:13:44.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:44 smithi148 ceph-mon[31198]: osdmap e35: 5 total, 5 up, 5 in 2024-02-13T14:13:45.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:44 smithi087 ceph-mon[27372]: Detected new or changed devices on smithi148 2024-02-13T14:13:45.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:44 smithi087 ceph-mon[27372]: Adjusting osd_memory_target on smithi148 to 17088M 2024-02-13T14:13:45.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:44 smithi087 ceph-mon[27372]: osdmap e35: 5 total, 5 up, 5 in 2024-02-13T14:13:45.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:44 smithi087 ceph-mon[39120]: Detected new or changed devices on smithi148 2024-02-13T14:13:45.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:44 smithi087 ceph-mon[39120]: Adjusting osd_memory_target on smithi148 to 17088M 2024-02-13T14:13:45.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:44 smithi087 ceph-mon[39120]: osdmap e35: 5 total, 5 up, 5 in 2024-02-13T14:13:46.118 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:45 smithi148 ceph-mon[31198]: pgmap v123: 1 pgs: 1 remapped+peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-13T14:13:46.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:45 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-13T14:13:46.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:45 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-13T14:13:46.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:45 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:13:46.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:45 smithi087 ceph-mon[39120]: pgmap v123: 1 pgs: 1 remapped+peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-13T14:13:46.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:45 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-13T14:13:46.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:45 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-13T14:13:46.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:45 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:13:46.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:45 smithi087 ceph-mon[27372]: pgmap v123: 1 pgs: 1 remapped+peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-13T14:13:46.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:45 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-13T14:13:46.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:45 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-13T14:13:46.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:45 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:13:47.118 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:46 smithi148 ceph-mon[31198]: from='client.24220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T14:13:47.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:46 smithi087 ceph-mon[27372]: from='client.24220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T14:13:47.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:46 smithi087 ceph-mon[39120]: from='client.24220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T14:13:48.118 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:47 smithi148 ceph-mon[31198]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 136 MiB used, 447 GiB / 447 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-02-13T14:13:48.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:47 smithi087 ceph-mon[27372]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 136 MiB used, 447 GiB / 447 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-02-13T14:13:48.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:47 smithi087 ceph-mon[39120]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 136 MiB used, 447 GiB / 447 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-02-13T14:13:49.118 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:48 smithi148 ceph-mon[31198]: from='client.? 172.21.15.148:0/1417007518' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cba0ca7e-9575-410b-92d6-d9bb992608f4"}]: dispatch 2024-02-13T14:13:49.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:48 smithi148 ceph-mon[31198]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cba0ca7e-9575-410b-92d6-d9bb992608f4"}]: dispatch 2024-02-13T14:13:49.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:48 smithi148 ceph-mon[31198]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cba0ca7e-9575-410b-92d6-d9bb992608f4"}]': finished 2024-02-13T14:13:49.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:48 smithi148 ceph-mon[31198]: osdmap e36: 6 total, 5 up, 6 in 2024-02-13T14:13:49.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:48 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-13T14:13:49.143 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:48 smithi087 ceph-mon[27372]: from='client.? 172.21.15.148:0/1417007518' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cba0ca7e-9575-410b-92d6-d9bb992608f4"}]: dispatch 2024-02-13T14:13:49.143 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:48 smithi087 ceph-mon[27372]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cba0ca7e-9575-410b-92d6-d9bb992608f4"}]: dispatch 2024-02-13T14:13:49.144 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:48 smithi087 ceph-mon[27372]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cba0ca7e-9575-410b-92d6-d9bb992608f4"}]': finished 2024-02-13T14:13:49.144 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:48 smithi087 ceph-mon[27372]: osdmap e36: 6 total, 5 up, 6 in 2024-02-13T14:13:49.144 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:48 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-13T14:13:49.144 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:48 smithi087 ceph-mon[39120]: from='client.? 172.21.15.148:0/1417007518' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cba0ca7e-9575-410b-92d6-d9bb992608f4"}]: dispatch 2024-02-13T14:13:49.144 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:48 smithi087 ceph-mon[39120]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cba0ca7e-9575-410b-92d6-d9bb992608f4"}]: dispatch 2024-02-13T14:13:49.144 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:48 smithi087 ceph-mon[39120]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cba0ca7e-9575-410b-92d6-d9bb992608f4"}]': finished 2024-02-13T14:13:49.144 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:48 smithi087 ceph-mon[39120]: osdmap e36: 6 total, 5 up, 6 in 2024-02-13T14:13:49.144 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:48 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-13T14:13:50.118 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:49 smithi148 ceph-mon[31198]: from='client.? 172.21.15.148:0/3099846317' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-13T14:13:50.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:49 smithi148 ceph-mon[31198]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-02-13T14:13:50.241 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:49 smithi087 ceph-mon[39120]: from='client.? 172.21.15.148:0/3099846317' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-13T14:13:50.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:49 smithi087 ceph-mon[39120]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-02-13T14:13:50.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:49 smithi087 ceph-mon[27372]: from='client.? 172.21.15.148:0/3099846317' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-13T14:13:50.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:49 smithi087 ceph-mon[27372]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-02-13T14:13:52.118 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:51 smithi148 ceph-mon[31198]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-02-13T14:13:52.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:51 smithi087 ceph-mon[27372]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-02-13T14:13:52.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:51 smithi087 ceph-mon[39120]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-02-13T14:13:53.999 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:53 smithi148 ceph-mon[31198]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-02-13T14:13:54.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:53 smithi087 ceph-mon[27372]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-02-13T14:13:54.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:53 smithi087 ceph-mon[39120]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-02-13T14:13:55.241 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:54 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-02-13T14:13:55.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:54 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:13:55.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:54 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-02-13T14:13:55.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:54 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:13:55.311 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:54 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-02-13T14:13:55.311 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:54 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:13:56.118 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:55 smithi148 ceph-mon[31198]: Deploying daemon osd.5 on smithi148 2024-02-13T14:13:56.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:55 smithi148 ceph-mon[31198]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-02-13T14:13:56.243 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:55 smithi087 ceph-mon[27372]: Deploying daemon osd.5 on smithi148 2024-02-13T14:13:56.243 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:55 smithi087 ceph-mon[27372]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-02-13T14:13:56.244 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:55 smithi087 ceph-mon[39120]: Deploying daemon osd.5 on smithi148 2024-02-13T14:13:56.244 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:55 smithi087 ceph-mon[39120]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-02-13T14:13:58.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:57 smithi087 ceph-mon[27372]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-13T14:13:58.346 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:57 smithi087 ceph-mon[39120]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-13T14:13:58.346 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:57 smithi148 ceph-mon[31198]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-13T14:14:00.241 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:59 smithi087 ceph-mon[27372]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-13T14:14:00.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:59 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:14:00.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:59 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:14:00.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:13:59 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:14:00.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:59 smithi087 ceph-mon[39120]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-13T14:14:00.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:59 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:14:00.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:59 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:14:00.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:13:59 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:14:00.365 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:59 smithi148 ceph-mon[31198]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-13T14:14:00.365 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:59 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:14:00.365 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:59 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:14:00.365 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:13:59 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:14:02.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:01 smithi087 ceph-mon[27372]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-13T14:14:02.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:01 smithi087 ceph-mon[39120]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-13T14:14:02.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:01 smithi148 ceph-mon[31198]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-13T14:14:03.625 INFO:teuthology.orchestra.run.smithi148.stdout:Created osd(s) 5 on host 'smithi148' 2024-02-13T14:14:03.868 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:03 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:14:03.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:03 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:14:03.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:03 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:14:03.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:03 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:14:03.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:03 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:14:03.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:03 smithi148 ceph-mon[31198]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-13T14:14:03.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:03 smithi148 ceph-mon[31198]: from='osd.5 [v2:172.21.15.148:6808/4089539002,v1:172.21.15.148:6809/4089539002]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-13T14:14:03.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:03 smithi148 ceph-mon[31198]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-13T14:14:03.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:03 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:14:03.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:03 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:14:03.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:03 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:14:04.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:03 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:14:04.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:03 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:14:04.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:03 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:14:04.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:03 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:14:04.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:03 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:14:04.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:03 smithi087 ceph-mon[27372]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-13T14:14:04.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:03 smithi087 ceph-mon[27372]: from='osd.5 [v2:172.21.15.148:6808/4089539002,v1:172.21.15.148:6809/4089539002]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-13T14:14:04.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:03 smithi087 ceph-mon[27372]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-13T14:14:04.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:03 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:14:04.243 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:03 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:14:04.243 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:03 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:14:04.243 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:03 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:14:04.243 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:03 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:14:04.243 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:03 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:14:04.243 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:03 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:14:04.243 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:03 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:14:04.243 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:03 smithi087 ceph-mon[39120]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-13T14:14:04.243 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:03 smithi087 ceph-mon[39120]: from='osd.5 [v2:172.21.15.148:6808/4089539002,v1:172.21.15.148:6809/4089539002]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-13T14:14:04.243 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:03 smithi087 ceph-mon[39120]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-13T14:14:04.243 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:03 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:14:04.243 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:03 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:14:04.244 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:03 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:14:04.282 DEBUG:teuthology.orchestra.run.smithi148:osd.5> sudo journalctl -f -n 0 -u ceph-5c04b074-ca79-11ee-95b9-87774f69a715@osd.5.service 2024-02-13T14:14:04.284 INFO:tasks.cephadm:Deploying osd.6 on smithi148 with /dev/vg_nvme/lv_2... 2024-02-13T14:14:04.284 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5c04b074-ca79-11ee-95b9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-02-13T14:14:04.868 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:04 smithi148 ceph-mon[31198]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-02-13T14:14:04.868 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:04 smithi148 ceph-mon[31198]: from='osd.5 [v2:172.21.15.148:6808/4089539002,v1:172.21.15.148:6809/4089539002]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-02-13T14:14:04.868 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:04 smithi148 ceph-mon[31198]: osdmap e37: 6 total, 5 up, 6 in 2024-02-13T14:14:04.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:04 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-13T14:14:04.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:04 smithi148 ceph-mon[31198]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-02-13T14:14:05.010 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/mon.b/config 2024-02-13T14:14:05.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:04 smithi087 ceph-mon[27372]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-02-13T14:14:05.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:04 smithi087 ceph-mon[27372]: from='osd.5 [v2:172.21.15.148:6808/4089539002,v1:172.21.15.148:6809/4089539002]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-02-13T14:14:05.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:04 smithi087 ceph-mon[27372]: osdmap e37: 6 total, 5 up, 6 in 2024-02-13T14:14:05.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:04 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-13T14:14:05.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:04 smithi087 ceph-mon[27372]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-02-13T14:14:05.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:04 smithi087 ceph-mon[39120]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-02-13T14:14:05.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:04 smithi087 ceph-mon[39120]: from='osd.5 [v2:172.21.15.148:6808/4089539002,v1:172.21.15.148:6809/4089539002]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-02-13T14:14:05.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:04 smithi087 ceph-mon[39120]: osdmap e37: 6 total, 5 up, 6 in 2024-02-13T14:14:05.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:04 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-13T14:14:05.243 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:04 smithi087 ceph-mon[39120]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-02-13T14:14:05.382 INFO:journalctl@ceph.osd.5.smithi148.stdout:Feb 13 14:14:05 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-5[42659]: 2024-02-13T14:14:05.280+0000 7f973d1d1700 -1 osd.5 0 waiting for initial osdmap 2024-02-13T14:14:05.382 INFO:journalctl@ceph.osd.5.smithi148.stdout:Feb 13 14:14:05 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-5[42659]: 2024-02-13T14:14:05.284+0000 7f9734fbc700 -1 osd.5 38 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-13T14:14:05.925 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:05 smithi148 ceph-mon[31198]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]': finished 2024-02-13T14:14:05.925 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:05 smithi148 ceph-mon[31198]: osdmap e38: 6 total, 5 up, 6 in 2024-02-13T14:14:05.925 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:05 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-13T14:14:05.925 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:05 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-13T14:14:05.925 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:05 smithi148 ceph-mon[31198]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-13T14:14:05.925 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:05 smithi148 ceph-mon[31198]: from='osd.5 ' entity='osd.5' 2024-02-13T14:14:05.925 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:05 smithi148 ceph-mon[31198]: osd.5 [v2:172.21.15.148:6808/4089539002,v1:172.21.15.148:6809/4089539002] boot 2024-02-13T14:14:05.925 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:05 smithi148 ceph-mon[31198]: osdmap e39: 6 total, 6 up, 6 in 2024-02-13T14:14:05.925 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:05 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-13T14:14:06.241 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:05 smithi087 ceph-mon[27372]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]': finished 2024-02-13T14:14:06.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:05 smithi087 ceph-mon[27372]: osdmap e38: 6 total, 5 up, 6 in 2024-02-13T14:14:06.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:05 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-13T14:14:06.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:05 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-13T14:14:06.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:05 smithi087 ceph-mon[27372]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-13T14:14:06.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:05 smithi087 ceph-mon[27372]: from='osd.5 ' entity='osd.5' 2024-02-13T14:14:06.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:05 smithi087 ceph-mon[27372]: osd.5 [v2:172.21.15.148:6808/4089539002,v1:172.21.15.148:6809/4089539002] boot 2024-02-13T14:14:06.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:05 smithi087 ceph-mon[27372]: osdmap e39: 6 total, 6 up, 6 in 2024-02-13T14:14:06.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:05 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-13T14:14:06.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:05 smithi087 ceph-mon[39120]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]': finished 2024-02-13T14:14:06.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:05 smithi087 ceph-mon[39120]: osdmap e38: 6 total, 5 up, 6 in 2024-02-13T14:14:06.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:05 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-13T14:14:06.243 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:05 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-13T14:14:06.243 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:05 smithi087 ceph-mon[39120]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-13T14:14:06.243 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:05 smithi087 ceph-mon[39120]: from='osd.5 ' entity='osd.5' 2024-02-13T14:14:06.243 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:05 smithi087 ceph-mon[39120]: osd.5 [v2:172.21.15.148:6808/4089539002,v1:172.21.15.148:6809/4089539002] boot 2024-02-13T14:14:06.243 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:05 smithi087 ceph-mon[39120]: osdmap e39: 6 total, 6 up, 6 in 2024-02-13T14:14:06.243 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:05 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-13T14:14:07.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:06 smithi148 ceph-mon[31198]: purged_snaps scrub starts 2024-02-13T14:14:07.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:06 smithi148 ceph-mon[31198]: purged_snaps scrub ok 2024-02-13T14:14:07.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:06 smithi148 ceph-mon[31198]: osdmap e40: 6 total, 6 up, 6 in 2024-02-13T14:14:07.241 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:06 smithi087 ceph-mon[27372]: purged_snaps scrub starts 2024-02-13T14:14:07.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:06 smithi087 ceph-mon[27372]: purged_snaps scrub ok 2024-02-13T14:14:07.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:06 smithi087 ceph-mon[27372]: osdmap e40: 6 total, 6 up, 6 in 2024-02-13T14:14:07.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:06 smithi087 ceph-mon[39120]: purged_snaps scrub starts 2024-02-13T14:14:07.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:06 smithi087 ceph-mon[39120]: purged_snaps scrub ok 2024-02-13T14:14:07.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:06 smithi087 ceph-mon[39120]: osdmap e40: 6 total, 6 up, 6 in 2024-02-13T14:14:07.712 INFO:teuthology.orchestra.run.smithi148.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-02-13T14:14:07.715 INFO:teuthology.orchestra.run.smithi148.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-02-13T14:14:07.715 INFO:teuthology.orchestra.run.smithi148.stderr: stderr: 10+0 records in 2024-02-13T14:14:07.715 INFO:teuthology.orchestra.run.smithi148.stderr:10+0 records out 2024-02-13T14:14:07.715 INFO:teuthology.orchestra.run.smithi148.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.012422 s, 844 MB/s 2024-02-13T14:14:07.715 INFO:teuthology.orchestra.run.smithi148.stderr:--> Zapping successful for: 2024-02-13T14:14:08.118 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:07 smithi148 ceph-mon[31198]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 473 MiB used, 536 GiB / 536 GiB avail 2024-02-13T14:14:08.118 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:07 smithi148 ceph-mon[31198]: osdmap e41: 6 total, 6 up, 6 in 2024-02-13T14:14:08.241 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:07 smithi087 ceph-mon[27372]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 473 MiB used, 536 GiB / 536 GiB avail 2024-02-13T14:14:08.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:07 smithi087 ceph-mon[27372]: osdmap e41: 6 total, 6 up, 6 in 2024-02-13T14:14:08.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:07 smithi087 ceph-mon[39120]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 473 MiB used, 536 GiB / 536 GiB avail 2024-02-13T14:14:08.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:07 smithi087 ceph-mon[39120]: osdmap e41: 6 total, 6 up, 6 in 2024-02-13T14:14:08.993 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5c04b074-ca79-11ee-95b9-87774f69a715 -- ceph orch daemon add osd smithi148:vg_nvme/lv_2 2024-02-13T14:14:09.234 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/mon.b/config 2024-02-13T14:14:09.618 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:09 smithi148 ceph-mon[31198]: Detected new or changed devices on smithi148 2024-02-13T14:14:09.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:09 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:14:09.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:09 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:14:09.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:09 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-13T14:14:09.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:09 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-13T14:14:09.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:09 smithi148 ceph-mon[31198]: Adjusting osd_memory_target on smithi148 to 8544M 2024-02-13T14:14:09.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:09 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:14:09.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:09 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:14:09.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:09 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:14:09.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:09 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:14:09.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:09 smithi148 ceph-mon[31198]: osdmap e42: 6 total, 6 up, 6 in 2024-02-13T14:14:09.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:09 smithi148 ceph-mon[31198]: pgmap v142: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 474 MiB used, 536 GiB / 536 GiB avail 2024-02-13T14:14:09.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:09 smithi087 ceph-mon[27372]: Detected new or changed devices on smithi148 2024-02-13T14:14:09.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:09 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:14:09.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:09 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:14:09.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:09 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-13T14:14:09.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:09 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-13T14:14:09.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:09 smithi087 ceph-mon[27372]: Adjusting osd_memory_target on smithi148 to 8544M 2024-02-13T14:14:09.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:09 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:14:09.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:09 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:14:09.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:09 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:14:09.993 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:09 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:14:09.993 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:09 smithi087 ceph-mon[27372]: osdmap e42: 6 total, 6 up, 6 in 2024-02-13T14:14:09.993 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:09 smithi087 ceph-mon[27372]: pgmap v142: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 474 MiB used, 536 GiB / 536 GiB avail 2024-02-13T14:14:09.993 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:09 smithi087 ceph-mon[39120]: Detected new or changed devices on smithi148 2024-02-13T14:14:09.993 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:09 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:14:09.993 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:09 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:14:09.993 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:09 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-13T14:14:09.993 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:09 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-13T14:14:09.993 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:09 smithi087 ceph-mon[39120]: Adjusting osd_memory_target on smithi148 to 8544M 2024-02-13T14:14:09.993 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:09 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:14:09.993 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:09 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:14:09.993 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:09 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:14:09.994 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:09 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:14:09.994 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:09 smithi087 ceph-mon[39120]: osdmap e42: 6 total, 6 up, 6 in 2024-02-13T14:14:09.994 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:09 smithi087 ceph-mon[39120]: pgmap v142: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 474 MiB used, 536 GiB / 536 GiB avail 2024-02-13T14:14:10.860 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:10 smithi148 ceph-mon[31198]: osdmap e43: 6 total, 6 up, 6 in 2024-02-13T14:14:11.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:10 smithi087 ceph-mon[27372]: osdmap e43: 6 total, 6 up, 6 in 2024-02-13T14:14:11.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:10 smithi087 ceph-mon[39120]: osdmap e43: 6 total, 6 up, 6 in 2024-02-13T14:14:12.002 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:11 smithi148 ceph-mon[31198]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 536 GiB / 536 GiB avail 2024-02-13T14:14:12.003 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:11 smithi148 ceph-mon[31198]: from='client.24247 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T14:14:12.003 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:11 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-13T14:14:12.003 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:11 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-13T14:14:12.003 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:11 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:14:12.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:11 smithi087 ceph-mon[27372]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 536 GiB / 536 GiB avail 2024-02-13T14:14:12.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:11 smithi087 ceph-mon[27372]: from='client.24247 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T14:14:12.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:11 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-13T14:14:12.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:11 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-13T14:14:12.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:11 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:14:12.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:11 smithi087 ceph-mon[39120]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 536 GiB / 536 GiB avail 2024-02-13T14:14:12.243 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:11 smithi087 ceph-mon[39120]: from='client.24247 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T14:14:12.243 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:11 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-13T14:14:12.243 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:11 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-13T14:14:12.243 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:11 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:14:14.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:13 smithi087 ceph-mon[27372]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 536 GiB / 536 GiB avail 2024-02-13T14:14:14.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:13 smithi087 ceph-mon[27372]: from='client.? 172.21.15.148:0/2545768161' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4f958ddf-d72c-434d-bdfc-fb039719e39b"}]: dispatch 2024-02-13T14:14:14.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:13 smithi087 ceph-mon[27372]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4f958ddf-d72c-434d-bdfc-fb039719e39b"}]: dispatch 2024-02-13T14:14:14.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:13 smithi087 ceph-mon[27372]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4f958ddf-d72c-434d-bdfc-fb039719e39b"}]': finished 2024-02-13T14:14:14.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:13 smithi087 ceph-mon[27372]: osdmap e44: 7 total, 6 up, 7 in 2024-02-13T14:14:14.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:13 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-13T14:14:14.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:13 smithi087 ceph-mon[39120]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 536 GiB / 536 GiB avail 2024-02-13T14:14:14.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:13 smithi087 ceph-mon[39120]: from='client.? 172.21.15.148:0/2545768161' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4f958ddf-d72c-434d-bdfc-fb039719e39b"}]: dispatch 2024-02-13T14:14:14.243 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:13 smithi087 ceph-mon[39120]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4f958ddf-d72c-434d-bdfc-fb039719e39b"}]: dispatch 2024-02-13T14:14:14.243 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:13 smithi087 ceph-mon[39120]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4f958ddf-d72c-434d-bdfc-fb039719e39b"}]': finished 2024-02-13T14:14:14.243 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:13 smithi087 ceph-mon[39120]: osdmap e44: 7 total, 6 up, 7 in 2024-02-13T14:14:14.243 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:13 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-13T14:14:14.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:13 smithi148 ceph-mon[31198]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 536 GiB / 536 GiB avail 2024-02-13T14:14:14.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:13 smithi148 ceph-mon[31198]: from='client.? 172.21.15.148:0/2545768161' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4f958ddf-d72c-434d-bdfc-fb039719e39b"}]: dispatch 2024-02-13T14:14:14.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:13 smithi148 ceph-mon[31198]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4f958ddf-d72c-434d-bdfc-fb039719e39b"}]: dispatch 2024-02-13T14:14:14.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:13 smithi148 ceph-mon[31198]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4f958ddf-d72c-434d-bdfc-fb039719e39b"}]': finished 2024-02-13T14:14:14.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:13 smithi148 ceph-mon[31198]: osdmap e44: 7 total, 6 up, 7 in 2024-02-13T14:14:14.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:13 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-13T14:14:15.241 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:14 smithi087 ceph-mon[27372]: from='client.? 172.21.15.148:0/2254189696' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-13T14:14:15.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:14 smithi087 ceph-mon[39120]: from='client.? 172.21.15.148:0/2254189696' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-13T14:14:15.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:14 smithi148 ceph-mon[31198]: from='client.? 172.21.15.148:0/2254189696' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-13T14:14:16.241 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:15 smithi087 ceph-mon[27372]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 536 GiB / 536 GiB avail 2024-02-13T14:14:16.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:15 smithi087 ceph-mon[39120]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 536 GiB / 536 GiB avail 2024-02-13T14:14:16.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:15 smithi148 ceph-mon[31198]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 536 GiB / 536 GiB avail 2024-02-13T14:14:18.158 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:17 smithi148 ceph-mon[31198]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-13T14:14:18.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:17 smithi087 ceph-mon[39120]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-13T14:14:18.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:17 smithi087 ceph-mon[27372]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-13T14:14:20.118 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:19 smithi148 ceph-mon[31198]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-13T14:14:20.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:19 smithi087 ceph-mon[39120]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-13T14:14:20.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:19 smithi087 ceph-mon[27372]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-13T14:14:21.112 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:20 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-02-13T14:14:21.112 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:20 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:14:21.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:20 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-02-13T14:14:21.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:20 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:14:21.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:20 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-02-13T14:14:21.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:20 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:14:22.191 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:21 smithi148 ceph-mon[31198]: Deploying daemon osd.6 on smithi148 2024-02-13T14:14:22.191 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:21 smithi148 ceph-mon[31198]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-13T14:14:22.244 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:21 smithi087 ceph-mon[27372]: Deploying daemon osd.6 on smithi148 2024-02-13T14:14:22.244 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:21 smithi087 ceph-mon[27372]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-13T14:14:22.245 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:21 smithi087 ceph-mon[39120]: Deploying daemon osd.6 on smithi148 2024-02-13T14:14:22.245 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:21 smithi087 ceph-mon[39120]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-13T14:14:24.243 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:23 smithi087 ceph-mon[27372]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-13T14:14:24.243 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:23 smithi087 ceph-mon[39120]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-13T14:14:24.319 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:23 smithi148 ceph-mon[31198]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-13T14:14:26.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:25 smithi148 ceph-mon[31198]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-13T14:14:26.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:25 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:14:26.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:25 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:14:26.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:25 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:14:26.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:25 smithi087 ceph-mon[27372]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-13T14:14:26.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:25 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:14:26.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:25 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:14:26.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:25 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:14:26.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:25 smithi087 ceph-mon[39120]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-13T14:14:26.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:25 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:14:26.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:25 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:14:26.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:25 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:14:28.118 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:27 smithi148 ceph-mon[31198]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-13T14:14:28.491 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:27 smithi087 ceph-mon[27372]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-13T14:14:28.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:27 smithi087 ceph-mon[39120]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-13T14:14:29.614 INFO:teuthology.orchestra.run.smithi148.stdout:Created osd(s) 6 on host 'smithi148' 2024-02-13T14:14:29.868 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:29 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:14:29.868 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:29 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:14:29.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:29 smithi148 ceph-mon[31198]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-13T14:14:29.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:29 smithi148 ceph-mon[31198]: from='osd.6 [v2:172.21.15.148:6816/1791408651,v1:172.21.15.148:6817/1791408651]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-02-13T14:14:29.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:29 smithi148 ceph-mon[31198]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-02-13T14:14:29.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:29 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:14:29.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:29 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:14:29.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:29 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:14:29.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:29 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:14:29.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:29 smithi087 ceph-mon[27372]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-13T14:14:29.993 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:29 smithi087 ceph-mon[27372]: from='osd.6 [v2:172.21.15.148:6816/1791408651,v1:172.21.15.148:6817/1791408651]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-02-13T14:14:29.993 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:29 smithi087 ceph-mon[27372]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-02-13T14:14:29.993 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:29 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:14:29.993 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:29 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:14:29.993 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:29 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:14:29.994 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:29 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:14:29.994 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:29 smithi087 ceph-mon[39120]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-13T14:14:29.994 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:29 smithi087 ceph-mon[39120]: from='osd.6 [v2:172.21.15.148:6816/1791408651,v1:172.21.15.148:6817/1791408651]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-02-13T14:14:29.994 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:29 smithi087 ceph-mon[39120]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-02-13T14:14:29.994 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:29 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:14:29.994 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:29 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:14:30.347 DEBUG:teuthology.orchestra.run.smithi148:osd.6> sudo journalctl -f -n 0 -u ceph-5c04b074-ca79-11ee-95b9-87774f69a715@osd.6.service 2024-02-13T14:14:30.349 INFO:tasks.cephadm:Deploying osd.7 on smithi148 with /dev/vg_nvme/lv_1... 2024-02-13T14:14:30.349 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5c04b074-ca79-11ee-95b9-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-02-13T14:14:30.916 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:30 smithi148 ceph-mon[31198]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-02-13T14:14:30.916 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:30 smithi148 ceph-mon[31198]: from='osd.6 [v2:172.21.15.148:6816/1791408651,v1:172.21.15.148:6817/1791408651]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-02-13T14:14:30.916 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:30 smithi148 ceph-mon[31198]: osdmap e45: 7 total, 6 up, 7 in 2024-02-13T14:14:30.916 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:30 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-13T14:14:30.916 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:30 smithi148 ceph-mon[31198]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-02-13T14:14:31.136 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/mon.b/config 2024-02-13T14:14:31.241 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:30 smithi087 ceph-mon[27372]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-02-13T14:14:31.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:30 smithi087 ceph-mon[27372]: from='osd.6 [v2:172.21.15.148:6816/1791408651,v1:172.21.15.148:6817/1791408651]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-02-13T14:14:31.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:30 smithi087 ceph-mon[27372]: osdmap e45: 7 total, 6 up, 7 in 2024-02-13T14:14:31.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:30 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-13T14:14:31.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:30 smithi087 ceph-mon[27372]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-02-13T14:14:31.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:30 smithi087 ceph-mon[39120]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-02-13T14:14:31.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:30 smithi087 ceph-mon[39120]: from='osd.6 [v2:172.21.15.148:6816/1791408651,v1:172.21.15.148:6817/1791408651]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-02-13T14:14:31.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:30 smithi087 ceph-mon[39120]: osdmap e45: 7 total, 6 up, 7 in 2024-02-13T14:14:31.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:30 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-13T14:14:31.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:30 smithi087 ceph-mon[39120]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-02-13T14:14:31.486 INFO:journalctl@ceph.osd.6.smithi148.stdout:Feb 13 14:14:31 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-6[48606]: 2024-02-13T14:14:31.194+0000 7f9ede646700 -1 osd.6 0 waiting for initial osdmap 2024-02-13T14:14:31.487 INFO:journalctl@ceph.osd.6.smithi148.stdout:Feb 13 14:14:31 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-6[48606]: 2024-02-13T14:14:31.201+0000 7f9ed5c30700 -1 osd.6 47 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-13T14:14:32.043 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:31 smithi148 ceph-mon[31198]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]': finished 2024-02-13T14:14:32.043 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:31 smithi148 ceph-mon[31198]: osdmap e46: 7 total, 6 up, 7 in 2024-02-13T14:14:32.043 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:31 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-13T14:14:32.043 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:31 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-13T14:14:32.043 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:31 smithi148 ceph-mon[31198]: osdmap e47: 7 total, 6 up, 7 in 2024-02-13T14:14:32.043 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:31 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-13T14:14:32.043 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:31 smithi148 ceph-mon[31198]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-13T14:14:32.044 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:31 smithi148 ceph-mon[31198]: from='osd.6 ' entity='osd.6' 2024-02-13T14:14:32.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:31 smithi087 ceph-mon[39120]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]': finished 2024-02-13T14:14:32.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:31 smithi087 ceph-mon[39120]: osdmap e46: 7 total, 6 up, 7 in 2024-02-13T14:14:32.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:31 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-13T14:14:32.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:31 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-13T14:14:32.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:31 smithi087 ceph-mon[39120]: osdmap e47: 7 total, 6 up, 7 in 2024-02-13T14:14:32.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:31 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-13T14:14:32.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:31 smithi087 ceph-mon[39120]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-13T14:14:32.243 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:31 smithi087 ceph-mon[39120]: from='osd.6 ' entity='osd.6' 2024-02-13T14:14:32.243 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:31 smithi087 ceph-mon[27372]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]': finished 2024-02-13T14:14:32.243 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:31 smithi087 ceph-mon[27372]: osdmap e46: 7 total, 6 up, 7 in 2024-02-13T14:14:32.243 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:31 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-13T14:14:32.243 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:31 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-13T14:14:32.243 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:31 smithi087 ceph-mon[27372]: osdmap e47: 7 total, 6 up, 7 in 2024-02-13T14:14:32.243 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:31 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-13T14:14:32.243 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:31 smithi087 ceph-mon[27372]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-13T14:14:32.243 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:31 smithi087 ceph-mon[27372]: from='osd.6 ' entity='osd.6' 2024-02-13T14:14:33.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:32 smithi148 ceph-mon[31198]: purged_snaps scrub starts 2024-02-13T14:14:33.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:32 smithi148 ceph-mon[31198]: purged_snaps scrub ok 2024-02-13T14:14:33.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:32 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-13T14:14:33.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:32 smithi148 ceph-mon[31198]: osd.6 [v2:172.21.15.148:6816/1791408651,v1:172.21.15.148:6817/1791408651] boot 2024-02-13T14:14:33.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:32 smithi148 ceph-mon[31198]: osdmap e48: 7 total, 7 up, 7 in 2024-02-13T14:14:33.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:32 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-13T14:14:33.241 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:32 smithi087 ceph-mon[39120]: purged_snaps scrub starts 2024-02-13T14:14:33.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:32 smithi087 ceph-mon[39120]: purged_snaps scrub ok 2024-02-13T14:14:33.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:32 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-13T14:14:33.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:32 smithi087 ceph-mon[39120]: osd.6 [v2:172.21.15.148:6816/1791408651,v1:172.21.15.148:6817/1791408651] boot 2024-02-13T14:14:33.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:32 smithi087 ceph-mon[39120]: osdmap e48: 7 total, 7 up, 7 in 2024-02-13T14:14:33.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:32 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-13T14:14:33.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:32 smithi087 ceph-mon[27372]: purged_snaps scrub starts 2024-02-13T14:14:33.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:32 smithi087 ceph-mon[27372]: purged_snaps scrub ok 2024-02-13T14:14:33.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:32 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-13T14:14:33.243 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:32 smithi087 ceph-mon[27372]: osd.6 [v2:172.21.15.148:6816/1791408651,v1:172.21.15.148:6817/1791408651] boot 2024-02-13T14:14:33.243 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:32 smithi087 ceph-mon[27372]: osdmap e48: 7 total, 7 up, 7 in 2024-02-13T14:14:33.243 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:32 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-13T14:14:34.060 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:33 smithi148 ceph-mon[31198]: osdmap e49: 7 total, 7 up, 7 in 2024-02-13T14:14:34.060 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:33 smithi148 ceph-mon[31198]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-02-13T14:14:34.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:33 smithi087 ceph-mon[39120]: osdmap e49: 7 total, 7 up, 7 in 2024-02-13T14:14:34.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:33 smithi087 ceph-mon[39120]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-02-13T14:14:34.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:33 smithi087 ceph-mon[27372]: osdmap e49: 7 total, 7 up, 7 in 2024-02-13T14:14:34.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:33 smithi087 ceph-mon[27372]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-02-13T14:14:34.272 INFO:teuthology.orchestra.run.smithi148.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-02-13T14:14:34.272 INFO:teuthology.orchestra.run.smithi148.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-02-13T14:14:34.272 INFO:teuthology.orchestra.run.smithi148.stderr: stderr: 10+0 records in 2024-02-13T14:14:34.272 INFO:teuthology.orchestra.run.smithi148.stderr:10+0 records out 2024-02-13T14:14:34.272 INFO:teuthology.orchestra.run.smithi148.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0149866 s, 700 MB/s 2024-02-13T14:14:34.272 INFO:teuthology.orchestra.run.smithi148.stderr: stderr: 2024-02-13T14:14:34.272 INFO:teuthology.orchestra.run.smithi148.stderr:--> Zapping successful for: 2024-02-13T14:14:35.006 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:34 smithi148 ceph-mon[31198]: osdmap e50: 7 total, 7 up, 7 in 2024-02-13T14:14:35.006 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:34 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:14:35.006 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:34 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:14:35.006 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:34 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-13T14:14:35.006 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:34 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-13T14:14:35.006 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:34 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-13T14:14:35.006 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:34 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:14:35.006 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:34 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:14:35.006 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:34 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:14:35.006 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:34 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:14:35.006 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:34 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:14:35.006 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:34 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:14:35.007 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:34 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:14:35.007 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:34 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:14:35.032 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5c04b074-ca79-11ee-95b9-87774f69a715 -- ceph orch daemon add osd smithi148:vg_nvme/lv_1 2024-02-13T14:14:35.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:34 smithi087 ceph-mon[27372]: osdmap e50: 7 total, 7 up, 7 in 2024-02-13T14:14:35.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:34 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:14:35.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:34 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:14:35.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:34 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-13T14:14:35.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:34 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-13T14:14:35.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:34 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-13T14:14:35.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:34 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:14:35.243 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:34 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:14:35.243 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:34 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:14:35.243 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:34 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:14:35.243 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:34 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:14:35.243 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:34 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:14:35.243 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:34 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:14:35.243 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:34 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:14:35.243 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:34 smithi087 ceph-mon[39120]: osdmap e50: 7 total, 7 up, 7 in 2024-02-13T14:14:35.243 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:34 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:14:35.243 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:34 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:14:35.243 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:34 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-13T14:14:35.243 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:34 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-13T14:14:35.243 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:34 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-13T14:14:35.244 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:34 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:14:35.244 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:34 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:14:35.244 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:34 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:14:35.244 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:34 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:14:35.244 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:34 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:14:35.244 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:34 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:14:35.244 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:34 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:14:35.244 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:34 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:14:35.270 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/mon.b/config 2024-02-13T14:14:36.095 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:35 smithi148 ceph-mon[31198]: Detected new or changed devices on smithi148 2024-02-13T14:14:36.095 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:35 smithi148 ceph-mon[31198]: Adjusting osd_memory_target on smithi148 to 5696M 2024-02-13T14:14:36.095 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:35 smithi148 ceph-mon[31198]: osdmap e51: 7 total, 7 up, 7 in 2024-02-13T14:14:36.095 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:35 smithi148 ceph-mon[31198]: pgmap v164: 1 pgs: 1 remapped+peering; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-02-13T14:14:36.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:35 smithi087 ceph-mon[27372]: Detected new or changed devices on smithi148 2024-02-13T14:14:36.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:35 smithi087 ceph-mon[27372]: Adjusting osd_memory_target on smithi148 to 5696M 2024-02-13T14:14:36.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:35 smithi087 ceph-mon[27372]: osdmap e51: 7 total, 7 up, 7 in 2024-02-13T14:14:36.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:35 smithi087 ceph-mon[27372]: pgmap v164: 1 pgs: 1 remapped+peering; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-02-13T14:14:36.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:35 smithi087 ceph-mon[39120]: Detected new or changed devices on smithi148 2024-02-13T14:14:36.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:35 smithi087 ceph-mon[39120]: Adjusting osd_memory_target on smithi148 to 5696M 2024-02-13T14:14:36.243 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:35 smithi087 ceph-mon[39120]: osdmap e51: 7 total, 7 up, 7 in 2024-02-13T14:14:36.243 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:35 smithi087 ceph-mon[39120]: pgmap v164: 1 pgs: 1 remapped+peering; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2024-02-13T14:14:37.118 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:36 smithi148 ceph-mon[31198]: osdmap e52: 7 total, 7 up, 7 in 2024-02-13T14:14:37.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:36 smithi087 ceph-mon[27372]: osdmap e52: 7 total, 7 up, 7 in 2024-02-13T14:14:37.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:36 smithi087 ceph-mon[39120]: osdmap e52: 7 total, 7 up, 7 in 2024-02-13T14:14:37.984 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:37 smithi148 ceph-mon[31198]: pgmap v166: 1 pgs: 1 remapped+peering; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2024-02-13T14:14:37.984 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:37 smithi148 ceph-mon[31198]: from='client.24274 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T14:14:37.984 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:37 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-13T14:14:37.984 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:37 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-13T14:14:37.984 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:37 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:14:38.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:37 smithi087 ceph-mon[27372]: pgmap v166: 1 pgs: 1 remapped+peering; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2024-02-13T14:14:38.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:37 smithi087 ceph-mon[27372]: from='client.24274 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T14:14:38.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:37 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-13T14:14:38.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:37 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-13T14:14:38.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:37 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:14:38.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:37 smithi087 ceph-mon[39120]: pgmap v166: 1 pgs: 1 remapped+peering; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2024-02-13T14:14:38.243 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:37 smithi087 ceph-mon[39120]: from='client.24274 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T14:14:38.243 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:37 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-13T14:14:38.243 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:37 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-13T14:14:38.243 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:37 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:14:40.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:39 smithi087 ceph-mon[39120]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-02-13T14:14:40.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:39 smithi087 ceph-mon[39120]: from='client.? 172.21.15.148:0/3878780607' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "528011a9-d721-42e5-ba03-d44ef41d1bf1"}]: dispatch 2024-02-13T14:14:40.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:39 smithi087 ceph-mon[39120]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "528011a9-d721-42e5-ba03-d44ef41d1bf1"}]: dispatch 2024-02-13T14:14:40.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:39 smithi087 ceph-mon[39120]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "528011a9-d721-42e5-ba03-d44ef41d1bf1"}]': finished 2024-02-13T14:14:40.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:39 smithi087 ceph-mon[39120]: osdmap e53: 8 total, 7 up, 8 in 2024-02-13T14:14:40.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:39 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-13T14:14:40.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:39 smithi087 ceph-mon[27372]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-02-13T14:14:40.243 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:39 smithi087 ceph-mon[27372]: from='client.? 172.21.15.148:0/3878780607' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "528011a9-d721-42e5-ba03-d44ef41d1bf1"}]: dispatch 2024-02-13T14:14:40.243 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:39 smithi087 ceph-mon[27372]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "528011a9-d721-42e5-ba03-d44ef41d1bf1"}]: dispatch 2024-02-13T14:14:40.243 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:39 smithi087 ceph-mon[27372]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "528011a9-d721-42e5-ba03-d44ef41d1bf1"}]': finished 2024-02-13T14:14:40.243 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:39 smithi087 ceph-mon[27372]: osdmap e53: 8 total, 7 up, 8 in 2024-02-13T14:14:40.243 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:39 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-13T14:14:40.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:39 smithi148 ceph-mon[31198]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-02-13T14:14:40.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:39 smithi148 ceph-mon[31198]: from='client.? 172.21.15.148:0/3878780607' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "528011a9-d721-42e5-ba03-d44ef41d1bf1"}]: dispatch 2024-02-13T14:14:40.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:39 smithi148 ceph-mon[31198]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "528011a9-d721-42e5-ba03-d44ef41d1bf1"}]: dispatch 2024-02-13T14:14:40.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:39 smithi148 ceph-mon[31198]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "528011a9-d721-42e5-ba03-d44ef41d1bf1"}]': finished 2024-02-13T14:14:40.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:39 smithi148 ceph-mon[31198]: osdmap e53: 8 total, 7 up, 8 in 2024-02-13T14:14:40.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:39 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-13T14:14:41.241 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:40 smithi087 ceph-mon[39120]: from='client.? 172.21.15.148:0/3539476326' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-13T14:14:41.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:40 smithi087 ceph-mon[27372]: from='client.? 172.21.15.148:0/3539476326' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-13T14:14:41.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:40 smithi148 ceph-mon[31198]: from='client.? 172.21.15.148:0/3539476326' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-13T14:14:42.241 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:41 smithi087 ceph-mon[39120]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-02-13T14:14:42.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:41 smithi087 ceph-mon[27372]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-02-13T14:14:42.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:41 smithi148 ceph-mon[31198]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-02-13T14:14:44.189 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:43 smithi148 ceph-mon[31198]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-02-13T14:14:44.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:43 smithi087 ceph-mon[39120]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-02-13T14:14:44.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:43 smithi087 ceph-mon[27372]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-02-13T14:14:46.082 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:45 smithi148 ceph-mon[31198]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-02-13T14:14:46.241 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:45 smithi087 ceph-mon[39120]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-02-13T14:14:46.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:45 smithi087 ceph-mon[27372]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-02-13T14:14:47.268 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:46 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-02-13T14:14:47.268 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:46 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:14:47.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:46 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-02-13T14:14:47.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:46 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:14:47.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:46 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-02-13T14:14:47.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:46 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:14:48.238 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:47 smithi148 ceph-mon[31198]: Deploying daemon osd.7 on smithi148 2024-02-13T14:14:48.238 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:47 smithi148 ceph-mon[31198]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-02-13T14:14:48.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:47 smithi087 ceph-mon[39120]: Deploying daemon osd.7 on smithi148 2024-02-13T14:14:48.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:47 smithi087 ceph-mon[39120]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-02-13T14:14:48.493 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:47 smithi087 ceph-mon[27372]: Deploying daemon osd.7 on smithi148 2024-02-13T14:14:48.493 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:47 smithi087 ceph-mon[27372]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-02-13T14:14:50.294 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:49 smithi087 ceph-mon[27372]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2024-02-13T14:14:50.294 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:49 smithi087 ceph-mon[39120]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2024-02-13T14:14:50.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:49 smithi148 ceph-mon[31198]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2024-02-13T14:14:52.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:52 smithi148 ceph-mon[31198]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2024-02-13T14:14:52.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:52 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:14:52.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:52 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:14:52.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:52 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:14:52.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:52 smithi087 ceph-mon[39120]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2024-02-13T14:14:52.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:52 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:14:52.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:52 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:14:52.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:52 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:14:52.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:52 smithi087 ceph-mon[27372]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2024-02-13T14:14:52.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:52 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:14:52.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:52 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:14:52.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:52 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:14:54.239 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:54 smithi148 ceph-mon[31198]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2024-02-13T14:14:54.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:54 smithi087 ceph-mon[27372]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2024-02-13T14:14:54.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:54 smithi087 ceph-mon[39120]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2024-02-13T14:14:55.813 INFO:teuthology.orchestra.run.smithi148.stdout:Created osd(s) 7 on host 'smithi148' 2024-02-13T14:14:56.118 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:55 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:14:56.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:55 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:14:56.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:55 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:14:56.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:55 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:14:56.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:55 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:14:56.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:55 smithi148 ceph-mon[31198]: from='osd.7 [v2:172.21.15.148:6824/2269337484,v1:172.21.15.148:6825/2269337484]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-02-13T14:14:56.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:55 smithi148 ceph-mon[31198]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2024-02-13T14:14:56.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:55 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:14:56.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:55 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:14:56.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:55 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:14:56.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:55 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:14:56.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:55 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:14:56.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:55 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:14:56.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:55 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:14:56.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:55 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:14:56.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:55 smithi087 ceph-mon[39120]: from='osd.7 [v2:172.21.15.148:6824/2269337484,v1:172.21.15.148:6825/2269337484]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-02-13T14:14:56.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:55 smithi087 ceph-mon[39120]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2024-02-13T14:14:56.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:55 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:14:56.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:55 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:14:56.243 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:55 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:14:56.243 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:55 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:14:56.243 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:55 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:14:56.243 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:55 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:14:56.243 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:55 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:14:56.243 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:55 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:14:56.243 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:55 smithi087 ceph-mon[27372]: from='osd.7 [v2:172.21.15.148:6824/2269337484,v1:172.21.15.148:6825/2269337484]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-02-13T14:14:56.243 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:55 smithi087 ceph-mon[27372]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2024-02-13T14:14:56.243 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:55 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:14:56.243 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:55 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:14:56.243 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:55 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:14:56.449 DEBUG:teuthology.orchestra.run.smithi148:osd.7> sudo journalctl -f -n 0 -u ceph-5c04b074-ca79-11ee-95b9-87774f69a715@osd.7.service 2024-02-13T14:14:56.451 INFO:tasks.cephadm:Waiting for 8 OSDs to come up... 2024-02-13T14:14:56.452 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5c04b074-ca79-11ee-95b9-87774f69a715 -- ceph osd stat -f json 2024-02-13T14:14:56.702 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/mon.a/config 2024-02-13T14:14:57.057 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:56 smithi148 ceph-mon[31198]: from='osd.7 [v2:172.21.15.148:6824/2269337484,v1:172.21.15.148:6825/2269337484]' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-02-13T14:14:57.057 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:56 smithi148 ceph-mon[31198]: osdmap e54: 8 total, 7 up, 8 in 2024-02-13T14:14:57.057 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:56 smithi148 ceph-mon[31198]: from='osd.7 [v2:172.21.15.148:6824/2269337484,v1:172.21.15.148:6825/2269337484]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-02-13T14:14:57.057 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:56 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-13T14:14:57.096 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:56 smithi087 ceph-mon[27372]: from='osd.7 [v2:172.21.15.148:6824/2269337484,v1:172.21.15.148:6825/2269337484]' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-02-13T14:14:57.097 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:56 smithi087 ceph-mon[27372]: osdmap e54: 8 total, 7 up, 8 in 2024-02-13T14:14:57.097 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:56 smithi087 ceph-mon[27372]: from='osd.7 [v2:172.21.15.148:6824/2269337484,v1:172.21.15.148:6825/2269337484]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-02-13T14:14:57.097 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:56 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-13T14:14:57.097 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:56 smithi087 ceph-mon[39120]: from='osd.7 [v2:172.21.15.148:6824/2269337484,v1:172.21.15.148:6825/2269337484]' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-02-13T14:14:57.097 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:56 smithi087 ceph-mon[39120]: osdmap e54: 8 total, 7 up, 8 in 2024-02-13T14:14:57.097 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:56 smithi087 ceph-mon[39120]: from='osd.7 [v2:172.21.15.148:6824/2269337484,v1:172.21.15.148:6825/2269337484]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-02-13T14:14:57.097 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:56 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-13T14:14:57.564 INFO:journalctl@ceph.osd.7.smithi148.stdout:Feb 13 14:14:57 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-7[54686]: 2024-02-13T14:14:57.335+0000 7f402c02b700 -1 osd.7 0 waiting for initial osdmap 2024-02-13T14:14:57.564 INFO:journalctl@ceph.osd.7.smithi148.stdout:Feb 13 14:14:57 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-7[54686]: 2024-02-13T14:14:57.342+0000 7f4023e16700 -1 osd.7 55 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-13T14:14:58.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:57 smithi148 ceph-mon[31198]: from='osd.7 [v2:172.21.15.148:6824/2269337484,v1:172.21.15.148:6825/2269337484]' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]': finished 2024-02-13T14:14:58.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:57 smithi148 ceph-mon[31198]: osdmap e55: 8 total, 7 up, 8 in 2024-02-13T14:14:58.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:57 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-13T14:14:58.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:57 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-13T14:14:58.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:57 smithi148 ceph-mon[31198]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2024-02-13T14:14:58.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:57 smithi148 ceph-mon[31198]: from='osd.7 [v2:172.21.15.148:6824/2269337484,v1:172.21.15.148:6825/2269337484]' entity='osd.7' 2024-02-13T14:14:58.176 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:57 smithi087 ceph-mon[27372]: from='osd.7 [v2:172.21.15.148:6824/2269337484,v1:172.21.15.148:6825/2269337484]' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]': finished 2024-02-13T14:14:58.176 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:57 smithi087 ceph-mon[27372]: osdmap e55: 8 total, 7 up, 8 in 2024-02-13T14:14:58.176 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:57 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-13T14:14:58.176 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:57 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-13T14:14:58.176 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:57 smithi087 ceph-mon[27372]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2024-02-13T14:14:58.176 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:57 smithi087 ceph-mon[27372]: from='osd.7 [v2:172.21.15.148:6824/2269337484,v1:172.21.15.148:6825/2269337484]' entity='osd.7' 2024-02-13T14:14:58.176 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:57 smithi087 ceph-mon[39120]: from='osd.7 [v2:172.21.15.148:6824/2269337484,v1:172.21.15.148:6825/2269337484]' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]': finished 2024-02-13T14:14:58.176 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:57 smithi087 ceph-mon[39120]: osdmap e55: 8 total, 7 up, 8 in 2024-02-13T14:14:58.176 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:57 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-13T14:14:58.176 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:57 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-13T14:14:58.177 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:57 smithi087 ceph-mon[39120]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 190 MiB used, 626 GiB / 626 GiB avail 2024-02-13T14:14:58.177 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:57 smithi087 ceph-mon[39120]: from='osd.7 [v2:172.21.15.148:6824/2269337484,v1:172.21.15.148:6825/2269337484]' entity='osd.7' 2024-02-13T14:14:58.693 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-02-13T14:14:59.012 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:58 smithi148 ceph-mon[31198]: purged_snaps scrub starts 2024-02-13T14:14:59.012 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:58 smithi148 ceph-mon[31198]: purged_snaps scrub ok 2024-02-13T14:14:59.012 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:58 smithi148 ceph-mon[31198]: osd.7 [v2:172.21.15.148:6824/2269337484,v1:172.21.15.148:6825/2269337484] boot 2024-02-13T14:14:59.012 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:58 smithi148 ceph-mon[31198]: osdmap e56: 8 total, 8 up, 8 in 2024-02-13T14:14:59.012 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:58 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-13T14:14:59.012 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:58 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/3869658858' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-13T14:14:59.241 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:58 smithi087 ceph-mon[39120]: purged_snaps scrub starts 2024-02-13T14:14:59.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:58 smithi087 ceph-mon[39120]: purged_snaps scrub ok 2024-02-13T14:14:59.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:58 smithi087 ceph-mon[39120]: osd.7 [v2:172.21.15.148:6824/2269337484,v1:172.21.15.148:6825/2269337484] boot 2024-02-13T14:14:59.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:58 smithi087 ceph-mon[39120]: osdmap e56: 8 total, 8 up, 8 in 2024-02-13T14:14:59.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:58 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-13T14:14:59.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:58 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/3869658858' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-13T14:14:59.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:58 smithi087 ceph-mon[27372]: purged_snaps scrub starts 2024-02-13T14:14:59.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:58 smithi087 ceph-mon[27372]: purged_snaps scrub ok 2024-02-13T14:14:59.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:58 smithi087 ceph-mon[27372]: osd.7 [v2:172.21.15.148:6824/2269337484,v1:172.21.15.148:6825/2269337484] boot 2024-02-13T14:14:59.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:58 smithi087 ceph-mon[27372]: osdmap e56: 8 total, 8 up, 8 in 2024-02-13T14:14:59.243 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:58 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-13T14:14:59.243 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:58 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/3869658858' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-13T14:14:59.360 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":56,"num_osds":8,"num_up_osds":8,"osd_up_since":1707833697,"num_in_osds":8,"osd_in_since":1707833679,"num_remapped_pgs":1} 2024-02-13T14:14:59.361 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662 shell --fsid 5c04b074-ca79-11ee-95b9-87774f69a715 -- ceph osd dump --format=json 2024-02-13T14:14:59.604 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/mon.a/config 2024-02-13T14:14:59.978 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:59 smithi087 ceph-mon[39120]: osdmap e57: 8 total, 8 up, 8 in 2024-02-13T14:14:59.978 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:59 smithi087 ceph-mon[39120]: pgmap v182: 1 pgs: 1 remapped+peering; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-02-13T14:14:59.978 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:59 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:14:59.978 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:59 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:14:59.978 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:59 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-13T14:14:59.978 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:59 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-13T14:14:59.978 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:59 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-13T14:14:59.978 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:59 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-13T14:14:59.978 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:59 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:14:59.978 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:59 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:14:59.978 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:59 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:14:59.979 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:14:59 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:14:59.979 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:59 smithi087 ceph-mon[27372]: osdmap e57: 8 total, 8 up, 8 in 2024-02-13T14:14:59.979 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:59 smithi087 ceph-mon[27372]: pgmap v182: 1 pgs: 1 remapped+peering; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-02-13T14:14:59.979 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:59 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:14:59.979 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:59 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:14:59.979 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:59 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-13T14:14:59.979 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:59 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-13T14:14:59.979 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:59 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-13T14:14:59.979 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:59 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-13T14:14:59.979 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:59 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:14:59.979 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:59 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:14:59.979 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:59 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:14:59.980 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:14:59 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:15:00.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:59 smithi148 ceph-mon[31198]: osdmap e57: 8 total, 8 up, 8 in 2024-02-13T14:15:00.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:59 smithi148 ceph-mon[31198]: pgmap v182: 1 pgs: 1 remapped+peering; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-02-13T14:15:00.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:59 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:15:00.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:59 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:15:00.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:59 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-13T14:15:00.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:59 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-13T14:15:00.370 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:59 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-13T14:15:00.370 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:59 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-13T14:15:00.370 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:59 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:15:00.370 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:59 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:15:00.370 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:59 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:15:00.370 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:14:59 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:15:00.991 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:00 smithi087 ceph-mon[27372]: Detected new or changed devices on smithi148 2024-02-13T14:15:00.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:00 smithi087 ceph-mon[27372]: Adjusting osd_memory_target on smithi148 to 4272M 2024-02-13T14:15:00.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:00 smithi087 ceph-mon[27372]: osdmap e58: 8 total, 8 up, 8 in 2024-02-13T14:15:00.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:00 smithi087 ceph-mon[27372]: osdmap e59: 8 total, 8 up, 8 in 2024-02-13T14:15:00.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:00 smithi087 ceph-mon[39120]: Detected new or changed devices on smithi148 2024-02-13T14:15:00.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:00 smithi087 ceph-mon[39120]: Adjusting osd_memory_target on smithi148 to 4272M 2024-02-13T14:15:00.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:00 smithi087 ceph-mon[39120]: osdmap e58: 8 total, 8 up, 8 in 2024-02-13T14:15:00.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:00 smithi087 ceph-mon[39120]: osdmap e59: 8 total, 8 up, 8 in 2024-02-13T14:15:01.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:00 smithi148 ceph-mon[31198]: Detected new or changed devices on smithi148 2024-02-13T14:15:01.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:00 smithi148 ceph-mon[31198]: Adjusting osd_memory_target on smithi148 to 4272M 2024-02-13T14:15:01.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:00 smithi148 ceph-mon[31198]: osdmap e58: 8 total, 8 up, 8 in 2024-02-13T14:15:01.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:00 smithi148 ceph-mon[31198]: osdmap e59: 8 total, 8 up, 8 in 2024-02-13T14:15:01.531 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-02-13T14:15:01.531 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":59,"fsid":"5c04b074-ca79-11ee-95b9-87774f69a715","created":"2024-02-13T14:09:30.938170+0000","modified":"2024-02-13T14:15:00.777991+0000","last_up_change":"2024-02-13T14:14:57.861459+0000","last_in_change":"2024-02-13T14:14:39.797605+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-13T14:12:53.216571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"efef6830-b1ff-4496-a434-dfd495a55a6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":2357055010},{"type":"v1","addr":"172.21.15.87:6803","nonce":2357055010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":2357055010},{"type":"v1","addr":"172.21.15.87:6805","nonce":2357055010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":2357055010},{"type":"v1","addr":"172.21.15.87:6809","nonce":2357055010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":2357055010},{"type":"v1","addr":"172.21.15.87:6807","nonce":2357055010}]},"public_addr":"172.21.15.87:6803/2357055010","cluster_addr":"172.21.15.87:6805/2357055010","heartbeat_back_addr":"172.21.15.87:6809/2357055010","heartbeat_front_addr":"172.21.15.87:6807/2357055010","state":["exists","up"]},{"osd":1,"uuid":"399d507f-ed8f-4f3f-83c3-a3f5fdbaeefd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":1433765078},{"type":"v1","addr":"172.21.15.87:6811","nonce":1433765078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":1433765078},{"type":"v1","addr":"172.21.15.87:6813","nonce":1433765078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":1433765078},{"type":"v1","addr":"172.21.15.87:6817","nonce":1433765078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":1433765078},{"type":"v1","addr":"172.21.15.87:6815","nonce":1433765078}]},"public_addr":"172.21.15.87:6811/1433765078","cluster_addr":"172.21.15.87:6813/1433765078","heartbeat_back_addr":"172.21.15.87:6817/1433765078","heartbeat_front_addr":"172.21.15.87:6815/1433765078","state":["exists","up"]},{"osd":2,"uuid":"9517393e-da9c-46b8-90ab-9937cf20659c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":1764493594},{"type":"v1","addr":"172.21.15.87:6819","nonce":1764493594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":1764493594},{"type":"v1","addr":"172.21.15.87:6821","nonce":1764493594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1764493594},{"type":"v1","addr":"172.21.15.87:6825","nonce":1764493594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":1764493594},{"type":"v1","addr":"172.21.15.87:6823","nonce":1764493594}]},"public_addr":"172.21.15.87:6819/1764493594","cluster_addr":"172.21.15.87:6821/1764493594","heartbeat_back_addr":"172.21.15.87:6825/1764493594","heartbeat_front_addr":"172.21.15.87:6823/1764493594","state":["exists","up"]},{"osd":3,"uuid":"dfa2f275-b84b-4227-8135-e009b2cb461e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1162405572},{"type":"v1","addr":"172.21.15.87:6827","nonce":1162405572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1162405572},{"type":"v1","addr":"172.21.15.87:6829","nonce":1162405572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":1162405572},{"type":"v1","addr":"172.21.15.87:6833","nonce":1162405572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1162405572},{"type":"v1","addr":"172.21.15.87:6831","nonce":1162405572}]},"public_addr":"172.21.15.87:6827/1162405572","cluster_addr":"172.21.15.87:6829/1162405572","heartbeat_back_addr":"172.21.15.87:6833/1162405572","heartbeat_front_addr":"172.21.15.87:6831/1162405572","state":["exists","up"]},{"osd":4,"uuid":"9b07d44a-ddd1-4b31-8492-3e1459c0ad1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":1403587816},{"type":"v1","addr":"172.21.15.148:6801","nonce":1403587816}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1403587816},{"type":"v1","addr":"172.21.15.148:6803","nonce":1403587816}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1403587816},{"type":"v1","addr":"172.21.15.148:6807","nonce":1403587816}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1403587816},{"type":"v1","addr":"172.21.15.148:6805","nonce":1403587816}]},"public_addr":"172.21.15.148:6801/1403587816","cluster_addr":"172.21.15.148:6803/1403587816","heartbeat_back_addr":"172.21.15.148:6807/1403587816","heartbeat_front_addr":"172.21.15.148:6805/1403587816","state":["exists","up"]},{"osd":5,"uuid":"cba0ca7e-9575-410b-92d6-d9bb992608f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":4089539002},{"type":"v1","addr":"172.21.15.148:6809","nonce":4089539002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":4089539002},{"type":"v1","addr":"172.21.15.148:6811","nonce":4089539002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":4089539002},{"type":"v1","addr":"172.21.15.148:6815","nonce":4089539002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":4089539002},{"type":"v1","addr":"172.21.15.148:6813","nonce":4089539002}]},"public_addr":"172.21.15.148:6809/4089539002","cluster_addr":"172.21.15.148:6811/4089539002","heartbeat_back_addr":"172.21.15.148:6815/4089539002","heartbeat_front_addr":"172.21.15.148:6813/4089539002","state":["exists","up"]},{"osd":6,"uuid":"4f958ddf-d72c-434d-bdfc-fb039719e39b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":1791408651},{"type":"v1","addr":"172.21.15.148:6817","nonce":1791408651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":1791408651},{"type":"v1","addr":"172.21.15.148:6819","nonce":1791408651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":1791408651},{"type":"v1","addr":"172.21.15.148:6823","nonce":1791408651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":1791408651},{"type":"v1","addr":"172.21.15.148:6821","nonce":1791408651}]},"public_addr":"172.21.15.148:6817/1791408651","cluster_addr":"172.21.15.148:6819/1791408651","heartbeat_back_addr":"172.21.15.148:6823/1791408651","heartbeat_front_addr":"172.21.15.148:6821/1791408651","state":["exists","up"]},{"osd":7,"uuid":"528011a9-d721-42e5-ba03-d44ef41d1bf1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":2269337484},{"type":"v1","addr":"172.21.15.148:6825","nonce":2269337484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2269337484},{"type":"v1","addr":"172.21.15.148:6827","nonce":2269337484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":2269337484},{"type":"v1","addr":"172.21.15.148:6831","nonce":2269337484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":2269337484},{"type":"v1","addr":"172.21.15.148:6829","nonce":2269337484}]},"public_addr":"172.21.15.148:6825/2269337484","cluster_addr":"172.21.15.148:6827/2269337484","heartbeat_back_addr":"172.21.15.148:6831/2269337484","heartbeat_front_addr":"172.21.15.148:6829/2269337484","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-13T14:11:55.754942+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-13T14:12:22.510245+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-13T14:12:49.085565+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-13T14:13:15.484384+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-13T14:13:38.935614+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-13T14:14:04.066142+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-13T14:14:30.071935+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-13T14:14:55.918850+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.87:0/3893540759":"2024-02-14T14:10:28.875829+0000","172.21.15.87:0/179577207":"2024-02-14T14:10:01.886668+0000","172.21.15.87:0/3243120218":"2024-02-14T14:10:28.875829+0000","172.21.15.87:0/1580943540":"2024-02-14T14:10:01.886668+0000","172.21.15.87:6800/2307544135":"2024-02-14T14:10:28.875829+0000","172.21.15.87:6800/1895635638":"2024-02-14T14:10:01.886668+0000","172.21.15.87:0/2387914205":"2024-02-14T14:10:28.875829+0000","172.21.15.87:6801/2307544135":"2024-02-14T14:10:28.875829+0000","172.21.15.87:0/1196696660":"2024-02-14T14:10:01.886668+0000","172.21.15.87:6801/1895635638":"2024-02-14T14:10:01.886668+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-13T14:15:02.140 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:01 smithi087 ceph-mon[27372]: pgmap v185: 1 pgs: 1 remapped+peering; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-02-13T14:15:02.140 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:01 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/1233994016' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-13T14:15:02.140 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:01 smithi087 ceph-mon[27372]: osdmap e60: 8 total, 8 up, 8 in 2024-02-13T14:15:02.140 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:01 smithi087 ceph-mon[39120]: pgmap v185: 1 pgs: 1 remapped+peering; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-02-13T14:15:02.141 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:01 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/1233994016' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-13T14:15:02.141 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:01 smithi087 ceph-mon[39120]: osdmap e60: 8 total, 8 up, 8 in 2024-02-13T14:15:02.168 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-13T14:12:53.216571+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 7.889999866485596, 'score_stable': 7.889999866485596, 'optimal_score': 0.3799999952316284, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-13T14:15:02.169 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662 shell --fsid 5c04b074-ca79-11ee-95b9-87774f69a715 -- ceph osd pool get .mgr pg_num 2024-02-13T14:15:02.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:01 smithi148 ceph-mon[31198]: pgmap v185: 1 pgs: 1 remapped+peering; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-02-13T14:15:02.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:01 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/1233994016' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-13T14:15:02.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:01 smithi148 ceph-mon[31198]: osdmap e60: 8 total, 8 up, 8 in 2024-02-13T14:15:02.405 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/mon.a/config 2024-02-13T14:15:04.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:03 smithi087 ceph-mon[27372]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 142 KiB/s, 0 objects/s recovering 2024-02-13T14:15:04.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:03 smithi087 ceph-mon[39120]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 142 KiB/s, 0 objects/s recovering 2024-02-13T14:15:04.334 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 1 2024-02-13T14:15:04.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:03 smithi148 ceph-mon[31198]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 715 GiB / 715 GiB avail; 142 KiB/s, 0 objects/s recovering 2024-02-13T14:15:05.091 INFO:tasks.cephadm:Adding ceph.rgw.foo.a on smithi087 2024-02-13T14:15:05.092 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5c04b074-ca79-11ee-95b9-87774f69a715 -- ceph orch apply rgw foo.a --placement '1;smithi087=foo.a' 2024-02-13T14:15:05.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:04 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/1660280717' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-13T14:15:05.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:04 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/1660280717' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-13T14:15:05.344 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/mon.b/config 2024-02-13T14:15:05.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:04 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/1660280717' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-13T14:15:06.118 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:05 smithi148 ceph-mon[31198]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-02-13T14:15:06.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:05 smithi087 ceph-mon[27372]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-02-13T14:15:06.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:05 smithi087 ceph-mon[39120]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-02-13T14:15:07.138 INFO:teuthology.orchestra.run.smithi148.stdout:Scheduled rgw.foo.a update... 2024-02-13T14:15:07.803 DEBUG:teuthology.orchestra.run.smithi087:rgw.foo.a> sudo journalctl -f -n 0 -u ceph-5c04b074-ca79-11ee-95b9-87774f69a715@rgw.foo.a.service 2024-02-13T14:15:07.806 INFO:tasks.cephadm:Adding ceph.iscsi.iscsi.a on smithi148 2024-02-13T14:15:07.806 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5c04b074-ca79-11ee-95b9-87774f69a715 -- ceph osd pool create datapool 3 3 replicated 2024-02-13T14:15:08.045 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/mon.b/config 2024-02-13T14:15:08.402 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:08 smithi148 ceph-mon[31198]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-02-13T14:15:08.402 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:08 smithi148 ceph-mon[31198]: from='client.24310 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi087=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T14:15:08.402 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:08 smithi148 ceph-mon[31198]: Saving service rgw.foo.a spec with placement smithi087=foo.a;count:1 2024-02-13T14:15:08.402 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:08 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:15:08.402 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:08 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:15:08.402 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:08 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:15:08.402 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:08 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:15:08.403 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:08 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:15:08.403 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:08 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-02-13T14:15:08.403 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:08 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-02-13T14:15:08.403 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:08 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:15:08.403 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:08 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:15:08.432 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:08 smithi087 ceph-mon[27372]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-02-13T14:15:08.432 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:08 smithi087 ceph-mon[27372]: from='client.24310 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi087=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T14:15:08.432 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:08 smithi087 ceph-mon[27372]: Saving service rgw.foo.a spec with placement smithi087=foo.a;count:1 2024-02-13T14:15:08.432 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:08 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:15:08.432 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:08 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:15:08.432 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:08 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:15:08.433 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:08 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:15:08.433 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:08 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:15:08.433 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:08 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-02-13T14:15:08.433 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:08 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-02-13T14:15:08.433 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:08 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:15:08.433 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:08 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:15:08.433 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:08 smithi087 ceph-mon[39120]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-02-13T14:15:08.433 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:08 smithi087 ceph-mon[39120]: from='client.24310 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi087=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T14:15:08.433 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:08 smithi087 ceph-mon[39120]: Saving service rgw.foo.a spec with placement smithi087=foo.a;count:1 2024-02-13T14:15:08.433 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:08 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:15:08.433 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:08 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:15:08.433 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:08 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:15:08.434 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:08 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:15:08.434 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:08 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:15:08.434 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:08 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-02-13T14:15:08.434 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:08 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-02-13T14:15:08.434 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:08 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:15:08.434 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:08 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:15:09.458 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:09 smithi148 ceph-mon[31198]: Deploying daemon rgw.foo.a on smithi087 2024-02-13T14:15:09.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:09 smithi087 ceph-mon[27372]: Deploying daemon rgw.foo.a on smithi087 2024-02-13T14:15:09.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:09 smithi087 ceph-mon[39120]: Deploying daemon rgw.foo.a on smithi087 2024-02-13T14:15:10.148 INFO:journalctl@ceph.rgw.foo.a.smithi087.stdout:Feb 13 14:15:09 smithi087 systemd[1]: Starting Ceph rgw.foo.a for 5c04b074-ca79-11ee-95b9-87774f69a715... 2024-02-13T14:15:10.157 INFO:teuthology.orchestra.run.smithi148.stderr:pool 'datapool' created 2024-02-13T14:15:10.467 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:10 smithi087 ceph-mon[27372]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-02-13T14:15:10.467 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:10 smithi087 ceph-mon[27372]: from='client.? 172.21.15.148:0/223968882' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-02-13T14:15:10.467 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:10 smithi087 ceph-mon[27372]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-02-13T14:15:10.468 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:10 smithi087 ceph-mon[39120]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-02-13T14:15:10.468 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:10 smithi087 ceph-mon[39120]: from='client.? 172.21.15.148:0/223968882' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-02-13T14:15:10.468 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:10 smithi087 ceph-mon[39120]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-02-13T14:15:10.468 INFO:journalctl@ceph.rgw.foo.a.smithi087.stdout:Feb 13 14:15:10 smithi087 podman[67175]: 2024-02-13 14:15:10.11022377 +0000 UTC m=+0.017936811 image pull 28480620e5edd298d55c0e6673dcbad9ed23edd37269028a11063c1b8925676a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f 2024-02-13T14:15:10.468 INFO:journalctl@ceph.rgw.foo.a.smithi087.stdout:Feb 13 14:15:10 smithi087 podman[67175]: 2024-02-13 14:15:10.23082308 +0000 UTC m=+0.138536121 container create 77dce37a552b69f34de726522f2be80384bea6754b5d4d82a38a3bb53b6d2ee8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-rgw-foo-a, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, io.buildah.version=1.33.2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, org.label-schema.build-date=20240131, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-02-13T14:15:10.468 INFO:journalctl@ceph.rgw.foo.a.smithi087.stdout:Feb 13 14:15:10 smithi087 podman[67175]: 2024-02-13 14:15:10.466618511 +0000 UTC m=+0.374331551 container init 77dce37a552b69f34de726522f2be80384bea6754b5d4d82a38a3bb53b6d2ee8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-rgw-foo-a, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240131, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5) 2024-02-13T14:15:10.469 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:10 smithi148 ceph-mon[31198]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-02-13T14:15:10.469 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:10 smithi148 ceph-mon[31198]: from='client.? 172.21.15.148:0/223968882' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-02-13T14:15:10.469 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:10 smithi148 ceph-mon[31198]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-02-13T14:15:10.741 INFO:journalctl@ceph.rgw.foo.a.smithi087.stdout:Feb 13 14:15:10 smithi087 podman[67175]: 2024-02-13 14:15:10.472297734 +0000 UTC m=+0.380010783 container start 77dce37a552b69f34de726522f2be80384bea6754b5d4d82a38a3bb53b6d2ee8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-rgw-foo-a, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, ceph=True, io.buildah.version=1.33.2, GIT_CLEAN=True, org.label-schema.build-date=20240131, maintainer=Guillaume Abrioux , RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5) 2024-02-13T14:15:10.742 INFO:journalctl@ceph.rgw.foo.a.smithi087.stdout:Feb 13 14:15:10 smithi087 bash[67175]: 77dce37a552b69f34de726522f2be80384bea6754b5d4d82a38a3bb53b6d2ee8 2024-02-13T14:15:10.742 INFO:journalctl@ceph.rgw.foo.a.smithi087.stdout:Feb 13 14:15:10 smithi087 systemd[1]: Started Ceph rgw.foo.a for 5c04b074-ca79-11ee-95b9-87774f69a715. 2024-02-13T14:15:10.781 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5c04b074-ca79-11ee-95b9-87774f69a715 -- rbd pool init datapool 2024-02-13T14:15:11.025 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/mon.b/config 2024-02-13T14:15:11.482 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:11 smithi148 ceph-mon[31198]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-02-13T14:15:11.482 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:11 smithi148 ceph-mon[31198]: osdmap e61: 8 total, 8 up, 8 in 2024-02-13T14:15:11.482 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:11 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:15:11.482 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:11 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:15:11.482 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:11 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:15:11.482 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:11 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:15:11.482 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:11 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:15:11.482 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:11 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:15:11.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:11 smithi087 ceph-mon[27372]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-02-13T14:15:11.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:11 smithi087 ceph-mon[27372]: osdmap e61: 8 total, 8 up, 8 in 2024-02-13T14:15:11.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:11 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:15:11.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:11 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:15:11.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:11 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:15:11.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:11 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:15:11.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:11 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:15:11.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:11 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:15:11.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:11 smithi087 ceph-mon[39120]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-02-13T14:15:11.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:11 smithi087 ceph-mon[39120]: osdmap e61: 8 total, 8 up, 8 in 2024-02-13T14:15:11.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:11 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:15:11.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:11 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:15:11.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:11 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:15:11.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:11 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:15:11.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:11 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:15:11.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:11 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:15:12.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:12 smithi087 ceph-mon[27372]: Saving service rgw.foo.a spec with placement smithi087=foo.a;count:1 2024-02-13T14:15:12.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:12 smithi087 ceph-mon[27372]: pgmap v192: 4 pgs: 1 creating+peering, 1 active+clean, 2 unknown; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-02-13T14:15:12.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:12 smithi087 ceph-mon[27372]: osdmap e62: 8 total, 8 up, 8 in 2024-02-13T14:15:12.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:12 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/3485939692' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-02-13T14:15:12.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:12 smithi087 ceph-mon[27372]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T14:15:12.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:12 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/3485939692' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-02-13T14:15:12.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:12 smithi087 ceph-mon[27372]: osdmap e63: 8 total, 8 up, 8 in 2024-02-13T14:15:12.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:12 smithi087 ceph-mon[39120]: Saving service rgw.foo.a spec with placement smithi087=foo.a;count:1 2024-02-13T14:15:12.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:12 smithi087 ceph-mon[39120]: pgmap v192: 4 pgs: 1 creating+peering, 1 active+clean, 2 unknown; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-02-13T14:15:12.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:12 smithi087 ceph-mon[39120]: osdmap e62: 8 total, 8 up, 8 in 2024-02-13T14:15:12.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:12 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/3485939692' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-02-13T14:15:12.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:12 smithi087 ceph-mon[39120]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T14:15:12.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:12 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/3485939692' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-02-13T14:15:12.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:12 smithi087 ceph-mon[39120]: osdmap e63: 8 total, 8 up, 8 in 2024-02-13T14:15:12.513 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:12 smithi148 ceph-mon[31198]: Saving service rgw.foo.a spec with placement smithi087=foo.a;count:1 2024-02-13T14:15:12.513 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:12 smithi148 ceph-mon[31198]: pgmap v192: 4 pgs: 1 creating+peering, 1 active+clean, 2 unknown; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-02-13T14:15:12.513 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:12 smithi148 ceph-mon[31198]: osdmap e62: 8 total, 8 up, 8 in 2024-02-13T14:15:12.513 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:12 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/3485939692' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-02-13T14:15:12.513 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:12 smithi148 ceph-mon[31198]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T14:15:12.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:12 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/3485939692' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-02-13T14:15:12.514 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:12 smithi148 ceph-mon[31198]: osdmap e63: 8 total, 8 up, 8 in 2024-02-13T14:15:13.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:13 smithi087 ceph-mon[27372]: from='client.? 172.21.15.148:0/3577089937' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-02-13T14:15:13.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:13 smithi087 ceph-mon[27372]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-02-13T14:15:13.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:13 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:15:13.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:13 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:15:13.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:13 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:15:13.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:13 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:15:13.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:13 smithi087 ceph-mon[27372]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-02-13T14:15:13.493 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:13 smithi087 ceph-mon[27372]: osdmap e64: 8 total, 8 up, 8 in 2024-02-13T14:15:13.493 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:13 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/46388278' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-13T14:15:13.493 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:13 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/577345735' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-13T14:15:13.493 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:13 smithi087 ceph-mon[27372]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-13T14:15:13.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:13 smithi087 ceph-mon[39120]: from='client.? 172.21.15.148:0/3577089937' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-02-13T14:15:13.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:13 smithi087 ceph-mon[39120]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-02-13T14:15:13.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:13 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:15:13.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:13 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:15:13.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:13 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:15:13.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:13 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:15:13.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:13 smithi087 ceph-mon[39120]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-02-13T14:15:13.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:13 smithi087 ceph-mon[39120]: osdmap e64: 8 total, 8 up, 8 in 2024-02-13T14:15:13.494 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:13 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/46388278' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-13T14:15:13.494 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:13 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/577345735' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-13T14:15:13.494 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:13 smithi087 ceph-mon[39120]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-13T14:15:13.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:13 smithi148 ceph-mon[31198]: from='client.? 172.21.15.148:0/3577089937' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-02-13T14:15:13.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:13 smithi148 ceph-mon[31198]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-02-13T14:15:13.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:13 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:15:13.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:13 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:15:13.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:13 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:15:13.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:13 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:15:13.620 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:13 smithi148 ceph-mon[31198]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-02-13T14:15:13.620 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:13 smithi148 ceph-mon[31198]: osdmap e64: 8 total, 8 up, 8 in 2024-02-13T14:15:13.620 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:13 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/46388278' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-13T14:15:13.620 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:13 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/577345735' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-13T14:15:13.620 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:13 smithi148 ceph-mon[31198]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-13T14:15:14.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:14 smithi087 ceph-mon[27372]: Checking dashboard <-> RGW credentials 2024-02-13T14:15:14.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:14 smithi087 ceph-mon[27372]: pgmap v195: 36 pgs: 1 creating+peering, 11 active+clean, 24 unknown; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-13T14:15:14.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:14 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:15:14.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:14 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/46388278' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-02-13T14:15:14.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:14 smithi087 ceph-mon[27372]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-02-13T14:15:14.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:14 smithi087 ceph-mon[27372]: osdmap e65: 8 total, 8 up, 8 in 2024-02-13T14:15:14.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:14 smithi087 ceph-mon[39120]: Checking dashboard <-> RGW credentials 2024-02-13T14:15:14.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:14 smithi087 ceph-mon[39120]: pgmap v195: 36 pgs: 1 creating+peering, 11 active+clean, 24 unknown; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-13T14:15:14.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:14 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:15:14.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:14 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/46388278' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-02-13T14:15:14.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:14 smithi087 ceph-mon[39120]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-02-13T14:15:14.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:14 smithi087 ceph-mon[39120]: osdmap e65: 8 total, 8 up, 8 in 2024-02-13T14:15:14.618 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:14 smithi148 ceph-mon[31198]: Checking dashboard <-> RGW credentials 2024-02-13T14:15:14.618 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:14 smithi148 ceph-mon[31198]: pgmap v195: 36 pgs: 1 creating+peering, 11 active+clean, 24 unknown; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-13T14:15:14.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:14 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:15:14.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:14 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/46388278' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-02-13T14:15:14.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:14 smithi148 ceph-mon[31198]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-02-13T14:15:14.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:14 smithi148 ceph-mon[31198]: osdmap e65: 8 total, 8 up, 8 in 2024-02-13T14:15:15.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:15 smithi087 ceph-mon[27372]: pgmap v198: 68 pgs: 6 creating+peering, 23 active+clean, 39 unknown; 578 KiB data, 222 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.2 KiB/s wr, 4 op/s 2024-02-13T14:15:15.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:15 smithi087 ceph-mon[27372]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-02-13T14:15:15.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:15 smithi087 ceph-mon[27372]: Cluster is now healthy 2024-02-13T14:15:15.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:15 smithi087 ceph-mon[27372]: osdmap e66: 8 total, 8 up, 8 in 2024-02-13T14:15:15.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:15 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/577345735' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-13T14:15:15.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:15 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/46388278' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-13T14:15:15.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:15 smithi087 ceph-mon[27372]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-13T14:15:15.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:15 smithi087 ceph-mon[39120]: pgmap v198: 68 pgs: 6 creating+peering, 23 active+clean, 39 unknown; 578 KiB data, 222 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.2 KiB/s wr, 4 op/s 2024-02-13T14:15:15.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:15 smithi087 ceph-mon[39120]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-02-13T14:15:15.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:15 smithi087 ceph-mon[39120]: Cluster is now healthy 2024-02-13T14:15:15.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:15 smithi087 ceph-mon[39120]: osdmap e66: 8 total, 8 up, 8 in 2024-02-13T14:15:15.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:15 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/577345735' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-13T14:15:15.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:15 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/46388278' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-13T14:15:15.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:15 smithi087 ceph-mon[39120]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-13T14:15:15.618 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:15 smithi148 ceph-mon[31198]: pgmap v198: 68 pgs: 6 creating+peering, 23 active+clean, 39 unknown; 578 KiB data, 222 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.2 KiB/s wr, 4 op/s 2024-02-13T14:15:15.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:15 smithi148 ceph-mon[31198]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-02-13T14:15:15.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:15 smithi148 ceph-mon[31198]: Cluster is now healthy 2024-02-13T14:15:15.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:15 smithi148 ceph-mon[31198]: osdmap e66: 8 total, 8 up, 8 in 2024-02-13T14:15:15.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:15 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/577345735' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-13T14:15:15.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:15 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/46388278' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-13T14:15:15.620 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:15 smithi148 ceph-mon[31198]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-13T14:15:15.882 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5c04b074-ca79-11ee-95b9-87774f69a715 -- ceph orch apply iscsi datapool admin admin --trusted_ip_list 172.21.15.148 --placement '1;smithi148=iscsi.a' 2024-02-13T14:15:16.127 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/mon.b/config 2024-02-13T14:15:17.306 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:17 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/46388278' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-02-13T14:15:17.306 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:17 smithi148 ceph-mon[31198]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-02-13T14:15:17.306 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:17 smithi148 ceph-mon[31198]: osdmap e67: 8 total, 8 up, 8 in 2024-02-13T14:15:17.306 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:17 smithi148 ceph-mon[31198]: pgmap v201: 100 pgs: 14 creating+peering, 52 active+clean, 34 unknown; 578 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-13T14:15:17.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:17 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/46388278' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-02-13T14:15:17.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:17 smithi087 ceph-mon[27372]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-02-13T14:15:17.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:17 smithi087 ceph-mon[27372]: osdmap e67: 8 total, 8 up, 8 in 2024-02-13T14:15:17.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:17 smithi087 ceph-mon[27372]: pgmap v201: 100 pgs: 14 creating+peering, 52 active+clean, 34 unknown; 578 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-13T14:15:17.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:17 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/46388278' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-02-13T14:15:17.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:17 smithi087 ceph-mon[39120]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-02-13T14:15:17.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:17 smithi087 ceph-mon[39120]: osdmap e67: 8 total, 8 up, 8 in 2024-02-13T14:15:17.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:17 smithi087 ceph-mon[39120]: pgmap v201: 100 pgs: 14 creating+peering, 52 active+clean, 34 unknown; 578 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-13T14:15:18.071 INFO:teuthology.orchestra.run.smithi148.stdout:Scheduled iscsi.datapool update... 2024-02-13T14:15:18.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:18 smithi148 ceph-mon[31198]: osdmap e68: 8 total, 8 up, 8 in 2024-02-13T14:15:18.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:18 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/577345735' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-13T14:15:18.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:18 smithi148 ceph-mon[31198]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-13T14:15:18.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:18 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/46388278' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-13T14:15:18.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:18 smithi148 ceph-mon[31198]: from='client.24364 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.148", "placement": "1;smithi148=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T14:15:18.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:18 smithi148 ceph-mon[31198]: Saving service iscsi.datapool spec with placement smithi148=iscsi.a;count:1 2024-02-13T14:15:18.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:18 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:15:18.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:18 smithi087 ceph-mon[27372]: osdmap e68: 8 total, 8 up, 8 in 2024-02-13T14:15:18.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:18 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/577345735' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-13T14:15:18.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:18 smithi087 ceph-mon[27372]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-13T14:15:18.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:18 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/46388278' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-13T14:15:18.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:18 smithi087 ceph-mon[27372]: from='client.24364 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.148", "placement": "1;smithi148=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T14:15:18.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:18 smithi087 ceph-mon[27372]: Saving service iscsi.datapool spec with placement smithi148=iscsi.a;count:1 2024-02-13T14:15:18.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:18 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:15:18.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:18 smithi087 ceph-mon[39120]: osdmap e68: 8 total, 8 up, 8 in 2024-02-13T14:15:18.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:18 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/577345735' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-13T14:15:18.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:18 smithi087 ceph-mon[39120]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-13T14:15:18.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:18 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/46388278' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-13T14:15:18.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:18 smithi087 ceph-mon[39120]: from='client.24364 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.148", "placement": "1;smithi148=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T14:15:18.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:18 smithi087 ceph-mon[39120]: Saving service iscsi.datapool spec with placement smithi148=iscsi.a;count:1 2024-02-13T14:15:18.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:18 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:15:18.698 INFO:tasks.cephadm:Distributing iscsi-gateway.cfg... 2024-02-13T14:15:18.699 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-02-13T14:15:18.699 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd of=/etc/ceph/iscsi-gateway.cfg 2024-02-13T14:15:18.734 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-02-13T14:15:18.734 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/etc/ceph/iscsi-gateway.cfg 2024-02-13T14:15:18.764 DEBUG:teuthology.orchestra.run.smithi148:iscsi.iscsi.a> sudo journalctl -f -n 0 -u ceph-5c04b074-ca79-11ee-95b9-87774f69a715@iscsi.iscsi.a.service 2024-02-13T14:15:18.807 INFO:tasks.cephadm:Adding prometheus.a on smithi148 2024-02-13T14:15:18.807 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5c04b074-ca79-11ee-95b9-87774f69a715 -- ceph orch apply prometheus '1;smithi148=a' 2024-02-13T14:15:19.084 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/mon.b/config 2024-02-13T14:15:19.475 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:19 smithi148 ceph-mon[31198]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-02-13T14:15:19.475 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:19 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/46388278' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-02-13T14:15:19.475 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:19 smithi148 ceph-mon[31198]: osdmap e69: 8 total, 8 up, 8 in 2024-02-13T14:15:19.475 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:19 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/46388278' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-13T14:15:19.475 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:19 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/577345735' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-13T14:15:19.475 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:19 smithi148 ceph-mon[31198]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-13T14:15:19.475 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:19 smithi148 ceph-mon[31198]: pgmap v204: 132 pgs: 22 creating+peering, 92 active+clean, 18 unknown; 578 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1023 B/s wr, 3 op/s 2024-02-13T14:15:19.475 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:19 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/46388278' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-02-13T14:15:19.475 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:19 smithi148 ceph-mon[31198]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-02-13T14:15:19.475 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:19 smithi148 ceph-mon[31198]: osdmap e70: 8 total, 8 up, 8 in 2024-02-13T14:15:19.491 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:19 smithi087 ceph-mon[39120]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-02-13T14:15:19.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:19 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/46388278' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-02-13T14:15:19.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:19 smithi087 ceph-mon[39120]: osdmap e69: 8 total, 8 up, 8 in 2024-02-13T14:15:19.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:19 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/46388278' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-13T14:15:19.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:19 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/577345735' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-13T14:15:19.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:19 smithi087 ceph-mon[39120]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-13T14:15:19.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:19 smithi087 ceph-mon[39120]: pgmap v204: 132 pgs: 22 creating+peering, 92 active+clean, 18 unknown; 578 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1023 B/s wr, 3 op/s 2024-02-13T14:15:19.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:19 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/46388278' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-02-13T14:15:19.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:19 smithi087 ceph-mon[39120]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-02-13T14:15:19.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:19 smithi087 ceph-mon[39120]: osdmap e70: 8 total, 8 up, 8 in 2024-02-13T14:15:19.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:19 smithi087 ceph-mon[27372]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-02-13T14:15:19.493 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:19 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/46388278' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-02-13T14:15:19.493 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:19 smithi087 ceph-mon[27372]: osdmap e69: 8 total, 8 up, 8 in 2024-02-13T14:15:19.493 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:19 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/46388278' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-13T14:15:19.493 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:19 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/577345735' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-13T14:15:19.493 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:19 smithi087 ceph-mon[27372]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-13T14:15:19.493 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:19 smithi087 ceph-mon[27372]: pgmap v204: 132 pgs: 22 creating+peering, 92 active+clean, 18 unknown; 578 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1023 B/s wr, 3 op/s 2024-02-13T14:15:19.493 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:19 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/46388278' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-02-13T14:15:19.493 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:19 smithi087 ceph-mon[27372]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-02-13T14:15:19.493 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:19 smithi087 ceph-mon[27372]: osdmap e70: 8 total, 8 up, 8 in 2024-02-13T14:15:20.496 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:20 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:15:20.496 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:20 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:15:20.497 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:20 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:15:20.497 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:20 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-02-13T14:15:20.497 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:20 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-02-13T14:15:20.497 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:20 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:15:20.497 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:20 smithi148 ceph-mon[31198]: Deploying daemon iscsi.iscsi.a on smithi148 2024-02-13T14:15:20.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:20 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:15:20.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:20 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:15:20.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:20 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:15:20.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:20 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-02-13T14:15:20.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:20 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-02-13T14:15:20.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:20 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:15:20.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:20 smithi087 ceph-mon[39120]: Deploying daemon iscsi.iscsi.a on smithi148 2024-02-13T14:15:20.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:20 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:15:20.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:20 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:15:20.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:20 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:15:20.743 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:20 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-02-13T14:15:20.743 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:20 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-02-13T14:15:20.743 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:20 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:15:20.743 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:20 smithi087 ceph-mon[27372]: Deploying daemon iscsi.iscsi.a on smithi148 2024-02-13T14:15:21.035 INFO:teuthology.orchestra.run.smithi148.stdout:Scheduled prometheus update... 2024-02-13T14:15:22.018 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:21 smithi087 ceph-mon[27372]: osdmap e71: 8 total, 8 up, 8 in 2024-02-13T14:15:22.019 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:21 smithi087 ceph-mon[27372]: pgmap v207: 132 pgs: 13 creating+peering, 117 active+clean, 2 unknown; 580 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 59 KiB/s rd, 5.2 KiB/s wr, 143 op/s 2024-02-13T14:15:22.019 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:21 smithi087 ceph-mon[27372]: from='client.24385 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi148=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T14:15:22.019 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:21 smithi087 ceph-mon[27372]: Saving service prometheus spec with placement smithi148=a;count:1 2024-02-13T14:15:22.019 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:21 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:15:22.019 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:21 smithi087 ceph-mon[39120]: osdmap e71: 8 total, 8 up, 8 in 2024-02-13T14:15:22.019 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:21 smithi087 ceph-mon[39120]: pgmap v207: 132 pgs: 13 creating+peering, 117 active+clean, 2 unknown; 580 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 59 KiB/s rd, 5.2 KiB/s wr, 143 op/s 2024-02-13T14:15:22.019 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:21 smithi087 ceph-mon[39120]: from='client.24385 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi148=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T14:15:22.019 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:21 smithi087 ceph-mon[39120]: Saving service prometheus spec with placement smithi148=a;count:1 2024-02-13T14:15:22.019 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:21 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:15:22.118 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:21 smithi148 ceph-mon[31198]: osdmap e71: 8 total, 8 up, 8 in 2024-02-13T14:15:22.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:21 smithi148 ceph-mon[31198]: pgmap v207: 132 pgs: 13 creating+peering, 117 active+clean, 2 unknown; 580 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 59 KiB/s rd, 5.2 KiB/s wr, 143 op/s 2024-02-13T14:15:22.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:21 smithi148 ceph-mon[31198]: from='client.24385 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi148=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T14:15:22.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:21 smithi148 ceph-mon[31198]: Saving service prometheus spec with placement smithi148=a;count:1 2024-02-13T14:15:22.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:21 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:15:22.778 DEBUG:teuthology.orchestra.run.smithi148:prometheus.a> sudo journalctl -f -n 0 -u ceph-5c04b074-ca79-11ee-95b9-87774f69a715@prometheus.a.service 2024-02-13T14:15:22.781 INFO:tasks.cephadm:Adding node-exporter.a on smithi087 2024-02-13T14:15:22.781 INFO:tasks.cephadm:Adding node-exporter.b on smithi148 2024-02-13T14:15:22.781 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5c04b074-ca79-11ee-95b9-87774f69a715 -- ceph orch apply node-exporter '2;smithi087=a;smithi148=b' 2024-02-13T14:15:23.119 INFO:journalctl@ceph.iscsi.iscsi.a.smithi148.stdout:Feb 13 14:15:23 smithi148 podman[59618]: 2024-02-13 14:15:23.092934946 +0000 UTC m=+0.137043700 container create fb74c3c1636a61cb3b30bc126b6e04cf9a478cad3ab539e2c543e2e32f4f5a8c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-iscsi-iscsi-a, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, org.label-schema.vendor=CentOS, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240131) 2024-02-13T14:15:23.502 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/mon.b/config 2024-02-13T14:15:23.606 INFO:journalctl@ceph.iscsi.iscsi.a.smithi148.stdout:Feb 13 14:15:23 smithi148 podman[59618]: 2024-02-13 14:15:23.320627158 +0000 UTC m=+0.364735925 container init fb74c3c1636a61cb3b30bc126b6e04cf9a478cad3ab539e2c543e2e32f4f5a8c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-iscsi-iscsi-a, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240131, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-02-13T14:15:23.606 INFO:journalctl@ceph.iscsi.iscsi.a.smithi148.stdout:Feb 13 14:15:23 smithi148 podman[59618]: 2024-02-13 14:15:23.326361308 +0000 UTC m=+0.370470063 container start fb74c3c1636a61cb3b30bc126b6e04cf9a478cad3ab539e2c543e2e32f4f5a8c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-iscsi-iscsi-a, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.build-date=20240131, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.2) 2024-02-13T14:15:23.606 INFO:journalctl@ceph.iscsi.iscsi.a.smithi148.stdout:Feb 13 14:15:23 smithi148 bash[59618]: fb74c3c1636a61cb3b30bc126b6e04cf9a478cad3ab539e2c543e2e32f4f5a8c 2024-02-13T14:15:23.606 INFO:journalctl@ceph.iscsi.iscsi.a.smithi148.stdout:Feb 13 14:15:23 smithi148 systemd[1]: Started Ceph iscsi.iscsi.a for 5c04b074-ca79-11ee-95b9-87774f69a715. 2024-02-13T14:15:23.858 INFO:journalctl@ceph.iscsi.iscsi.a.smithi148.stdout:Feb 13 14:15:23 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-iscsi-iscsi-a[59644]: debug Started the configuration object watcher 2024-02-13T14:15:23.858 INFO:journalctl@ceph.iscsi.iscsi.a.smithi148.stdout:Feb 13 14:15:23 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-iscsi-iscsi-a[59644]: debug Checking for config object changes every 1s 2024-02-13T14:15:23.858 INFO:journalctl@ceph.iscsi.iscsi.a.smithi148.stdout:Feb 13 14:15:23 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-iscsi-iscsi-a[59644]: debug Processing osd blocklist entries for this node 2024-02-13T14:15:24.118 INFO:journalctl@ceph.iscsi.iscsi.a.smithi148.stdout:Feb 13 14:15:23 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-iscsi-iscsi-a[59644]: debug Reading the configuration object to update local LIO configuration 2024-02-13T14:15:24.119 INFO:journalctl@ceph.iscsi.iscsi.a.smithi148.stdout:Feb 13 14:15:23 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-iscsi-iscsi-a[59644]: debug Configuration does not have an entry for this host(smithi148.front.sepia.ceph.com) - nothing to define to LIO 2024-02-13T14:15:24.119 INFO:journalctl@ceph.iscsi.iscsi.a.smithi148.stdout:Feb 13 14:15:23 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-iscsi-iscsi-a[59644]: debug * Running on http://[::]:5000/ (Press CTRL+C to quit) 2024-02-13T14:15:24.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:23 smithi148 ceph-mon[31198]: pgmap v208: 132 pgs: 7 creating+peering, 125 active+clean; 581 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 54 KiB/s rd, 4.4 KiB/s wr, 129 op/s 2024-02-13T14:15:24.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:23 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:15:24.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:23 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:15:24.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:23 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:15:24.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:23 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:15:24.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:23 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:15:24.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:23 smithi148 ceph-mon[31198]: from='client.? 172.21.15.148:0/177478359' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-02-13T14:15:24.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:23 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:15:24.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:23 smithi087 ceph-mon[27372]: pgmap v208: 132 pgs: 7 creating+peering, 125 active+clean; 581 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 54 KiB/s rd, 4.4 KiB/s wr, 129 op/s 2024-02-13T14:15:24.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:23 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:15:24.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:23 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:15:24.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:23 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:15:24.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:23 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:15:24.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:23 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:15:24.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:23 smithi087 ceph-mon[27372]: from='client.? 172.21.15.148:0/177478359' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-02-13T14:15:24.243 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:23 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:15:24.243 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:23 smithi087 ceph-mon[39120]: pgmap v208: 132 pgs: 7 creating+peering, 125 active+clean; 581 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 54 KiB/s rd, 4.4 KiB/s wr, 129 op/s 2024-02-13T14:15:24.243 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:23 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:15:24.243 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:23 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:15:24.243 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:23 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:15:24.243 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:23 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:15:24.243 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:23 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:15:24.243 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:23 smithi087 ceph-mon[39120]: from='client.? 172.21.15.148:0/177478359' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-02-13T14:15:24.244 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:23 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:15:25.118 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:24 smithi148 ceph-mon[31198]: Checking pool "datapool" exists for service iscsi.datapool 2024-02-13T14:15:25.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:24 smithi087 ceph-mon[27372]: Checking pool "datapool" exists for service iscsi.datapool 2024-02-13T14:15:25.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:24 smithi087 ceph-mon[39120]: Checking pool "datapool" exists for service iscsi.datapool 2024-02-13T14:15:25.990 INFO:teuthology.orchestra.run.smithi148.stdout:Scheduled node-exporter update... 2024-02-13T14:15:26.025 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:25 smithi148 ceph-mon[31198]: pgmap v209: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 69 KiB/s rd, 5.2 KiB/s wr, 166 op/s 2024-02-13T14:15:26.025 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:25 smithi148 ceph-mon[31198]: mgrmap e16: y(active, since 4m), standbys: x 2024-02-13T14:15:26.025 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:25 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:15:26.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:25 smithi087 ceph-mon[27372]: pgmap v209: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 69 KiB/s rd, 5.2 KiB/s wr, 166 op/s 2024-02-13T14:15:26.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:25 smithi087 ceph-mon[27372]: mgrmap e16: y(active, since 4m), standbys: x 2024-02-13T14:15:26.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:25 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:15:26.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:25 smithi087 ceph-mon[39120]: pgmap v209: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 69 KiB/s rd, 5.2 KiB/s wr, 166 op/s 2024-02-13T14:15:26.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:25 smithi087 ceph-mon[39120]: mgrmap e16: y(active, since 4m), standbys: x 2024-02-13T14:15:26.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:25 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:15:26.810 DEBUG:teuthology.orchestra.run.smithi087:node-exporter.a> sudo journalctl -f -n 0 -u ceph-5c04b074-ca79-11ee-95b9-87774f69a715@node-exporter.a.service 2024-02-13T14:15:26.813 DEBUG:teuthology.orchestra.run.smithi148:node-exporter.b> sudo journalctl -f -n 0 -u ceph-5c04b074-ca79-11ee-95b9-87774f69a715@node-exporter.b.service 2024-02-13T14:15:26.815 INFO:tasks.cephadm:Adding alertmanager.a on smithi087 2024-02-13T14:15:26.816 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5c04b074-ca79-11ee-95b9-87774f69a715 -- ceph orch apply alertmanager '1;smithi087=a' 2024-02-13T14:15:27.118 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:26 smithi148 ceph-mon[31198]: from='client.24421 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi087=a;smithi148=b", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T14:15:27.118 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:26 smithi148 ceph-mon[31198]: Saving service node-exporter spec with placement smithi087=a;smithi148=b;count:2 2024-02-13T14:15:27.243 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:26 smithi087 ceph-mon[27372]: from='client.24421 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi087=a;smithi148=b", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T14:15:27.243 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:26 smithi087 ceph-mon[27372]: Saving service node-exporter spec with placement smithi087=a;smithi148=b;count:2 2024-02-13T14:15:27.244 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:26 smithi087 ceph-mon[39120]: from='client.24421 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi087=a;smithi148=b", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T14:15:27.244 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:26 smithi087 ceph-mon[39120]: Saving service node-exporter spec with placement smithi087=a;smithi148=b;count:2 2024-02-13T14:15:27.337 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/mon.b/config 2024-02-13T14:15:28.118 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:27 smithi148 ceph-mon[31198]: pgmap v210: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s rd, 4.4 KiB/s wr, 139 op/s 2024-02-13T14:15:28.491 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:27 smithi087 ceph-mon[27372]: pgmap v210: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s rd, 4.4 KiB/s wr, 139 op/s 2024-02-13T14:15:28.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:27 smithi087 ceph-mon[39120]: pgmap v210: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s rd, 4.4 KiB/s wr, 139 op/s 2024-02-13T14:15:29.196 INFO:teuthology.orchestra.run.smithi148.stdout:Scheduled alertmanager update... 2024-02-13T14:15:29.317 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:29 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]: dispatch 2024-02-13T14:15:29.318 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:29 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2024-02-13T14:15:29.318 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:29 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2024-02-13T14:15:29.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:29 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]: dispatch 2024-02-13T14:15:29.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:29 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2024-02-13T14:15:29.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:29 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2024-02-13T14:15:29.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:29 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]: dispatch 2024-02-13T14:15:29.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:29 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2024-02-13T14:15:29.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:29 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2024-02-13T14:15:29.886 DEBUG:teuthology.orchestra.run.smithi087:alertmanager.a> sudo journalctl -f -n 0 -u ceph-5c04b074-ca79-11ee-95b9-87774f69a715@alertmanager.a.service 2024-02-13T14:15:29.889 INFO:tasks.cephadm:Adding grafana.a on smithi148 2024-02-13T14:15:29.889 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5c04b074-ca79-11ee-95b9-87774f69a715 -- ceph orch apply grafana '1;smithi148=a' 2024-02-13T14:15:30.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:30 smithi148 ceph-mon[31198]: pgmap v211: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 48 KiB/s rd, 3.7 KiB/s wr, 116 op/s 2024-02-13T14:15:30.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:30 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]': finished 2024-02-13T14:15:30.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:30 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]': finished 2024-02-13T14:15:30.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:30 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]': finished 2024-02-13T14:15:30.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:30 smithi148 ceph-mon[31198]: osdmap e72: 8 total, 8 up, 8 in 2024-02-13T14:15:30.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:30 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:15:30.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:30 smithi087 ceph-mon[27372]: pgmap v211: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 48 KiB/s rd, 3.7 KiB/s wr, 116 op/s 2024-02-13T14:15:30.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:30 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]': finished 2024-02-13T14:15:30.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:30 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]': finished 2024-02-13T14:15:30.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:30 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]': finished 2024-02-13T14:15:30.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:30 smithi087 ceph-mon[27372]: osdmap e72: 8 total, 8 up, 8 in 2024-02-13T14:15:30.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:30 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:15:30.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:30 smithi087 ceph-mon[39120]: pgmap v211: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 48 KiB/s rd, 3.7 KiB/s wr, 116 op/s 2024-02-13T14:15:30.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:30 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]': finished 2024-02-13T14:15:30.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:30 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]': finished 2024-02-13T14:15:30.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:30 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]': finished 2024-02-13T14:15:30.243 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:30 smithi087 ceph-mon[39120]: osdmap e72: 8 total, 8 up, 8 in 2024-02-13T14:15:30.243 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:30 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:15:30.243 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/mon.b/config 2024-02-13T14:15:31.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:31 smithi148 ceph-mon[31198]: from='client.24427 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi087=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T14:15:31.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:31 smithi148 ceph-mon[31198]: Saving service alertmanager spec with placement smithi087=a;count:1 2024-02-13T14:15:31.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:31 smithi148 ceph-mon[31198]: osdmap e73: 8 total, 8 up, 8 in 2024-02-13T14:15:31.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:31 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:15:31.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:31 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:15:31.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:31 smithi087 ceph-mon[27372]: from='client.24427 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi087=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T14:15:31.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:31 smithi087 ceph-mon[27372]: Saving service alertmanager spec with placement smithi087=a;count:1 2024-02-13T14:15:31.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:31 smithi087 ceph-mon[27372]: osdmap e73: 8 total, 8 up, 8 in 2024-02-13T14:15:31.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:31 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:15:31.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:31 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:15:31.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:31 smithi087 ceph-mon[39120]: from='client.24427 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi087=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T14:15:31.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:31 smithi087 ceph-mon[39120]: Saving service alertmanager spec with placement smithi087=a;count:1 2024-02-13T14:15:31.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:31 smithi087 ceph-mon[39120]: osdmap e73: 8 total, 8 up, 8 in 2024-02-13T14:15:31.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:31 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:15:31.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:31 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:15:32.118 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:32 smithi148 ceph-mon[31198]: pgmap v214: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 21 KiB/s rd, 1.5 KiB/s wr, 50 op/s; 13 B/s, 0 objects/s recovering 2024-02-13T14:15:32.118 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:32 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:15:32.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:32 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:15:32.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:32 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:15:32.233 INFO:teuthology.orchestra.run.smithi148.stdout:Scheduled grafana update... 2024-02-13T14:15:32.438 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:32 smithi087 ceph-mon[27372]: pgmap v214: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 21 KiB/s rd, 1.5 KiB/s wr, 50 op/s; 13 B/s, 0 objects/s recovering 2024-02-13T14:15:32.438 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:32 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:15:32.438 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:32 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:15:32.438 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:32 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:15:32.439 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:32 smithi087 ceph-mon[39120]: pgmap v214: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 21 KiB/s rd, 1.5 KiB/s wr, 50 op/s; 13 B/s, 0 objects/s recovering 2024-02-13T14:15:32.439 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:32 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:15:32.439 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:32 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:15:32.439 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:32 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:15:32.900 DEBUG:teuthology.orchestra.run.smithi148:grafana.a> sudo journalctl -f -n 0 -u ceph-5c04b074-ca79-11ee-95b9-87774f69a715@grafana.a.service 2024-02-13T14:15:32.904 INFO:tasks.cephadm:Setting up client nodes... 2024-02-13T14:15:32.904 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5c04b074-ca79-11ee-95b9-87774f69a715 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-02-13T14:15:33.152 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/mon.a/config 2024-02-13T14:15:33.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:33 smithi087 ceph-mon[27372]: Deploying daemon prometheus.a on smithi148 2024-02-13T14:15:33.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:33 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:15:33.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:33 smithi087 ceph-mon[39120]: Deploying daemon prometheus.a on smithi148 2024-02-13T14:15:33.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:33 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:15:33.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:33 smithi148 ceph-mon[31198]: Deploying daemon prometheus.a on smithi148 2024-02-13T14:15:33.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:33 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:15:34.017 INFO:journalctl@ceph.iscsi.iscsi.a.smithi148.stdout:Feb 13 14:15:33 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-iscsi-iscsi-a[59644]: debug there is no tcmu-runner data available 2024-02-13T14:15:34.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:34 smithi148 ceph-mon[31198]: from='client.24433 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi148=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T14:15:34.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:34 smithi148 ceph-mon[31198]: Saving service grafana spec with placement smithi148=a;count:1 2024-02-13T14:15:34.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:34 smithi148 ceph-mon[31198]: pgmap v215: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 127 B/s wr, 2 op/s; 54 B/s, 1 objects/s recovering 2024-02-13T14:15:34.396 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:34 smithi087 ceph-mon[27372]: from='client.24433 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi148=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T14:15:34.396 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:34 smithi087 ceph-mon[27372]: Saving service grafana spec with placement smithi148=a;count:1 2024-02-13T14:15:34.397 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:34 smithi087 ceph-mon[27372]: pgmap v215: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 127 B/s wr, 2 op/s; 54 B/s, 1 objects/s recovering 2024-02-13T14:15:34.397 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:34 smithi087 ceph-mon[39120]: from='client.24433 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi148=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T14:15:34.397 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:34 smithi087 ceph-mon[39120]: Saving service grafana spec with placement smithi148=a;count:1 2024-02-13T14:15:34.397 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:34 smithi087 ceph-mon[39120]: pgmap v215: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 127 B/s wr, 2 op/s; 54 B/s, 1 objects/s recovering 2024-02-13T14:15:35.225 INFO:teuthology.orchestra.run.smithi087.stdout:[client.0] 2024-02-13T14:15:35.225 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQCHectlk4U4DRAAX8jGRE5S4miGaveKhVU87w== 2024-02-13T14:15:35.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:35 smithi148 ceph-mon[31198]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:15:35.491 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:35 smithi087 ceph-mon[27372]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:15:35.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:35 smithi087 ceph-mon[39120]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:15:35.898 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-02-13T14:15:35.898 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-02-13T14:15:35.898 DEBUG:teuthology.orchestra.run.smithi087:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-02-13T14:15:35.941 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5c04b074-ca79-11ee-95b9-87774f69a715 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-02-13T14:15:36.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:36 smithi087 ceph-mon[39120]: pgmap v216: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 127 B/s wr, 2 op/s; 54 B/s, 2 objects/s recovering 2024-02-13T14:15:36.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:36 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/4220118829' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-13T14:15:36.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:36 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/4220118829' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-13T14:15:36.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:36 smithi087 ceph-mon[27372]: pgmap v216: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 127 B/s wr, 2 op/s; 54 B/s, 2 objects/s recovering 2024-02-13T14:15:36.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:36 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/4220118829' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-13T14:15:36.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:36 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/4220118829' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-13T14:15:36.246 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/mon.b/config 2024-02-13T14:15:36.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:36 smithi148 ceph-mon[31198]: pgmap v216: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 127 B/s wr, 2 op/s; 54 B/s, 2 objects/s recovering 2024-02-13T14:15:36.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:36 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/4220118829' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-13T14:15:36.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:36 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/4220118829' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-13T14:15:38.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:38 smithi148 ceph-mon[31198]: pgmap v217: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 511 B/s rd, 0 op/s; 54 B/s, 2 objects/s recovering 2024-02-13T14:15:38.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:38 smithi087 ceph-mon[27372]: pgmap v217: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 511 B/s rd, 0 op/s; 54 B/s, 2 objects/s recovering 2024-02-13T14:15:38.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:38 smithi087 ceph-mon[39120]: pgmap v217: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 511 B/s rd, 0 op/s; 54 B/s, 2 objects/s recovering 2024-02-13T14:15:40.311 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:40 smithi148 ceph-mon[31198]: pgmap v218: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 927 B/s rd, 0 op/s; 44 B/s, 1 objects/s recovering 2024-02-13T14:15:40.491 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:40 smithi087 ceph-mon[27372]: pgmap v218: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 927 B/s rd, 0 op/s; 44 B/s, 1 objects/s recovering 2024-02-13T14:15:40.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:40 smithi087 ceph-mon[39120]: pgmap v218: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 927 B/s rd, 0 op/s; 44 B/s, 1 objects/s recovering 2024-02-13T14:15:41.078 INFO:teuthology.orchestra.run.smithi148.stdout:[client.1] 2024-02-13T14:15:41.079 INFO:teuthology.orchestra.run.smithi148.stdout: key = AQCNectl1oNpBBAAf3HP75wW95KZdorg7FkYDA== 2024-02-13T14:15:42.327 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:42 smithi148 ceph-mon[31198]: pgmap v219: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 843 B/s rd, 0 op/s; 40 B/s, 1 objects/s recovering 2024-02-13T14:15:42.328 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:42 smithi148 ceph-mon[31198]: from='client.? 172.21.15.148:0/1792112420' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-13T14:15:42.328 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:42 smithi148 ceph-mon[31198]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-13T14:15:42.328 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:42 smithi148 ceph-mon[31198]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-13T14:15:42.358 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-02-13T14:15:42.358 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2024-02-13T14:15:42.358 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-02-13T14:15:42.404 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-02-13T14:15:42.404 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-02-13T14:15:42.404 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662 shell --fsid 5c04b074-ca79-11ee-95b9-87774f69a715 -- ceph mgr dump --format=json 2024-02-13T14:15:42.433 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:42 smithi087 ceph-mon[27372]: pgmap v219: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 843 B/s rd, 0 op/s; 40 B/s, 1 objects/s recovering 2024-02-13T14:15:42.433 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:42 smithi087 ceph-mon[27372]: from='client.? 172.21.15.148:0/1792112420' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-13T14:15:42.433 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:42 smithi087 ceph-mon[27372]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-13T14:15:42.433 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:42 smithi087 ceph-mon[27372]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-13T14:15:42.434 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:42 smithi087 ceph-mon[39120]: pgmap v219: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 843 B/s rd, 0 op/s; 40 B/s, 1 objects/s recovering 2024-02-13T14:15:42.434 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:42 smithi087 ceph-mon[39120]: from='client.? 172.21.15.148:0/1792112420' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-13T14:15:42.434 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:42 smithi087 ceph-mon[39120]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-13T14:15:42.434 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:42 smithi087 ceph-mon[39120]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-13T14:15:42.654 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/mon.a/config 2024-02-13T14:15:43.805 INFO:journalctl@ceph.iscsi.iscsi.a.smithi148.stdout:Feb 13 14:15:43 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-iscsi-iscsi-a[59644]: debug there is no tcmu-runner data available 2024-02-13T14:15:44.096 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:44 smithi148 ceph-mon[31198]: pgmap v220: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 27 B/s, 0 objects/s recovering 2024-02-13T14:15:44.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:44 smithi087 ceph-mon[27372]: pgmap v220: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 27 B/s, 0 objects/s recovering 2024-02-13T14:15:44.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:44 smithi087 ceph-mon[39120]: pgmap v220: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 27 B/s, 0 objects/s recovering 2024-02-13T14:15:44.771 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-02-13T14:15:44.854 INFO:journalctl@ceph.prometheus.a.smithi148.stdout:Feb 13 14:15:44 smithi148 systemd[1]: Starting Ceph prometheus.a for 5c04b074-ca79-11ee-95b9-87774f69a715... 2024-02-13T14:15:45.118 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:45 smithi148 ceph-mon[31198]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:15:45.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:45 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/2873977512' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-02-13T14:15:45.119 INFO:journalctl@ceph.prometheus.a.smithi148.stdout:Feb 13 14:15:44 smithi148 podman[61647]: 2024-02-13 14:15:44.756691263 +0000 UTC m=+0.019712853 image pull a07b618ecd1dce142bce4c52f0e80982eaf1f14265a2415c2d35978ccaa0a464 quay.io/prometheus/prometheus:v2.43.0 2024-02-13T14:15:45.119 INFO:journalctl@ceph.prometheus.a.smithi148.stdout:Feb 13 14:15:44 smithi148 podman[61647]: 2024-02-13 14:15:44.857050511 +0000 UTC m=+0.120072139 container create 73aa09db99e18dfa227340e972c88a5eee1f238d191d1b6934605dde7b54cb4b (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-02-13T14:15:45.477 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:45 smithi087 ceph-mon[27372]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:15:45.477 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:45 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/2873977512' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-02-13T14:15:45.477 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:45 smithi087 ceph-mon[39120]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:15:45.477 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:45 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/2873977512' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-02-13T14:15:45.506 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":16,"flags":0,"active_gid":14150,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":3827695211},{"type":"v1","addr":"172.21.15.87:6801","nonce":3827695211}]},"active_addr":"172.21.15.87:6801/3827695211","active_change":"2024-02-13T14:10:28.875980+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24115,"name":"x","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.87:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"cephadm","addrvec":[{"type":"v2","addr":"172.21.15.87:0","nonce":864280483}]},{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.87:0","nonce":3749745598}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.87:0","nonce":878623867}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.87:0","nonce":3369062331}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.87:0","nonce":800679329}]}]} 2024-02-13T14:15:45.509 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-02-13T14:15:45.510 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-02-13T14:15:45.510 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662 shell --fsid 5c04b074-ca79-11ee-95b9-87774f69a715 -- ceph osd dump --format=json 2024-02-13T14:15:45.619 INFO:journalctl@ceph.prometheus.a.smithi148.stdout:Feb 13 14:15:45 smithi148 podman[61647]: 2024-02-13 14:15:45.123480795 +0000 UTC m=+0.386502377 container init 73aa09db99e18dfa227340e972c88a5eee1f238d191d1b6934605dde7b54cb4b (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-02-13T14:15:45.619 INFO:journalctl@ceph.prometheus.a.smithi148.stdout:Feb 13 14:15:45 smithi148 podman[61647]: 2024-02-13 14:15:45.128826606 +0000 UTC m=+0.391848190 container start 73aa09db99e18dfa227340e972c88a5eee1f238d191d1b6934605dde7b54cb4b (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-02-13T14:15:45.619 INFO:journalctl@ceph.prometheus.a.smithi148.stdout:Feb 13 14:15:45 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-prometheus-a[61663]: ts=2024-02-13T14:15:45.152Z caller=main.go:564 level=info msg="Starting Prometheus Server" mode=server version="(version=2.43.0, branch=HEAD, revision=edfc3bcd025dd6fe296c167a14a216cab1e552ee)" 2024-02-13T14:15:45.619 INFO:journalctl@ceph.prometheus.a.smithi148.stdout:Feb 13 14:15:45 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-prometheus-a[61663]: ts=2024-02-13T14:15:45.152Z caller=main.go:569 level=info build_context="(go=go1.19.7, platform=linux/amd64, user=root@8a0ee342e522, date=20230321-12:56:07, tags=netgo,builtinassets)" 2024-02-13T14:15:45.620 INFO:journalctl@ceph.prometheus.a.smithi148.stdout:Feb 13 14:15:45 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-prometheus-a[61663]: ts=2024-02-13T14:15:45.152Z caller=main.go:570 level=info host_details="(Linux 5.14.0-419.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Wed Feb 7 23:01:41 UTC 2024 x86_64 smithi148 (none))" 2024-02-13T14:15:45.620 INFO:journalctl@ceph.prometheus.a.smithi148.stdout:Feb 13 14:15:45 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-prometheus-a[61663]: ts=2024-02-13T14:15:45.152Z caller=main.go:571 level=info fd_limits="(soft=1048576, hard=1048576)" 2024-02-13T14:15:45.620 INFO:journalctl@ceph.prometheus.a.smithi148.stdout:Feb 13 14:15:45 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-prometheus-a[61663]: ts=2024-02-13T14:15:45.152Z caller=main.go:572 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2024-02-13T14:15:45.620 INFO:journalctl@ceph.prometheus.a.smithi148.stdout:Feb 13 14:15:45 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-prometheus-a[61663]: ts=2024-02-13T14:15:45.154Z caller=web.go:561 level=info component=web msg="Start listening for connections" address=:9095 2024-02-13T14:15:45.620 INFO:journalctl@ceph.prometheus.a.smithi148.stdout:Feb 13 14:15:45 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-prometheus-a[61663]: ts=2024-02-13T14:15:45.154Z caller=main.go:1005 level=info msg="Starting TSDB ..." 2024-02-13T14:15:45.620 INFO:journalctl@ceph.prometheus.a.smithi148.stdout:Feb 13 14:15:45 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-prometheus-a[61663]: ts=2024-02-13T14:15:45.154Z caller=tls_config.go:232 level=info component=web msg="Listening on" address=[::]:9095 2024-02-13T14:15:45.620 INFO:journalctl@ceph.prometheus.a.smithi148.stdout:Feb 13 14:15:45 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-prometheus-a[61663]: ts=2024-02-13T14:15:45.154Z caller=tls_config.go:235 level=info component=web msg="TLS is disabled." http2=false address=[::]:9095 2024-02-13T14:15:45.620 INFO:journalctl@ceph.prometheus.a.smithi148.stdout:Feb 13 14:15:45 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-prometheus-a[61663]: ts=2024-02-13T14:15:45.156Z caller=head.go:587 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2024-02-13T14:15:45.620 INFO:journalctl@ceph.prometheus.a.smithi148.stdout:Feb 13 14:15:45 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-prometheus-a[61663]: ts=2024-02-13T14:15:45.156Z caller=head.go:658 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=1.556µs 2024-02-13T14:15:45.620 INFO:journalctl@ceph.prometheus.a.smithi148.stdout:Feb 13 14:15:45 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-prometheus-a[61663]: ts=2024-02-13T14:15:45.156Z caller=head.go:664 level=info component=tsdb msg="Replaying WAL, this may take a while" 2024-02-13T14:15:45.620 INFO:journalctl@ceph.prometheus.a.smithi148.stdout:Feb 13 14:15:45 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-prometheus-a[61663]: ts=2024-02-13T14:15:45.157Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=0 2024-02-13T14:15:45.620 INFO:journalctl@ceph.prometheus.a.smithi148.stdout:Feb 13 14:15:45 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-prometheus-a[61663]: ts=2024-02-13T14:15:45.157Z caller=head.go:772 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=15.7µs wal_replay_duration=989.025µs wbl_replay_duration=134ns total_replay_duration=1.021467ms 2024-02-13T14:15:45.621 INFO:journalctl@ceph.prometheus.a.smithi148.stdout:Feb 13 14:15:45 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-prometheus-a[61663]: ts=2024-02-13T14:15:45.158Z caller=main.go:1026 level=info fs_type=XFS_SUPER_MAGIC 2024-02-13T14:15:45.621 INFO:journalctl@ceph.prometheus.a.smithi148.stdout:Feb 13 14:15:45 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-prometheus-a[61663]: ts=2024-02-13T14:15:45.158Z caller=main.go:1029 level=info msg="TSDB started" 2024-02-13T14:15:45.621 INFO:journalctl@ceph.prometheus.a.smithi148.stdout:Feb 13 14:15:45 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-prometheus-a[61663]: ts=2024-02-13T14:15:45.158Z caller=main.go:1209 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2024-02-13T14:15:45.621 INFO:journalctl@ceph.prometheus.a.smithi148.stdout:Feb 13 14:15:45 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-prometheus-a[61663]: ts=2024-02-13T14:15:45.174Z caller=main.go:1246 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=16.408767ms db_storage=890ns remote_storage=993ns web_handler=328ns query_engine=584ns scrape=198.098µs scrape_sd=85.049µs notify=931ns notify_sd=1.89µs rules=15.888695ms tracing=5.307µs 2024-02-13T14:15:45.621 INFO:journalctl@ceph.prometheus.a.smithi148.stdout:Feb 13 14:15:45 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-prometheus-a[61663]: ts=2024-02-13T14:15:45.174Z caller=main.go:990 level=info msg="Server is ready to receive web requests." 2024-02-13T14:15:45.621 INFO:journalctl@ceph.prometheus.a.smithi148.stdout:Feb 13 14:15:45 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-prometheus-a[61663]: ts=2024-02-13T14:15:45.174Z caller=manager.go:974 level=info component="rule manager" msg="Starting rule manager..." 2024-02-13T14:15:45.621 INFO:journalctl@ceph.prometheus.a.smithi148.stdout:Feb 13 14:15:45 smithi148 bash[61647]: 73aa09db99e18dfa227340e972c88a5eee1f238d191d1b6934605dde7b54cb4b 2024-02-13T14:15:45.621 INFO:journalctl@ceph.prometheus.a.smithi148.stdout:Feb 13 14:15:45 smithi148 systemd[1]: Started Ceph prometheus.a for 5c04b074-ca79-11ee-95b9-87774f69a715. 2024-02-13T14:15:45.787 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/mon.a/config 2024-02-13T14:15:46.229 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:46 smithi087 ceph-mon[27372]: pgmap v221: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 0 B/s, 0 objects/s recovering 2024-02-13T14:15:46.230 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:46 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:15:46.230 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:46 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:15:46.230 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:46 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:15:46.230 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:46 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-02-13T14:15:46.230 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:46 smithi087 ceph-mon[39120]: pgmap v221: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 0 B/s, 0 objects/s recovering 2024-02-13T14:15:46.230 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:46 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:15:46.230 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:46 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:15:46.230 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:46 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:15:46.230 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:46 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-02-13T14:15:46.334 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:46 smithi148 ceph-mon[31198]: pgmap v221: 132 pgs: 132 active+clean; 582 KiB data, 243 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 0 B/s, 0 objects/s recovering 2024-02-13T14:15:46.334 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:46 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:15:46.335 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:46 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:15:46.335 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:46 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' 2024-02-13T14:15:46.335 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:46 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-02-13T14:15:46.492 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:15:46 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: ignoring --setuser ceph since I am not root 2024-02-13T14:15:46.492 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:15:46 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: ignoring --setgroup ceph since I am not root 2024-02-13T14:15:46.618 INFO:journalctl@ceph.mgr.x.smithi148.stdout:Feb 13 14:15:46 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-x[33278]: ignoring --setuser ceph since I am not root 2024-02-13T14:15:46.619 INFO:journalctl@ceph.mgr.x.smithi148.stdout:Feb 13 14:15:46 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-x[33278]: ignoring --setgroup ceph since I am not root 2024-02-13T14:15:46.619 INFO:journalctl@ceph.mgr.x.smithi148.stdout:Feb 13 14:15:46 smithi148 ceph-mgr[33301]: -- 172.21.15.148:0/4176045059 <== mon.1 v2:172.21.15.148:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56050ed801a0 con 0x56050ed56800 2024-02-13T14:15:46.964 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:15:46 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: 2024-02-13T14:15:46.868+0000 7f2d98aef200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-13T14:15:47.241 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:15:46 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: 2024-02-13T14:15:46.962+0000 7f2d98aef200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-13T14:15:47.303 INFO:journalctl@ceph.mgr.x.smithi148.stdout:Feb 13 14:15:46 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-x[33278]: 2024-02-13T14:15:46.868+0000 7f00402eb200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-13T14:15:47.303 INFO:journalctl@ceph.mgr.x.smithi148.stdout:Feb 13 14:15:46 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-x[33278]: 2024-02-13T14:15:46.961+0000 7f00402eb200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-13T14:15:47.601 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:47 smithi148 ceph-mon[31198]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-02-13T14:15:47.601 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:47 smithi148 ceph-mon[31198]: mgrmap e17: y(active, since 5m), standbys: x 2024-02-13T14:15:47.636 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:47 smithi087 ceph-mon[27372]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-02-13T14:15:47.636 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:47 smithi087 ceph-mon[27372]: mgrmap e17: y(active, since 5m), standbys: x 2024-02-13T14:15:47.637 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:47 smithi087 ceph-mon[39120]: from='mgr.14150 172.21.15.87:0/2260971040' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-02-13T14:15:47.637 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:47 smithi087 ceph-mon[39120]: mgrmap e17: y(active, since 5m), standbys: x 2024-02-13T14:15:47.755 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-02-13T14:15:47.756 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":73,"fsid":"5c04b074-ca79-11ee-95b9-87774f69a715","created":"2024-02-13T14:09:30.938170+0000","modified":"2024-02-13T14:15:30.001561+0000","last_up_change":"2024-02-13T14:14:57.861459+0000","last_in_change":"2024-02-13T14:14:39.797605+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-13T14:12:53.216571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"datapool","create_time":"2024-02-13T14:15:10.110566+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":66,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.6500000953674316,"score_stable":2.6500000953674316,"optimal_score":0.87999999523162842,"raw_score_acting":2.3299999237060547,"raw_score_stable":2.3299999237060547,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":".rgw.root","create_time":"2024-02-13T14:15:10.744089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"default.rgw.log","create_time":"2024-02-13T14:15:12.716985+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"default.rgw.control","create_time":"2024-02-13T14:15:14.422718+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-02-13T14:15:16.484523+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"efef6830-b1ff-4496-a434-dfd495a55a6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":2357055010},{"type":"v1","addr":"172.21.15.87:6803","nonce":2357055010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":2357055010},{"type":"v1","addr":"172.21.15.87:6805","nonce":2357055010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":2357055010},{"type":"v1","addr":"172.21.15.87:6809","nonce":2357055010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":2357055010},{"type":"v1","addr":"172.21.15.87:6807","nonce":2357055010}]},"public_addr":"172.21.15.87:6803/2357055010","cluster_addr":"172.21.15.87:6805/2357055010","heartbeat_back_addr":"172.21.15.87:6809/2357055010","heartbeat_front_addr":"172.21.15.87:6807/2357055010","state":["exists","up"]},{"osd":1,"uuid":"399d507f-ed8f-4f3f-83c3-a3f5fdbaeefd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":1433765078},{"type":"v1","addr":"172.21.15.87:6811","nonce":1433765078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":1433765078},{"type":"v1","addr":"172.21.15.87:6813","nonce":1433765078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":1433765078},{"type":"v1","addr":"172.21.15.87:6817","nonce":1433765078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":1433765078},{"type":"v1","addr":"172.21.15.87:6815","nonce":1433765078}]},"public_addr":"172.21.15.87:6811/1433765078","cluster_addr":"172.21.15.87:6813/1433765078","heartbeat_back_addr":"172.21.15.87:6817/1433765078","heartbeat_front_addr":"172.21.15.87:6815/1433765078","state":["exists","up"]},{"osd":2,"uuid":"9517393e-da9c-46b8-90ab-9937cf20659c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":1764493594},{"type":"v1","addr":"172.21.15.87:6819","nonce":1764493594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":1764493594},{"type":"v1","addr":"172.21.15.87:6821","nonce":1764493594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1764493594},{"type":"v1","addr":"172.21.15.87:6825","nonce":1764493594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":1764493594},{"type":"v1","addr":"172.21.15.87:6823","nonce":1764493594}]},"public_addr":"172.21.15.87:6819/1764493594","cluster_addr":"172.21.15.87:6821/1764493594","heartbeat_back_addr":"172.21.15.87:6825/1764493594","heartbeat_front_addr":"172.21.15.87:6823/1764493594","state":["exists","up"]},{"osd":3,"uuid":"dfa2f275-b84b-4227-8135-e009b2cb461e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1162405572},{"type":"v1","addr":"172.21.15.87:6827","nonce":1162405572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1162405572},{"type":"v1","addr":"172.21.15.87:6829","nonce":1162405572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":1162405572},{"type":"v1","addr":"172.21.15.87:6833","nonce":1162405572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1162405572},{"type":"v1","addr":"172.21.15.87:6831","nonce":1162405572}]},"public_addr":"172.21.15.87:6827/1162405572","cluster_addr":"172.21.15.87:6829/1162405572","heartbeat_back_addr":"172.21.15.87:6833/1162405572","heartbeat_front_addr":"172.21.15.87:6831/1162405572","state":["exists","up"]},{"osd":4,"uuid":"9b07d44a-ddd1-4b31-8492-3e1459c0ad1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":1403587816},{"type":"v1","addr":"172.21.15.148:6801","nonce":1403587816}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1403587816},{"type":"v1","addr":"172.21.15.148:6803","nonce":1403587816}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1403587816},{"type":"v1","addr":"172.21.15.148:6807","nonce":1403587816}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1403587816},{"type":"v1","addr":"172.21.15.148:6805","nonce":1403587816}]},"public_addr":"172.21.15.148:6801/1403587816","cluster_addr":"172.21.15.148:6803/1403587816","heartbeat_back_addr":"172.21.15.148:6807/1403587816","heartbeat_front_addr":"172.21.15.148:6805/1403587816","state":["exists","up"]},{"osd":5,"uuid":"cba0ca7e-9575-410b-92d6-d9bb992608f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":4089539002},{"type":"v1","addr":"172.21.15.148:6809","nonce":4089539002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":4089539002},{"type":"v1","addr":"172.21.15.148:6811","nonce":4089539002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":4089539002},{"type":"v1","addr":"172.21.15.148:6815","nonce":4089539002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":4089539002},{"type":"v1","addr":"172.21.15.148:6813","nonce":4089539002}]},"public_addr":"172.21.15.148:6809/4089539002","cluster_addr":"172.21.15.148:6811/4089539002","heartbeat_back_addr":"172.21.15.148:6815/4089539002","heartbeat_front_addr":"172.21.15.148:6813/4089539002","state":["exists","up"]},{"osd":6,"uuid":"4f958ddf-d72c-434d-bdfc-fb039719e39b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":1791408651},{"type":"v1","addr":"172.21.15.148:6817","nonce":1791408651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":1791408651},{"type":"v1","addr":"172.21.15.148:6819","nonce":1791408651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":1791408651},{"type":"v1","addr":"172.21.15.148:6823","nonce":1791408651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":1791408651},{"type":"v1","addr":"172.21.15.148:6821","nonce":1791408651}]},"public_addr":"172.21.15.148:6817/1791408651","cluster_addr":"172.21.15.148:6819/1791408651","heartbeat_back_addr":"172.21.15.148:6823/1791408651","heartbeat_front_addr":"172.21.15.148:6821/1791408651","state":["exists","up"]},{"osd":7,"uuid":"528011a9-d721-42e5-ba03-d44ef41d1bf1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":2269337484},{"type":"v1","addr":"172.21.15.148:6825","nonce":2269337484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2269337484},{"type":"v1","addr":"172.21.15.148:6827","nonce":2269337484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":2269337484},{"type":"v1","addr":"172.21.15.148:6831","nonce":2269337484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":2269337484},{"type":"v1","addr":"172.21.15.148:6829","nonce":2269337484}]},"public_addr":"172.21.15.148:6825/2269337484","cluster_addr":"172.21.15.148:6827/2269337484","heartbeat_back_addr":"172.21.15.148:6831/2269337484","heartbeat_front_addr":"172.21.15.148:6829/2269337484","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-13T14:11:55.754942+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-13T14:12:22.510245+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-13T14:12:49.085565+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-13T14:13:15.484384+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-13T14:13:38.935614+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-13T14:14:04.066142+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-13T14:14:30.071935+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-13T14:14:55.918850+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]},{"pgid":"4.14","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.87:0/3893540759":"2024-02-14T14:10:28.875829+0000","172.21.15.87:0/179577207":"2024-02-14T14:10:01.886668+0000","172.21.15.87:0/3243120218":"2024-02-14T14:10:28.875829+0000","172.21.15.87:0/1580943540":"2024-02-14T14:10:01.886668+0000","172.21.15.87:6800/2307544135":"2024-02-14T14:10:28.875829+0000","172.21.15.87:6800/1895635638":"2024-02-14T14:10:01.886668+0000","172.21.15.87:0/2387914205":"2024-02-14T14:10:28.875829+0000","172.21.15.87:6801/2307544135":"2024-02-14T14:10:28.875829+0000","172.21.15.87:0/1196696660":"2024-02-14T14:10:01.886668+0000","172.21.15.87:6801/1895635638":"2024-02-14T14:10:01.886668+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-13T14:15:47.868 INFO:journalctl@ceph.mgr.x.smithi148.stdout:Feb 13 14:15:47 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-x[33278]: 2024-02-13T14:15:47.599+0000 7f00402eb200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-13T14:15:47.868 INFO:journalctl@ceph.mgr.x.smithi148.stdout:Feb 13 14:15:47 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-x[33278]: 2024-02-13T14:15:47.692+0000 7f00402eb200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-13T14:15:47.869 INFO:journalctl@ceph.mgr.x.smithi148.stdout:Feb 13 14:15:47 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-x[33278]: 2024-02-13T14:15:47.836+0000 7f00402eb200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-13T14:15:47.977 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:15:47 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: 2024-02-13T14:15:47.634+0000 7f2d98aef200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-13T14:15:47.977 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:15:47 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: 2024-02-13T14:15:47.727+0000 7f2d98aef200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-13T14:15:47.977 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:15:47 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: 2024-02-13T14:15:47.878+0000 7f2d98aef200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-13T14:15:47.977 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:15:47 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: 2024-02-13T14:15:47.975+0000 7f2d98aef200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-13T14:15:48.241 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:15:48 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: 2024-02-13T14:15:48.060+0000 7f2d98aef200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-13T14:15:48.284 INFO:journalctl@ceph.mgr.x.smithi148.stdout:Feb 13 14:15:47 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-x[33278]: 2024-02-13T14:15:47.933+0000 7f00402eb200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-13T14:15:48.284 INFO:journalctl@ceph.mgr.x.smithi148.stdout:Feb 13 14:15:48 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-x[33278]: 2024-02-13T14:15:48.016+0000 7f00402eb200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-13T14:15:48.284 INFO:journalctl@ceph.mgr.x.smithi148.stdout:Feb 13 14:15:48 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-x[33278]: 2024-02-13T14:15:48.282+0000 7f00402eb200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-13T14:15:48.414 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-02-13T14:15:48.414 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662 shell --fsid 5c04b074-ca79-11ee-95b9-87774f69a715 -- ceph osd dump --format=json 2024-02-13T14:15:48.575 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:48 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/3050412449' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-13T14:15:48.575 INFO:journalctl@ceph.mgr.x.smithi148.stdout:Feb 13 14:15:48 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-x[33278]: 2024-02-13T14:15:48.392+0000 7f00402eb200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-13T14:15:48.576 INFO:journalctl@ceph.mgr.x.smithi148.stdout:Feb 13 14:15:48 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-x[33278]: 2024-02-13T14:15:48.478+0000 7f00402eb200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-13T14:15:48.642 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:48 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/3050412449' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-13T14:15:48.642 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:48 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/3050412449' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-13T14:15:48.642 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:15:48 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: 2024-02-13T14:15:48.331+0000 7f2d98aef200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-13T14:15:48.643 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:15:48 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: 2024-02-13T14:15:48.444+0000 7f2d98aef200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-13T14:15:48.643 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:15:48 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: 2024-02-13T14:15:48.535+0000 7f2d98aef200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-13T14:15:48.668 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/mon.a/config 2024-02-13T14:15:48.868 INFO:journalctl@ceph.mgr.x.smithi148.stdout:Feb 13 14:15:48 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-x[33278]: 2024-02-13T14:15:48.573+0000 7f00402eb200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-13T14:15:48.868 INFO:journalctl@ceph.mgr.x.smithi148.stdout:Feb 13 14:15:48 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-x[33278]: 2024-02-13T14:15:48.749+0000 7f00402eb200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-13T14:15:48.893 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:15:48 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: 2024-02-13T14:15:48.640+0000 7f2d98aef200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-13T14:15:48.893 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:15:48 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: 2024-02-13T14:15:48.827+0000 7f2d98aef200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-13T14:15:50.118 INFO:journalctl@ceph.mgr.x.smithi148.stdout:Feb 13 14:15:49 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-x[33278]: 2024-02-13T14:15:49.716+0000 7f00402eb200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-13T14:15:50.118 INFO:journalctl@ceph.mgr.x.smithi148.stdout:Feb 13 14:15:49 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-x[33278]: 2024-02-13T14:15:49.847+0000 7f00402eb200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-13T14:15:50.145 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:15:49 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: 2024-02-13T14:15:49.823+0000 7f2d98aef200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-13T14:15:50.145 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:15:49 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: 2024-02-13T14:15:49.965+0000 7f2d98aef200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-13T14:15:50.424 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:15:50 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: 2024-02-13T14:15:50.313+0000 7f2d98aef200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-13T14:15:50.477 INFO:journalctl@ceph.mgr.x.smithi148.stdout:Feb 13 14:15:50 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-x[33278]: 2024-02-13T14:15:50.183+0000 7f00402eb200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-13T14:15:50.477 INFO:journalctl@ceph.mgr.x.smithi148.stdout:Feb 13 14:15:50 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-x[33278]: 2024-02-13T14:15:50.275+0000 7f00402eb200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-13T14:15:50.477 INFO:journalctl@ceph.mgr.x.smithi148.stdout:Feb 13 14:15:50 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-x[33278]: 2024-02-13T14:15:50.475+0000 7f00402eb200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-13T14:15:50.627 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-02-13T14:15:50.628 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":73,"fsid":"5c04b074-ca79-11ee-95b9-87774f69a715","created":"2024-02-13T14:09:30.938170+0000","modified":"2024-02-13T14:15:30.001561+0000","last_up_change":"2024-02-13T14:14:57.861459+0000","last_in_change":"2024-02-13T14:14:39.797605+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-13T14:12:53.216571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"datapool","create_time":"2024-02-13T14:15:10.110566+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":66,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.6500000953674316,"score_stable":2.6500000953674316,"optimal_score":0.87999999523162842,"raw_score_acting":2.3299999237060547,"raw_score_stable":2.3299999237060547,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":".rgw.root","create_time":"2024-02-13T14:15:10.744089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"default.rgw.log","create_time":"2024-02-13T14:15:12.716985+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"default.rgw.control","create_time":"2024-02-13T14:15:14.422718+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-02-13T14:15:16.484523+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"efef6830-b1ff-4496-a434-dfd495a55a6e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":2357055010},{"type":"v1","addr":"172.21.15.87:6803","nonce":2357055010}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":2357055010},{"type":"v1","addr":"172.21.15.87:6805","nonce":2357055010}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":2357055010},{"type":"v1","addr":"172.21.15.87:6809","nonce":2357055010}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":2357055010},{"type":"v1","addr":"172.21.15.87:6807","nonce":2357055010}]},"public_addr":"172.21.15.87:6803/2357055010","cluster_addr":"172.21.15.87:6805/2357055010","heartbeat_back_addr":"172.21.15.87:6809/2357055010","heartbeat_front_addr":"172.21.15.87:6807/2357055010","state":["exists","up"]},{"osd":1,"uuid":"399d507f-ed8f-4f3f-83c3-a3f5fdbaeefd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":1433765078},{"type":"v1","addr":"172.21.15.87:6811","nonce":1433765078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":1433765078},{"type":"v1","addr":"172.21.15.87:6813","nonce":1433765078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":1433765078},{"type":"v1","addr":"172.21.15.87:6817","nonce":1433765078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":1433765078},{"type":"v1","addr":"172.21.15.87:6815","nonce":1433765078}]},"public_addr":"172.21.15.87:6811/1433765078","cluster_addr":"172.21.15.87:6813/1433765078","heartbeat_back_addr":"172.21.15.87:6817/1433765078","heartbeat_front_addr":"172.21.15.87:6815/1433765078","state":["exists","up"]},{"osd":2,"uuid":"9517393e-da9c-46b8-90ab-9937cf20659c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":1764493594},{"type":"v1","addr":"172.21.15.87:6819","nonce":1764493594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":1764493594},{"type":"v1","addr":"172.21.15.87:6821","nonce":1764493594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1764493594},{"type":"v1","addr":"172.21.15.87:6825","nonce":1764493594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":1764493594},{"type":"v1","addr":"172.21.15.87:6823","nonce":1764493594}]},"public_addr":"172.21.15.87:6819/1764493594","cluster_addr":"172.21.15.87:6821/1764493594","heartbeat_back_addr":"172.21.15.87:6825/1764493594","heartbeat_front_addr":"172.21.15.87:6823/1764493594","state":["exists","up"]},{"osd":3,"uuid":"dfa2f275-b84b-4227-8135-e009b2cb461e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1162405572},{"type":"v1","addr":"172.21.15.87:6827","nonce":1162405572}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1162405572},{"type":"v1","addr":"172.21.15.87:6829","nonce":1162405572}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":1162405572},{"type":"v1","addr":"172.21.15.87:6833","nonce":1162405572}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1162405572},{"type":"v1","addr":"172.21.15.87:6831","nonce":1162405572}]},"public_addr":"172.21.15.87:6827/1162405572","cluster_addr":"172.21.15.87:6829/1162405572","heartbeat_back_addr":"172.21.15.87:6833/1162405572","heartbeat_front_addr":"172.21.15.87:6831/1162405572","state":["exists","up"]},{"osd":4,"uuid":"9b07d44a-ddd1-4b31-8492-3e1459c0ad1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":1403587816},{"type":"v1","addr":"172.21.15.148:6801","nonce":1403587816}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":1403587816},{"type":"v1","addr":"172.21.15.148:6803","nonce":1403587816}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":1403587816},{"type":"v1","addr":"172.21.15.148:6807","nonce":1403587816}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":1403587816},{"type":"v1","addr":"172.21.15.148:6805","nonce":1403587816}]},"public_addr":"172.21.15.148:6801/1403587816","cluster_addr":"172.21.15.148:6803/1403587816","heartbeat_back_addr":"172.21.15.148:6807/1403587816","heartbeat_front_addr":"172.21.15.148:6805/1403587816","state":["exists","up"]},{"osd":5,"uuid":"cba0ca7e-9575-410b-92d6-d9bb992608f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":4089539002},{"type":"v1","addr":"172.21.15.148:6809","nonce":4089539002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":4089539002},{"type":"v1","addr":"172.21.15.148:6811","nonce":4089539002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":4089539002},{"type":"v1","addr":"172.21.15.148:6815","nonce":4089539002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":4089539002},{"type":"v1","addr":"172.21.15.148:6813","nonce":4089539002}]},"public_addr":"172.21.15.148:6809/4089539002","cluster_addr":"172.21.15.148:6811/4089539002","heartbeat_back_addr":"172.21.15.148:6815/4089539002","heartbeat_front_addr":"172.21.15.148:6813/4089539002","state":["exists","up"]},{"osd":6,"uuid":"4f958ddf-d72c-434d-bdfc-fb039719e39b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":1791408651},{"type":"v1","addr":"172.21.15.148:6817","nonce":1791408651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":1791408651},{"type":"v1","addr":"172.21.15.148:6819","nonce":1791408651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":1791408651},{"type":"v1","addr":"172.21.15.148:6823","nonce":1791408651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":1791408651},{"type":"v1","addr":"172.21.15.148:6821","nonce":1791408651}]},"public_addr":"172.21.15.148:6817/1791408651","cluster_addr":"172.21.15.148:6819/1791408651","heartbeat_back_addr":"172.21.15.148:6823/1791408651","heartbeat_front_addr":"172.21.15.148:6821/1791408651","state":["exists","up"]},{"osd":7,"uuid":"528011a9-d721-42e5-ba03-d44ef41d1bf1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":2269337484},{"type":"v1","addr":"172.21.15.148:6825","nonce":2269337484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":2269337484},{"type":"v1","addr":"172.21.15.148:6827","nonce":2269337484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":2269337484},{"type":"v1","addr":"172.21.15.148:6831","nonce":2269337484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":2269337484},{"type":"v1","addr":"172.21.15.148:6829","nonce":2269337484}]},"public_addr":"172.21.15.148:6825/2269337484","cluster_addr":"172.21.15.148:6827/2269337484","heartbeat_back_addr":"172.21.15.148:6831/2269337484","heartbeat_front_addr":"172.21.15.148:6829/2269337484","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-13T14:11:55.754942+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-13T14:12:22.510245+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-13T14:12:49.085565+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-13T14:13:15.484384+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-13T14:13:38.935614+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-13T14:14:04.066142+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-13T14:14:30.071935+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-13T14:14:55.918850+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]},{"pgid":"4.14","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.87:0/3893540759":"2024-02-14T14:10:28.875829+0000","172.21.15.87:0/179577207":"2024-02-14T14:10:01.886668+0000","172.21.15.87:0/3243120218":"2024-02-14T14:10:28.875829+0000","172.21.15.87:0/1580943540":"2024-02-14T14:10:01.886668+0000","172.21.15.87:6800/2307544135":"2024-02-14T14:10:28.875829+0000","172.21.15.87:6800/1895635638":"2024-02-14T14:10:01.886668+0000","172.21.15.87:0/2387914205":"2024-02-14T14:10:28.875829+0000","172.21.15.87:6801/2307544135":"2024-02-14T14:10:28.875829+0000","172.21.15.87:0/1196696660":"2024-02-14T14:10:01.886668+0000","172.21.15.87:6801/1895635638":"2024-02-14T14:10:01.886668+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-13T14:15:50.676 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:15:50 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: 2024-02-13T14:15:50.422+0000 7f2d98aef200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-13T14:15:50.676 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:15:50 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: 2024-02-13T14:15:50.634+0000 7f2d98aef200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-13T14:15:50.868 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:50 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/1393371973' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-13T14:15:50.869 INFO:journalctl@ceph.mgr.x.smithi148.stdout:Feb 13 14:15:50 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-x[33278]: 2024-02-13T14:15:50.663+0000 7f00402eb200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-13T14:15:50.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:50 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/1393371973' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-13T14:15:50.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:50 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/1393371973' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-13T14:15:50.992 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:15:50 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: 2024-02-13T14:15:50.825+0000 7f2d98aef200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-13T14:15:51.217 INFO:journalctl@ceph.mgr.x.smithi148.stdout:Feb 13 14:15:50 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-x[33278]: 2024-02-13T14:15:50.953+0000 7f00402eb200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-13T14:15:51.309 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662 shell --fsid 5c04b074-ca79-11ee-95b9-87774f69a715 -- ceph tell osd.0 flush_pg_stats 2024-02-13T14:15:51.310 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662 shell --fsid 5c04b074-ca79-11ee-95b9-87774f69a715 -- ceph tell osd.1 flush_pg_stats 2024-02-13T14:15:51.310 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662 shell --fsid 5c04b074-ca79-11ee-95b9-87774f69a715 -- ceph tell osd.2 flush_pg_stats 2024-02-13T14:15:51.311 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662 shell --fsid 5c04b074-ca79-11ee-95b9-87774f69a715 -- ceph tell osd.3 flush_pg_stats 2024-02-13T14:15:51.311 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662 shell --fsid 5c04b074-ca79-11ee-95b9-87774f69a715 -- ceph tell osd.4 flush_pg_stats 2024-02-13T14:15:51.311 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662 shell --fsid 5c04b074-ca79-11ee-95b9-87774f69a715 -- ceph tell osd.5 flush_pg_stats 2024-02-13T14:15:51.312 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662 shell --fsid 5c04b074-ca79-11ee-95b9-87774f69a715 -- ceph tell osd.6 flush_pg_stats 2024-02-13T14:15:51.312 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662 shell --fsid 5c04b074-ca79-11ee-95b9-87774f69a715 -- ceph tell osd.7 flush_pg_stats 2024-02-13T14:15:51.390 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:15:51 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: 2024-02-13T14:15:51.118+0000 7f2d98aef200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-13T14:15:51.618 INFO:journalctl@ceph.mgr.x.smithi148.stdout:Feb 13 14:15:51 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-x[33278]: 2024-02-13T14:15:51.215+0000 7f00402eb200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-13T14:15:51.619 INFO:journalctl@ceph.mgr.x.smithi148.stdout:Feb 13 14:15:51 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-x[33278]: 2024-02-13T14:15:51.456+0000 7f00402eb200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-13T14:15:51.742 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:15:51 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: 2024-02-13T14:15:51.431+0000 7f2d98aef200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-13T14:15:51.926 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/mon.a/config 2024-02-13T14:15:51.927 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/mon.a/config 2024-02-13T14:15:51.996 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/mon.a/config 2024-02-13T14:15:52.003 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/mon.a/config 2024-02-13T14:15:52.076 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:15:51 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: 2024-02-13T14:15:51.863+0000 7f2d98aef200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-13T14:15:52.275 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/mon.a/config 2024-02-13T14:15:52.279 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/mon.a/config 2024-02-13T14:15:52.279 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/mon.a/config 2024-02-13T14:15:52.284 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/mon.a/config 2024-02-13T14:15:52.618 INFO:journalctl@ceph.mgr.x.smithi148.stdout:Feb 13 14:15:52 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-x[33278]: 2024-02-13T14:15:52.264+0000 7f00402eb200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-13T14:15:52.946 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:15:52 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: 2024-02-13T14:15:52.796+0000 7f2d98aef200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-13T14:15:53.118 INFO:journalctl@ceph.mgr.x.smithi148.stdout:Feb 13 14:15:52 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-x[33278]: 2024-02-13T14:15:52.843+0000 7f00402eb200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-13T14:15:53.119 INFO:journalctl@ceph.mgr.x.smithi148.stdout:Feb 13 14:15:52 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-x[33278]: 2024-02-13T14:15:52.946+0000 7f00402eb200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-13T14:15:53.502 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:15:53 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: 2024-02-13T14:15:53.378+0000 7f2d98aef200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-13T14:15:53.502 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:15:53 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: 2024-02-13T14:15:53.481+0000 7f2d98aef200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-13T14:15:53.502 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:53 smithi087 ceph-mon[27372]: Standby manager daemon x restarted 2024-02-13T14:15:53.502 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:53 smithi087 ceph-mon[27372]: Standby manager daemon x started 2024-02-13T14:15:53.502 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:53 smithi087 ceph-mon[27372]: from='mgr.? 172.21.15.148:0/2136774154' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-02-13T14:15:53.503 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:53 smithi087 ceph-mon[27372]: from='mgr.? 172.21.15.148:0/2136774154' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-13T14:15:53.503 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:53 smithi087 ceph-mon[27372]: from='mgr.? 172.21.15.148:0/2136774154' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-02-13T14:15:53.503 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:53 smithi087 ceph-mon[27372]: from='mgr.? 172.21.15.148:0/2136774154' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-13T14:15:53.503 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:53 smithi087 ceph-mon[39120]: Standby manager daemon x restarted 2024-02-13T14:15:53.503 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:53 smithi087 ceph-mon[39120]: Standby manager daemon x started 2024-02-13T14:15:53.503 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:53 smithi087 ceph-mon[39120]: from='mgr.? 172.21.15.148:0/2136774154' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-02-13T14:15:53.503 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:53 smithi087 ceph-mon[39120]: from='mgr.? 172.21.15.148:0/2136774154' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-13T14:15:53.503 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:53 smithi087 ceph-mon[39120]: from='mgr.? 172.21.15.148:0/2136774154' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-02-13T14:15:53.503 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:53 smithi087 ceph-mon[39120]: from='mgr.? 172.21.15.148:0/2136774154' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-13T14:15:53.618 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:53 smithi148 ceph-mon[31198]: Standby manager daemon x restarted 2024-02-13T14:15:53.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:53 smithi148 ceph-mon[31198]: Standby manager daemon x started 2024-02-13T14:15:53.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:53 smithi148 ceph-mon[31198]: from='mgr.? 172.21.15.148:0/2136774154' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-02-13T14:15:53.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:53 smithi148 ceph-mon[31198]: from='mgr.? 172.21.15.148:0/2136774154' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-13T14:15:53.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:53 smithi148 ceph-mon[31198]: from='mgr.? 172.21.15.148:0/2136774154' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-02-13T14:15:53.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:53 smithi148 ceph-mon[31198]: from='mgr.? 172.21.15.148:0/2136774154' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-13T14:15:53.619 INFO:journalctl@ceph.mgr.x.smithi148.stdout:Feb 13 14:15:53 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-x[33278]: 2024-02-13T14:15:53.166+0000 7f00402eb200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-13T14:15:53.619 INFO:journalctl@ceph.mgr.x.smithi148.stdout:Feb 13 14:15:53 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-x[33278]: [13/Feb/2024:14:15:53] ENGINE Bus STARTING 2024-02-13T14:15:53.619 INFO:journalctl@ceph.mgr.x.smithi148.stdout:Feb 13 14:15:53 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-x[33278]: CherryPy Checker: 2024-02-13T14:15:53.619 INFO:journalctl@ceph.mgr.x.smithi148.stdout:Feb 13 14:15:53 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-x[33278]: The Application mounted at '' has an empty config. 2024-02-13T14:15:53.620 INFO:journalctl@ceph.mgr.x.smithi148.stdout:Feb 13 14:15:53 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-x[33278]: 2024-02-13T14:15:53.620 INFO:journalctl@ceph.mgr.x.smithi148.stdout:Feb 13 14:15:53 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-x[33278]: [13/Feb/2024:14:15:53] ENGINE Serving on http://:::9283 2024-02-13T14:15:53.620 INFO:journalctl@ceph.mgr.x.smithi148.stdout:Feb 13 14:15:53 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-x[33278]: [13/Feb/2024:14:15:53] ENGINE Bus STARTED 2024-02-13T14:15:53.817 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:15:53 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: 2024-02-13T14:15:53.704+0000 7f2d98aef200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-13T14:15:54.242 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:15:53 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: [13/Feb/2024:14:15:53] ENGINE Bus STARTING 2024-02-13T14:15:54.242 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:15:53 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: CherryPy Checker: 2024-02-13T14:15:54.242 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:15:53 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: The Application mounted at '' has an empty config. 2024-02-13T14:15:54.242 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:15:53 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: 2024-02-13T14:15:54.242 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:15:53 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: [13/Feb/2024:14:15:53] ENGINE Serving on http://:::9283 2024-02-13T14:15:54.242 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:15:53 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: [13/Feb/2024:14:15:53] ENGINE Bus STARTED 2024-02-13T14:15:54.521 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:54 smithi087 ceph-mon[27372]: mgrmap e18: y(active, since 5m), standbys: x 2024-02-13T14:15:54.521 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:54 smithi087 ceph-mon[27372]: Active manager daemon y restarted 2024-02-13T14:15:54.521 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:54 smithi087 ceph-mon[27372]: Activating manager daemon y 2024-02-13T14:15:54.521 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:54 smithi087 ceph-mon[27372]: osdmap e74: 8 total, 8 up, 8 in 2024-02-13T14:15:54.521 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:54 smithi087 ceph-mon[27372]: mgrmap e19: y(active, starting, since 0.0119708s), standbys: x 2024-02-13T14:15:54.521 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:54 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-13T14:15:54.521 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:54 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-13T14:15:54.522 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:54 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-13T14:15:54.522 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:54 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-13T14:15:54.522 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:54 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-13T14:15:54.522 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:54 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-13T14:15:54.522 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:54 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-13T14:15:54.522 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:54 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-13T14:15:54.522 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:54 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-13T14:15:54.522 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:54 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-13T14:15:54.522 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:54 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-13T14:15:54.522 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:54 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-13T14:15:54.522 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:54 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-13T14:15:54.522 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:54 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-13T14:15:54.523 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:54 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-13T14:15:54.523 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:54 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-13T14:15:54.523 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:54 smithi087 ceph-mon[27372]: Manager daemon y is now available 2024-02-13T14:15:54.523 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:54 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:15:54.523 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:54 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:15:54.523 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:54 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:15:54.523 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:54 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-13T14:15:54.523 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:54 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-13T14:15:54.523 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:54 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-13T14:15:54.523 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:54 smithi087 ceph-mon[39120]: mgrmap e18: y(active, since 5m), standbys: x 2024-02-13T14:15:54.524 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:54 smithi087 ceph-mon[39120]: Active manager daemon y restarted 2024-02-13T14:15:54.524 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:54 smithi087 ceph-mon[39120]: Activating manager daemon y 2024-02-13T14:15:54.524 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:54 smithi087 ceph-mon[39120]: osdmap e74: 8 total, 8 up, 8 in 2024-02-13T14:15:54.524 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:54 smithi087 ceph-mon[39120]: mgrmap e19: y(active, starting, since 0.0119708s), standbys: x 2024-02-13T14:15:54.524 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:54 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-13T14:15:54.524 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:54 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-13T14:15:54.524 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:54 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-13T14:15:54.524 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:54 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-13T14:15:54.524 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:54 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-13T14:15:54.524 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:54 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-13T14:15:54.524 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:54 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-13T14:15:54.524 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:54 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-13T14:15:54.524 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:54 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-13T14:15:54.525 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:54 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-13T14:15:54.525 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:54 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-13T14:15:54.525 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:54 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-13T14:15:54.525 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:54 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-13T14:15:54.525 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:54 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-13T14:15:54.525 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:54 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-13T14:15:54.525 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:54 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-13T14:15:54.525 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:54 smithi087 ceph-mon[39120]: Manager daemon y is now available 2024-02-13T14:15:54.525 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:54 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:15:54.525 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:54 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:15:54.525 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:54 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:15:54.525 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:54 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-13T14:15:54.525 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:54 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-13T14:15:54.525 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:54 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-13T14:15:54.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:54 smithi148 ceph-mon[31198]: mgrmap e18: y(active, since 5m), standbys: x 2024-02-13T14:15:54.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:54 smithi148 ceph-mon[31198]: Active manager daemon y restarted 2024-02-13T14:15:54.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:54 smithi148 ceph-mon[31198]: Activating manager daemon y 2024-02-13T14:15:54.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:54 smithi148 ceph-mon[31198]: osdmap e74: 8 total, 8 up, 8 in 2024-02-13T14:15:54.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:54 smithi148 ceph-mon[31198]: mgrmap e19: y(active, starting, since 0.0119708s), standbys: x 2024-02-13T14:15:54.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:54 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-13T14:15:54.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:54 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-13T14:15:54.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:54 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-13T14:15:54.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:54 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-13T14:15:54.620 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:54 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-13T14:15:54.620 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:54 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-13T14:15:54.620 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:54 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-13T14:15:54.620 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:54 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-13T14:15:54.620 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:54 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-13T14:15:54.620 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:54 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-13T14:15:54.620 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:54 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-13T14:15:54.620 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:54 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-13T14:15:54.620 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:54 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-13T14:15:54.620 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:54 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-13T14:15:54.620 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:54 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-13T14:15:54.620 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:54 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-13T14:15:54.620 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:54 smithi148 ceph-mon[31198]: Manager daemon y is now available 2024-02-13T14:15:54.621 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:54 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:15:54.621 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:54 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:15:54.621 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:54 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:15:54.621 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:54 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-13T14:15:54.621 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:54 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-13T14:15:54.621 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:54 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-13T14:15:55.049 INFO:journalctl@ceph.iscsi.iscsi.a.smithi148.stdout:Feb 13 14:15:54 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-iscsi-iscsi-a[59644]: debug there is no tcmu-runner data available 2024-02-13T14:15:55.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:55 smithi148 ceph-mon[31198]: [13/Feb/2024:14:15:54] ENGINE Bus STARTING 2024-02-13T14:15:55.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:55 smithi148 ceph-mon[31198]: [13/Feb/2024:14:15:54] ENGINE Serving on https://172.21.15.87:7150 2024-02-13T14:15:55.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:55 smithi148 ceph-mon[31198]: mgrmap e20: y(active, since 1.0144s), standbys: x 2024-02-13T14:15:55.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:55 smithi087 ceph-mon[27372]: [13/Feb/2024:14:15:54] ENGINE Bus STARTING 2024-02-13T14:15:55.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:55 smithi087 ceph-mon[27372]: [13/Feb/2024:14:15:54] ENGINE Serving on https://172.21.15.87:7150 2024-02-13T14:15:55.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:55 smithi087 ceph-mon[27372]: mgrmap e20: y(active, since 1.0144s), standbys: x 2024-02-13T14:15:55.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:55 smithi087 ceph-mon[39120]: [13/Feb/2024:14:15:54] ENGINE Bus STARTING 2024-02-13T14:15:55.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:55 smithi087 ceph-mon[39120]: [13/Feb/2024:14:15:54] ENGINE Serving on https://172.21.15.87:7150 2024-02-13T14:15:55.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:55 smithi087 ceph-mon[39120]: mgrmap e20: y(active, since 1.0144s), standbys: x 2024-02-13T14:15:56.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:56 smithi148 ceph-mon[31198]: [13/Feb/2024:14:15:54] ENGINE Serving on http://172.21.15.87:8765 2024-02-13T14:15:56.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:56 smithi148 ceph-mon[31198]: [13/Feb/2024:14:15:54] ENGINE Bus STARTED 2024-02-13T14:15:56.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:56 smithi148 ceph-mon[31198]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:15:56.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:56 smithi148 ceph-mon[31198]: pgmap v3: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-02-13T14:15:56.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:56 smithi087 ceph-mon[27372]: [13/Feb/2024:14:15:54] ENGINE Serving on http://172.21.15.87:8765 2024-02-13T14:15:56.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:56 smithi087 ceph-mon[27372]: [13/Feb/2024:14:15:54] ENGINE Bus STARTED 2024-02-13T14:15:56.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:56 smithi087 ceph-mon[27372]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:15:56.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:56 smithi087 ceph-mon[27372]: pgmap v3: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-02-13T14:15:56.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:56 smithi087 ceph-mon[39120]: [13/Feb/2024:14:15:54] ENGINE Serving on http://172.21.15.87:8765 2024-02-13T14:15:56.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:56 smithi087 ceph-mon[39120]: [13/Feb/2024:14:15:54] ENGINE Bus STARTED 2024-02-13T14:15:56.743 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:56 smithi087 ceph-mon[39120]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:15:56.743 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:56 smithi087 ceph-mon[39120]: pgmap v3: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-02-13T14:15:57.416 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:15:57 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: ::ffff:172.21.15.148 - - [13/Feb/2024:14:15:57] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-13T14:15:57.416 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:57 smithi087 ceph-mon[39120]: pgmap v4: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-02-13T14:15:57.416 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:57 smithi087 ceph-mon[39120]: mgrmap e21: y(active, since 3s), standbys: x 2024-02-13T14:15:57.416 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:57 smithi087 ceph-mon[27372]: pgmap v4: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-02-13T14:15:57.417 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:57 smithi087 ceph-mon[27372]: mgrmap e21: y(active, since 3s), standbys: x 2024-02-13T14:15:57.586 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:57 smithi148 ceph-mon[31198]: pgmap v4: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-02-13T14:15:57.587 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:57 smithi148 ceph-mon[31198]: mgrmap e21: y(active, since 3s), standbys: x 2024-02-13T14:15:58.491 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:58 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:15:58.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:58 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:15:58.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:58 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:15:58.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:58 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:15:58.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:58 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:15:58.620 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:58 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:15:59.618 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:59 smithi148 ceph-mon[31198]: pgmap v5: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-02-13T14:15:59.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:59 smithi148 ceph-mon[31198]: mgrmap e22: y(active, since 4s), standbys: x 2024-02-13T14:15:59.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:59 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:15:59.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:59 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:15:59.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:59 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-13T14:15:59.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:59 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-13T14:15:59.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:59 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-13T14:15:59.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:59 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-13T14:15:59.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:15:59 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:15:59.697 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:59 smithi087 ceph-mon[27372]: pgmap v5: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-02-13T14:15:59.697 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:59 smithi087 ceph-mon[27372]: mgrmap e22: y(active, since 4s), standbys: x 2024-02-13T14:15:59.698 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:59 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:15:59.698 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:59 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:15:59.698 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:59 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-13T14:15:59.698 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:59 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-13T14:15:59.698 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:59 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-13T14:15:59.698 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:59 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-13T14:15:59.698 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:15:59 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:15:59.698 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:59 smithi087 ceph-mon[39120]: pgmap v5: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-02-13T14:15:59.698 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:59 smithi087 ceph-mon[39120]: mgrmap e22: y(active, since 4s), standbys: x 2024-02-13T14:15:59.698 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:59 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:15:59.699 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:59 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:15:59.699 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:59 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-13T14:15:59.699 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:59 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-13T14:15:59.699 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:59 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-13T14:15:59.699 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:59 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-13T14:15:59.699 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:15:59 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:16:00.604 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:16:00 smithi087 ceph-mon[27372]: Adjusting osd_memory_target on smithi148 to 3760M 2024-02-13T14:16:00.604 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:16:00 smithi087 ceph-mon[39120]: Adjusting osd_memory_target on smithi148 to 3760M 2024-02-13T14:16:00.618 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:16:00 smithi148 ceph-mon[31198]: Adjusting osd_memory_target on smithi148 to 3760M 2024-02-13T14:16:01.618 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:16:01 smithi148 ceph-mon[31198]: pgmap v6: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-02-13T14:16:01.631 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:16:01 smithi087 ceph-mon[27372]: pgmap v6: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-02-13T14:16:01.631 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:16:01 smithi087 ceph-mon[39120]: pgmap v6: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-02-13T14:16:02.276 INFO:teuthology.orchestra.run.smithi087.stdout:167503724567 2024-02-13T14:16:02.276 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662 shell --fsid 5c04b074-ca79-11ee-95b9-87774f69a715 -- ceph osd last-stat-seq osd.5 2024-02-13T14:16:03.261 INFO:teuthology.orchestra.run.smithi087.stdout:111669149730 2024-02-13T14:16:03.261 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662 shell --fsid 5c04b074-ca79-11ee-95b9-87774f69a715 -- ceph osd last-stat-seq osd.3 2024-02-13T14:16:03.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:16:03 smithi087 ceph-mon[27372]: pgmap v7: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 30 KiB/s rd, 0 B/s wr, 17 op/s 2024-02-13T14:16:03.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:16:03 smithi087 ceph-mon[39120]: pgmap v7: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 30 KiB/s rd, 0 B/s wr, 17 op/s 2024-02-13T14:16:03.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:16:03 smithi148 ceph-mon[31198]: pgmap v7: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 30 KiB/s rd, 0 B/s wr, 17 op/s 2024-02-13T14:16:05.118 INFO:journalctl@ceph.iscsi.iscsi.a.smithi148.stdout:Feb 13 14:16:04 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-iscsi-iscsi-a[59644]: debug there is no tcmu-runner data available 2024-02-13T14:16:05.265 INFO:teuthology.orchestra.run.smithi087.stdout:240518168590 2024-02-13T14:16:05.265 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662 shell --fsid 5c04b074-ca79-11ee-95b9-87774f69a715 -- ceph osd last-stat-seq osd.7 2024-02-13T14:16:05.491 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:16:05 smithi087 ceph-mon[27372]: pgmap v8: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 24 KiB/s rd, 0 B/s wr, 13 op/s 2024-02-13T14:16:05.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:16:05 smithi087 ceph-mon[39120]: pgmap v8: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 24 KiB/s rd, 0 B/s wr, 13 op/s 2024-02-13T14:16:05.618 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:16:05 smithi148 ceph-mon[31198]: pgmap v8: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 24 KiB/s rd, 0 B/s wr, 13 op/s 2024-02-13T14:16:05.899 INFO:teuthology.orchestra.run.smithi087.stdout:34359738419 2024-02-13T14:16:05.900 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662 shell --fsid 5c04b074-ca79-11ee-95b9-87774f69a715 -- ceph osd last-stat-seq osd.0 2024-02-13T14:16:06.431 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:16:06 smithi087 ceph-mon[27372]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:16:06.432 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:16:06 smithi087 ceph-mon[39120]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:16:06.618 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:16:06 smithi148 ceph-mon[31198]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:16:07.170 INFO:teuthology.orchestra.run.smithi087.stdout:133143986207 2024-02-13T14:16:07.171 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662 shell --fsid 5c04b074-ca79-11ee-95b9-87774f69a715 -- ceph osd last-stat-seq osd.4 2024-02-13T14:16:07.403 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:16:07 smithi087 ceph-mon[39120]: pgmap v9: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 11 op/s 2024-02-13T14:16:07.404 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:16:07 smithi087 ceph-mon[27372]: pgmap v9: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 11 op/s 2024-02-13T14:16:07.404 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:16:07 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: ::ffff:172.21.15.148 - - [13/Feb/2024:14:16:07] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-13T14:16:07.618 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:16:07 smithi148 ceph-mon[31198]: pgmap v9: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 11 op/s 2024-02-13T14:16:08.053 INFO:teuthology.orchestra.run.smithi087.stdout:77309411368 2024-02-13T14:16:08.054 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662 shell --fsid 5c04b074-ca79-11ee-95b9-87774f69a715 -- ceph osd last-stat-seq osd.2 2024-02-13T14:16:08.456 INFO:teuthology.orchestra.run.smithi087.stdout:55834574895 2024-02-13T14:16:08.457 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662 shell --fsid 5c04b074-ca79-11ee-95b9-87774f69a715 -- ceph osd last-stat-seq osd.1 2024-02-13T14:16:08.965 INFO:teuthology.orchestra.run.smithi087.stdout:206158430228 2024-02-13T14:16:08.966 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662 shell --fsid 5c04b074-ca79-11ee-95b9-87774f69a715 -- ceph osd last-stat-seq osd.6 2024-02-13T14:16:09.018 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/mon.a/config 2024-02-13T14:16:09.034 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/mon.a/config 2024-02-13T14:16:09.068 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/mon.a/config 2024-02-13T14:16:09.260 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/mon.a/config 2024-02-13T14:16:09.505 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:16:09 smithi087 ceph-mon[27372]: pgmap v10: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-02-13T14:16:09.506 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:16:09 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-13T14:16:09.506 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:16:09 smithi087 ceph-mon[39120]: pgmap v10: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-02-13T14:16:09.506 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:16:09 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-13T14:16:09.618 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:16:09 smithi148 ceph-mon[31198]: pgmap v10: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-02-13T14:16:09.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:16:09 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-13T14:16:11.618 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:16:11 smithi148 ceph-mon[31198]: pgmap v11: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-02-13T14:16:11.741 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:16:11 smithi087 ceph-mon[39120]: pgmap v11: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-02-13T14:16:11.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:16:11 smithi087 ceph-mon[27372]: pgmap v11: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-02-13T14:16:12.472 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/mon.a/config 2024-02-13T14:16:12.473 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/mon.a/config 2024-02-13T14:16:13.618 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:16:13 smithi148 ceph-mon[31198]: pgmap v12: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-02-13T14:16:13.711 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:16:13 smithi087 ceph-mon[27372]: pgmap v12: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-02-13T14:16:13.712 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:16:13 smithi087 ceph-mon[39120]: pgmap v12: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-02-13T14:16:13.791 INFO:teuthology.orchestra.run.smithi087.stdout:167503724570 2024-02-13T14:16:14.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:16:14 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/2046652207' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-02-13T14:16:14.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:16:14 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/2046652207' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-02-13T14:16:14.618 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:16:14 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/2046652207' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-02-13T14:16:15.118 INFO:journalctl@ceph.iscsi.iscsi.a.smithi148.stdout:Feb 13 14:16:14 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-iscsi-iscsi-a[59644]: debug there is no tcmu-runner data available 2024-02-13T14:16:15.147 INFO:teuthology.orchestra.run.smithi087.stdout:55834574896 2024-02-13T14:16:15.370 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:16:15 smithi087 ceph-mon[27372]: pgmap v13: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-13T14:16:15.370 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:16:15 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/4127687068' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-02-13T14:16:15.370 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:16:15 smithi087 ceph-mon[39120]: pgmap v13: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-13T14:16:15.370 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:16:15 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/4127687068' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-02-13T14:16:15.618 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:16:15 smithi148 ceph-mon[31198]: pgmap v13: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-13T14:16:15.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:16:15 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/4127687068' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-02-13T14:16:16.561 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:16:16 smithi087 ceph-mon[27372]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:16:16.561 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:16:16 smithi087 ceph-mon[39120]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:16:16.618 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:16:16 smithi148 ceph-mon[31198]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:16:17.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:16:17 smithi087 ceph-mon[27372]: pgmap v14: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:16:17.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:16:17 smithi087 ceph-mon[39120]: pgmap v14: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:16:17.492 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:16:17 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: ::ffff:172.21.15.148 - - [13/Feb/2024:14:16:17] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-13T14:16:17.618 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:16:17 smithi148 ceph-mon[31198]: pgmap v14: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:16:17.926 INFO:teuthology.orchestra.run.smithi087.stdout:111669149734 2024-02-13T14:16:18.151 INFO:teuthology.orchestra.run.smithi087.stdout:34359738422 2024-02-13T14:16:18.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:16:18 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/2464713555' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-02-13T14:16:18.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:16:18 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/2377785756' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-02-13T14:16:18.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:16:18 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/2464713555' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-02-13T14:16:18.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:16:18 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/2377785756' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-02-13T14:16:18.604 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149730 got 111669149734 for osd.3 2024-02-13T14:16:18.605 DEBUG:teuthology.parallel:result is None 2024-02-13T14:16:18.618 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:16:18 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/2464713555' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-02-13T14:16:18.618 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:16:18 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/2377785756' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-02-13T14:16:19.372 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724567 got 167503724570 for osd.5 2024-02-13T14:16:19.372 DEBUG:teuthology.parallel:result is None 2024-02-13T14:16:19.618 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:16:19 smithi148 ceph-mon[31198]: pgmap v15: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-13T14:16:19.737 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:16:19 smithi087 ceph-mon[39120]: pgmap v15: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-13T14:16:19.738 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:16:19 smithi087 ceph-mon[27372]: pgmap v15: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-13T14:16:19.764 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574895 got 55834574896 for osd.1 2024-02-13T14:16:19.765 DEBUG:teuthology.parallel:result is None 2024-02-13T14:16:20.648 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738419 got 34359738422 for osd.0 2024-02-13T14:16:20.648 DEBUG:teuthology.parallel:result is None 2024-02-13T14:16:21.354 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/mon.a/config 2024-02-13T14:16:21.355 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/mon.a/config 2024-02-13T14:16:21.374 INFO:teuthology.orchestra.run.smithi087.stdout:240518168594 2024-02-13T14:16:21.607 INFO:teuthology.orchestra.run.smithi087.stdout:133143986210 2024-02-13T14:16:21.626 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:16:21 smithi087 ceph-mon[27372]: pgmap v16: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:16:21.626 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:16:21 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:16:21.627 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:16:21 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:16:21.627 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:16:21 smithi087 ceph-mon[39120]: pgmap v16: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:16:21.627 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:16:21 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:16:21.627 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:16:21 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:16:21.627 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:16:21 smithi148 ceph-mon[31198]: pgmap v16: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:16:21.627 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:16:21 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:16:21.627 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:16:21 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:16:22.477 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:16:22 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/502440251' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-02-13T14:16:22.477 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:16:22 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/887511546' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-02-13T14:16:22.477 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:16:22 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/502440251' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-02-13T14:16:22.477 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:16:22 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/887511546' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-02-13T14:16:22.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:16:22 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/502440251' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-02-13T14:16:22.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:16:22 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/887511546' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-02-13T14:16:23.618 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:16:23 smithi148 ceph-mon[31198]: pgmap v17: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:16:23.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:16:23 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:16:23.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:16:23 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:16:23.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:16:23 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-13T14:16:23.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:16:23 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-13T14:16:23.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:16:23 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-13T14:16:23.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:16:23 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-13T14:16:23.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:16:23 smithi148 ceph-mon[31198]: Adjusting osd_memory_target on smithi087 to 3760M 2024-02-13T14:16:23.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:16:23 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:16:23.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:16:23 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:16:23.620 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:16:23 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:16:23.620 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:16:23 smithi148 ceph-mon[31198]: Updating smithi087:/etc/ceph/ceph.conf 2024-02-13T14:16:23.620 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:16:23 smithi148 ceph-mon[31198]: Updating smithi148:/etc/ceph/ceph.conf 2024-02-13T14:16:23.620 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:16:23 smithi148 ceph-mon[31198]: Updating smithi087:/var/lib/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/config/ceph.conf 2024-02-13T14:16:23.620 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:16:23 smithi148 ceph-mon[31198]: Updating smithi148:/var/lib/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/config/ceph.conf 2024-02-13T14:16:23.620 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:16:23 smithi148 ceph-mon[31198]: Updating smithi087:/etc/ceph/ceph.client.admin.keyring 2024-02-13T14:16:23.620 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:16:23 smithi148 ceph-mon[31198]: Updating smithi148:/etc/ceph/ceph.client.admin.keyring 2024-02-13T14:16:23.620 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:16:23 smithi148 ceph-mon[31198]: Updating smithi087:/var/lib/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/config/ceph.client.admin.keyring 2024-02-13T14:16:23.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:16:23 smithi087 ceph-mon[27372]: pgmap v17: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:16:23.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:16:23 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:16:23.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:16:23 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:16:23.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:16:23 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-13T14:16:23.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:16:23 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-13T14:16:23.743 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:16:23 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-13T14:16:23.743 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:16:23 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-13T14:16:23.743 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:16:23 smithi087 ceph-mon[27372]: Adjusting osd_memory_target on smithi087 to 3760M 2024-02-13T14:16:23.743 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:16:23 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:16:23.743 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:16:23 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:16:23.743 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:16:23 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:16:23.743 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:16:23 smithi087 ceph-mon[27372]: Updating smithi087:/etc/ceph/ceph.conf 2024-02-13T14:16:23.743 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:16:23 smithi087 ceph-mon[27372]: Updating smithi148:/etc/ceph/ceph.conf 2024-02-13T14:16:23.743 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:16:23 smithi087 ceph-mon[27372]: Updating smithi087:/var/lib/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/config/ceph.conf 2024-02-13T14:16:23.743 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:16:23 smithi087 ceph-mon[27372]: Updating smithi148:/var/lib/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/config/ceph.conf 2024-02-13T14:16:23.743 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:16:23 smithi087 ceph-mon[27372]: Updating smithi087:/etc/ceph/ceph.client.admin.keyring 2024-02-13T14:16:23.743 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:16:23 smithi087 ceph-mon[27372]: Updating smithi148:/etc/ceph/ceph.client.admin.keyring 2024-02-13T14:16:23.743 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:16:23 smithi087 ceph-mon[27372]: Updating smithi087:/var/lib/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/config/ceph.client.admin.keyring 2024-02-13T14:16:23.744 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:16:23 smithi087 ceph-mon[39120]: pgmap v17: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:16:23.744 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:16:23 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:16:23.744 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:16:23 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:16:23.744 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:16:23 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-13T14:16:23.744 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:16:23 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-13T14:16:23.744 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:16:23 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-13T14:16:23.744 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:16:23 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-13T14:16:23.744 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:16:23 smithi087 ceph-mon[39120]: Adjusting osd_memory_target on smithi087 to 3760M 2024-02-13T14:16:23.744 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:16:23 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:16:23.744 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:16:23 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:16:23.744 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:16:23 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:16:23.744 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:16:23 smithi087 ceph-mon[39120]: Updating smithi087:/etc/ceph/ceph.conf 2024-02-13T14:16:23.745 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:16:23 smithi087 ceph-mon[39120]: Updating smithi148:/etc/ceph/ceph.conf 2024-02-13T14:16:23.745 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:16:23 smithi087 ceph-mon[39120]: Updating smithi087:/var/lib/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/config/ceph.conf 2024-02-13T14:16:23.745 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:16:23 smithi087 ceph-mon[39120]: Updating smithi148:/var/lib/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/config/ceph.conf 2024-02-13T14:16:23.745 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:16:23 smithi087 ceph-mon[39120]: Updating smithi087:/etc/ceph/ceph.client.admin.keyring 2024-02-13T14:16:23.745 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:16:23 smithi087 ceph-mon[39120]: Updating smithi148:/etc/ceph/ceph.client.admin.keyring 2024-02-13T14:16:23.745 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:16:23 smithi087 ceph-mon[39120]: Updating smithi087:/var/lib/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/config/ceph.client.admin.keyring 2024-02-13T14:16:24.212 INFO:tasks.cephadm.ceph_manager.ceph:need seq 133143986207 got 133143986210 for osd.4 2024-02-13T14:16:24.212 DEBUG:teuthology.parallel:result is None 2024-02-13T14:16:24.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:16:24 smithi087 ceph-mon[39120]: Updating smithi148:/var/lib/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/config/ceph.client.admin.keyring 2024-02-13T14:16:24.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:16:24 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:16:24.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:16:24 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:16:24.743 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:16:24 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:16:24.743 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:16:24 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:16:24.743 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:16:24 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:16:24.743 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:16:24 smithi087 ceph-mon[39120]: Deploying daemon alertmanager.a on smithi087 2024-02-13T14:16:24.743 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:16:24 smithi087 ceph-mon[39120]: pgmap v18: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-13T14:16:24.743 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:16:24 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-13T14:16:24.743 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:16:24 smithi087 ceph-mon[27372]: Updating smithi148:/var/lib/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/config/ceph.client.admin.keyring 2024-02-13T14:16:24.744 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:16:24 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:16:24.744 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:16:24 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:16:24.744 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:16:24 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:16:24.744 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:16:24 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:16:24.744 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:16:24 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:16:24.744 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:16:24 smithi087 ceph-mon[27372]: Deploying daemon alertmanager.a on smithi087 2024-02-13T14:16:24.744 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:16:24 smithi087 ceph-mon[27372]: pgmap v18: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-13T14:16:24.745 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:16:24 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-13T14:16:24.763 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:16:24 smithi148 ceph-mon[31198]: Updating smithi148:/var/lib/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/config/ceph.client.admin.keyring 2024-02-13T14:16:24.763 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:16:24 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:16:24.763 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:16:24 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:16:24.763 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:16:24 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:16:24.763 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:16:24 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:16:24.763 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:16:24 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:16:24.763 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:16:24 smithi148 ceph-mon[31198]: Deploying daemon alertmanager.a on smithi087 2024-02-13T14:16:24.763 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:16:24 smithi148 ceph-mon[31198]: pgmap v18: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-13T14:16:24.763 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:16:24 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-13T14:16:25.118 INFO:journalctl@ceph.iscsi.iscsi.a.smithi148.stdout:Feb 13 14:16:24 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-iscsi-iscsi-a[59644]: debug there is no tcmu-runner data available 2024-02-13T14:16:25.248 INFO:teuthology.orchestra.run.smithi087.stdout:77309411372 2024-02-13T14:16:25.991 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:16:25 smithi087 ceph-mon[39120]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:16:25.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:16:25 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/1224047231' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-02-13T14:16:25.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:16:25 smithi087 ceph-mon[27372]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:16:25.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:16:25 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/1224047231' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-02-13T14:16:26.020 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:16:25 smithi148 ceph-mon[31198]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:16:26.020 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:16:25 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/1224047231' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-02-13T14:16:26.023 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411368 got 77309411372 for osd.2 2024-02-13T14:16:26.023 DEBUG:teuthology.parallel:result is None 2024-02-13T14:16:26.458 INFO:tasks.cephadm.ceph_manager.ceph:need seq 240518168590 got 240518168594 for osd.7 2024-02-13T14:16:26.458 DEBUG:teuthology.parallel:result is None 2024-02-13T14:16:26.741 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:16:26 smithi087 ceph-mon[27372]: pgmap v19: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:16:26.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:16:26 smithi087 ceph-mon[39120]: pgmap v19: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:16:26.890 INFO:teuthology.orchestra.run.smithi087.stdout:206158430233 2024-02-13T14:16:27.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:16:26 smithi148 ceph-mon[31198]: pgmap v19: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:16:27.492 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:16:27 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: ::ffff:172.21.15.148 - - [13/Feb/2024:14:16:27] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-13T14:16:27.899 INFO:tasks.cephadm.ceph_manager.ceph:need seq 206158430228 got 206158430233 for osd.6 2024-02-13T14:16:27.899 DEBUG:teuthology.parallel:result is None 2024-02-13T14:16:27.899 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-02-13T14:16:27.899 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662 shell --fsid 5c04b074-ca79-11ee-95b9-87774f69a715 -- ceph pg dump --format=json 2024-02-13T14:16:27.924 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:16:27 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/4051788047' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-02-13T14:16:27.925 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:16:27 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/4051788047' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-02-13T14:16:28.118 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:16:27 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/4051788047' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-02-13T14:16:28.140 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/mon.a/config 2024-02-13T14:16:28.991 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:16:28 smithi087 ceph-mon[39120]: pgmap v20: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-13T14:16:28.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:16:28 smithi087 ceph-mon[27372]: pgmap v20: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-13T14:16:29.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:16:28 smithi148 ceph-mon[31198]: pgmap v20: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-13T14:16:30.386 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-02-13T14:16:30.389 INFO:teuthology.orchestra.run.smithi087.stderr:dumped all 2024-02-13T14:16:31.118 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:16:30 smithi148 ceph-mon[31198]: pgmap v21: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:16:31.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:16:30 smithi087 ceph-mon[27372]: pgmap v21: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:16:31.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:16:30 smithi087 ceph-mon[39120]: pgmap v21: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:16:31.725 INFO:teuthology.orchestra.run.smithi087.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-02-13T14:16:29.743942+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596316,"num_objects":198,"num_object_clones":0,"num_object_copies":594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":198,"num_whiteouts":0,"num_read":1005,"num_read_kb":851,"num_write":615,"num_write_kb":2461,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":1771544,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":600,"ondisk_log_size":600,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":249364,"kb_used_data":8252,"kb_used_omap":12,"kb_used_meta":241011,"kb_avail":749711852,"statfs":{"total":767960285184,"available":767704936448,"internally_reserved":0,"allocated":8450048,"data_stored":4542164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":246795824},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002676"},"pg_stats":[{"pgid":"6.1a","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:54.276605+0000","last_change":"2024-02-13T14:15:18.215385+0000","last_active":"2024-02-13T14:15:54.276605+0000","last_peered":"2024-02-13T14:15:54.276605+0000","last_clean":"2024-02-13T14:15:54.276605+0000","last_became_active":"2024-02-13T14:15:18.215083+0000","last_became_peered":"2024-02-13T14:15:18.215083+0000","last_unstale":"2024-02-13T14:15:54.276605+0000","last_undegraded":"2024-02-13T14:15:54.276605+0000","last_fullsized":"2024-02-13T14:15:54.276605+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:17.185096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:17.185096+0000","last_clean_scrub_stamp":"2024-02-13T14:15:17.185096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T01:51:24.997993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:54.381794+0000","last_change":"2024-02-13T14:15:16.197449+0000","last_active":"2024-02-13T14:15:54.381794+0000","last_peered":"2024-02-13T14:15:54.381794+0000","last_clean":"2024-02-13T14:15:54.381794+0000","last_became_active":"2024-02-13T14:15:16.197108+0000","last_became_peered":"2024-02-13T14:15:16.197108+0000","last_unstale":"2024-02-13T14:15:54.381794+0000","last_undegraded":"2024-02-13T14:15:54.381794+0000","last_fullsized":"2024-02-13T14:15:54.381794+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:15.174019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:15.174019+0000","last_clean_scrub_stamp":"2024-02-13T14:15:15.174019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T15:09:12.659558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:54.772654+0000","last_change":"2024-02-13T14:15:12.416660+0000","last_active":"2024-02-13T14:15:54.772654+0000","last_peered":"2024-02-13T14:15:54.772654+0000","last_clean":"2024-02-13T14:15:54.772654+0000","last_became_active":"2024-02-13T14:15:12.416368+0000","last_became_peered":"2024-02-13T14:15:12.416368+0000","last_unstale":"2024-02-13T14:15:54.772654+0000","last_undegraded":"2024-02-13T14:15:54.772654+0000","last_fullsized":"2024-02-13T14:15:54.772654+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:11.150966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:11.150966+0000","last_clean_scrub_stamp":"2024-02-13T14:15:11.150966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T21:46:08.382574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"70'9","reported_seq":45,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:54.276057+0000","last_change":"2024-02-13T14:15:14.182625+0000","last_active":"2024-02-13T14:15:54.276057+0000","last_peered":"2024-02-13T14:15:54.276057+0000","last_clean":"2024-02-13T14:15:54.276057+0000","last_became_active":"2024-02-13T14:15:14.182282+0000","last_became_peered":"2024-02-13T14:15:14.182282+0000","last_unstale":"2024-02-13T14:15:54.276057+0000","last_undegraded":"2024-02-13T14:15:54.276057+0000","last_fullsized":"2024-02-13T14:15:54.276057+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:13.158947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:13.158947+0000","last_clean_scrub_stamp":"2024-02-13T14:15:13.158947+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T17:52:21.022729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.727607+0000","last_change":"2024-02-13T14:15:18.875889+0000","last_active":"2024-02-13T14:15:53.727607+0000","last_peered":"2024-02-13T14:15:53.727607+0000","last_clean":"2024-02-13T14:15:53.727607+0000","last_became_active":"2024-02-13T14:15:18.875348+0000","last_became_peered":"2024-02-13T14:15:18.875348+0000","last_unstale":"2024-02-13T14:15:53.727607+0000","last_undegraded":"2024-02-13T14:15:53.727607+0000","last_fullsized":"2024-02-13T14:15:53.727607+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:17.185096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:17.185096+0000","last_clean_scrub_stamp":"2024-02-13T14:15:17.185096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T18:21:06.075573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:54.276453+0000","last_change":"2024-02-13T14:15:16.197696+0000","last_active":"2024-02-13T14:15:54.276453+0000","last_peered":"2024-02-13T14:15:54.276453+0000","last_clean":"2024-02-13T14:15:54.276453+0000","last_became_active":"2024-02-13T14:15:16.197348+0000","last_became_peered":"2024-02-13T14:15:16.197348+0000","last_unstale":"2024-02-13T14:15:54.276453+0000","last_undegraded":"2024-02-13T14:15:54.276453+0000","last_fullsized":"2024-02-13T14:15:54.276453+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:15.174019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:15.174019+0000","last_clean_scrub_stamp":"2024-02-13T14:15:15.174019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T00:05:58.719527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.728384+0000","last_change":"2024-02-13T14:15:12.417522+0000","last_active":"2024-02-13T14:15:53.728384+0000","last_peered":"2024-02-13T14:15:53.728384+0000","last_clean":"2024-02-13T14:15:53.728384+0000","last_became_active":"2024-02-13T14:15:12.416419+0000","last_became_peered":"2024-02-13T14:15:12.416419+0000","last_unstale":"2024-02-13T14:15:53.728384+0000","last_undegraded":"2024-02-13T14:15:53.728384+0000","last_fullsized":"2024-02-13T14:15:53.728384+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:11.150966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:11.150966+0000","last_clean_scrub_stamp":"2024-02-13T14:15:11.150966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T18:01:22.555152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"70'15","reported_seq":54,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.727784+0000","last_change":"2024-02-13T14:15:14.173684+0000","last_active":"2024-02-13T14:15:53.727784+0000","last_peered":"2024-02-13T14:15:53.727784+0000","last_clean":"2024-02-13T14:15:53.727784+0000","last_became_active":"2024-02-13T14:15:14.173365+0000","last_became_peered":"2024-02-13T14:15:14.173365+0000","last_unstale":"2024-02-13T14:15:53.727784+0000","last_undegraded":"2024-02-13T14:15:53.727784+0000","last_fullsized":"2024-02-13T14:15:53.727784+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:13.158947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:13.158947+0000","last_clean_scrub_stamp":"2024-02-13T14:15:13.158947+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T20:34:45.824272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:54.772164+0000","last_change":"2024-02-13T14:15:18.215613+0000","last_active":"2024-02-13T14:15:54.772164+0000","last_peered":"2024-02-13T14:15:54.772164+0000","last_clean":"2024-02-13T14:15:54.772164+0000","last_became_active":"2024-02-13T14:15:18.215306+0000","last_became_peered":"2024-02-13T14:15:18.215306+0000","last_unstale":"2024-02-13T14:15:54.772164+0000","last_undegraded":"2024-02-13T14:15:54.772164+0000","last_fullsized":"2024-02-13T14:15:54.772164+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:17.185096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:17.185096+0000","last_clean_scrub_stamp":"2024-02-13T14:15:17.185096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T14:25:01.251773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.832598+0000","last_change":"2024-02-13T14:15:16.204071+0000","last_active":"2024-02-13T14:15:53.832598+0000","last_peered":"2024-02-13T14:15:53.832598+0000","last_clean":"2024-02-13T14:15:53.832598+0000","last_became_active":"2024-02-13T14:15:16.203759+0000","last_became_peered":"2024-02-13T14:15:16.203759+0000","last_unstale":"2024-02-13T14:15:53.832598+0000","last_undegraded":"2024-02-13T14:15:53.832598+0000","last_fullsized":"2024-02-13T14:15:53.832598+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:15.174019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:15.174019+0000","last_clean_scrub_stamp":"2024-02-13T14:15:15.174019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T19:28:19.698492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.832942+0000","last_change":"2024-02-13T14:15:12.175280+0000","last_active":"2024-02-13T14:15:53.832942+0000","last_peered":"2024-02-13T14:15:53.832942+0000","last_clean":"2024-02-13T14:15:53.832942+0000","last_became_active":"2024-02-13T14:15:12.174975+0000","last_became_peered":"2024-02-13T14:15:12.174975+0000","last_unstale":"2024-02-13T14:15:53.832942+0000","last_undegraded":"2024-02-13T14:15:53.832942+0000","last_fullsized":"2024-02-13T14:15:53.832942+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:11.150966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:11.150966+0000","last_clean_scrub_stamp":"2024-02-13T14:15:11.150966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T20:13:22.230505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"70'9","reported_seq":45,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:54.276060+0000","last_change":"2024-02-13T14:15:14.175228+0000","last_active":"2024-02-13T14:15:54.276060+0000","last_peered":"2024-02-13T14:15:54.276060+0000","last_clean":"2024-02-13T14:15:54.276060+0000","last_became_active":"2024-02-13T14:15:14.174677+0000","last_became_peered":"2024-02-13T14:15:14.174677+0000","last_unstale":"2024-02-13T14:15:54.276060+0000","last_undegraded":"2024-02-13T14:15:54.276060+0000","last_fullsized":"2024-02-13T14:15:54.276060+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:13.158947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:13.158947+0000","last_clean_scrub_stamp":"2024-02-13T14:15:13.158947+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T18:49:31.417441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.832366+0000","last_change":"2024-02-13T14:15:18.213356+0000","last_active":"2024-02-13T14:15:53.832366+0000","last_peered":"2024-02-13T14:15:53.832366+0000","last_clean":"2024-02-13T14:15:53.832366+0000","last_became_active":"2024-02-13T14:15:18.212835+0000","last_became_peered":"2024-02-13T14:15:18.212835+0000","last_unstale":"2024-02-13T14:15:53.832366+0000","last_undegraded":"2024-02-13T14:15:53.832366+0000","last_fullsized":"2024-02-13T14:15:53.832366+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:17.185096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:17.185096+0000","last_clean_scrub_stamp":"2024-02-13T14:15:17.185096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T22:35:59.324232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.727257+0000","last_change":"2024-02-13T14:15:16.200284+0000","last_active":"2024-02-13T14:15:53.727257+0000","last_peered":"2024-02-13T14:15:53.727257+0000","last_clean":"2024-02-13T14:15:53.727257+0000","last_became_active":"2024-02-13T14:15:16.199987+0000","last_became_peered":"2024-02-13T14:15:16.199987+0000","last_unstale":"2024-02-13T14:15:53.727257+0000","last_undegraded":"2024-02-13T14:15:53.727257+0000","last_fullsized":"2024-02-13T14:15:53.727257+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:15.174019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:15.174019+0000","last_clean_scrub_stamp":"2024-02-13T14:15:15.174019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T00:02:15.562600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.832679+0000","last_change":"2024-02-13T14:15:12.171453+0000","last_active":"2024-02-13T14:15:53.832679+0000","last_peered":"2024-02-13T14:15:53.832679+0000","last_clean":"2024-02-13T14:15:53.832679+0000","last_became_active":"2024-02-13T14:15:12.171075+0000","last_became_peered":"2024-02-13T14:15:12.171075+0000","last_unstale":"2024-02-13T14:15:53.832679+0000","last_undegraded":"2024-02-13T14:15:53.832679+0000","last_fullsized":"2024-02-13T14:15:53.832679+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:11.150966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:11.150966+0000","last_clean_scrub_stamp":"2024-02-13T14:15:11.150966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T19:41:07.694427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"70'5","reported_seq":39,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:54.276134+0000","last_change":"2024-02-13T14:15:14.174951+0000","last_active":"2024-02-13T14:15:54.276134+0000","last_peered":"2024-02-13T14:15:54.276134+0000","last_clean":"2024-02-13T14:15:54.276134+0000","last_became_active":"2024-02-13T14:15:14.174622+0000","last_became_peered":"2024-02-13T14:15:14.174622+0000","last_unstale":"2024-02-13T14:15:54.276134+0000","last_undegraded":"2024-02-13T14:15:54.276134+0000","last_fullsized":"2024-02-13T14:15:54.276134+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:13.158947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:13.158947+0000","last_clean_scrub_stamp":"2024-02-13T14:15:13.158947+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T23:02:13.770652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:54.277087+0000","last_change":"2024-02-13T14:15:18.874263+0000","last_active":"2024-02-13T14:15:54.277087+0000","last_peered":"2024-02-13T14:15:54.277087+0000","last_clean":"2024-02-13T14:15:54.277087+0000","last_became_active":"2024-02-13T14:15:18.873241+0000","last_became_peered":"2024-02-13T14:15:18.873241+0000","last_unstale":"2024-02-13T14:15:54.277087+0000","last_undegraded":"2024-02-13T14:15:54.277087+0000","last_fullsized":"2024-02-13T14:15:54.277087+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:17.185096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:17.185096+0000","last_clean_scrub_stamp":"2024-02-13T14:15:17.185096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T01:26:39.592611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:54.382447+0000","last_change":"2024-02-13T14:15:16.194648+0000","last_active":"2024-02-13T14:15:54.382447+0000","last_peered":"2024-02-13T14:15:54.382447+0000","last_clean":"2024-02-13T14:15:54.382447+0000","last_became_active":"2024-02-13T14:15:16.193624+0000","last_became_peered":"2024-02-13T14:15:16.193624+0000","last_unstale":"2024-02-13T14:15:54.382447+0000","last_undegraded":"2024-02-13T14:15:54.382447+0000","last_fullsized":"2024-02-13T14:15:54.382447+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:15.174019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:15.174019+0000","last_clean_scrub_stamp":"2024-02-13T14:15:15.174019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T16:13:17.644055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:54.772619+0000","last_change":"2024-02-13T14:15:12.175084+0000","last_active":"2024-02-13T14:15:54.772619+0000","last_peered":"2024-02-13T14:15:54.772619+0000","last_clean":"2024-02-13T14:15:54.772619+0000","last_became_active":"2024-02-13T14:15:12.174565+0000","last_became_peered":"2024-02-13T14:15:12.174565+0000","last_unstale":"2024-02-13T14:15:54.772619+0000","last_undegraded":"2024-02-13T14:15:54.772619+0000","last_fullsized":"2024-02-13T14:15:54.772619+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:11.150966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:11.150966+0000","last_clean_scrub_stamp":"2024-02-13T14:15:11.150966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T17:36:04.928340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"70'15","reported_seq":54,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.838258+0000","last_change":"2024-02-13T14:15:14.176409+0000","last_active":"2024-02-13T14:15:53.838258+0000","last_peered":"2024-02-13T14:15:53.838258+0000","last_clean":"2024-02-13T14:15:53.838258+0000","last_became_active":"2024-02-13T14:15:14.176071+0000","last_became_peered":"2024-02-13T14:15:14.176071+0000","last_unstale":"2024-02-13T14:15:53.838258+0000","last_undegraded":"2024-02-13T14:15:53.838258+0000","last_fullsized":"2024-02-13T14:15:53.838258+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:13.158947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:13.158947+0000","last_clean_scrub_stamp":"2024-02-13T14:15:13.158947+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T15:58:59.517593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.728628+0000","last_change":"2024-02-13T14:15:18.873756+0000","last_active":"2024-02-13T14:15:53.728628+0000","last_peered":"2024-02-13T14:15:53.728628+0000","last_clean":"2024-02-13T14:15:53.728628+0000","last_became_active":"2024-02-13T14:15:18.873280+0000","last_became_peered":"2024-02-13T14:15:18.873280+0000","last_unstale":"2024-02-13T14:15:53.728628+0000","last_undegraded":"2024-02-13T14:15:53.728628+0000","last_fullsized":"2024-02-13T14:15:53.728628+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:17.185096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:17.185096+0000","last_clean_scrub_stamp":"2024-02-13T14:15:17.185096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T01:44:10.218859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:54.276823+0000","last_change":"2024-02-13T14:15:16.199416+0000","last_active":"2024-02-13T14:15:54.276823+0000","last_peered":"2024-02-13T14:15:54.276823+0000","last_clean":"2024-02-13T14:15:54.276823+0000","last_became_active":"2024-02-13T14:15:16.199144+0000","last_became_peered":"2024-02-13T14:15:16.199144+0000","last_unstale":"2024-02-13T14:15:54.276823+0000","last_undegraded":"2024-02-13T14:15:54.276823+0000","last_fullsized":"2024-02-13T14:15:54.276823+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:15.174019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:15.174019+0000","last_clean_scrub_stamp":"2024-02-13T14:15:15.174019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T17:20:20.840702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:54.276761+0000","last_change":"2024-02-13T14:15:12.417462+0000","last_active":"2024-02-13T14:15:54.276761+0000","last_peered":"2024-02-13T14:15:54.276761+0000","last_clean":"2024-02-13T14:15:54.276761+0000","last_became_active":"2024-02-13T14:15:12.416542+0000","last_became_peered":"2024-02-13T14:15:12.416542+0000","last_unstale":"2024-02-13T14:15:54.276761+0000","last_undegraded":"2024-02-13T14:15:54.276761+0000","last_fullsized":"2024-02-13T14:15:54.276761+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:11.150966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:11.150966+0000","last_clean_scrub_stamp":"2024-02-13T14:15:11.150966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T01:29:09.612554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"70'12","reported_seq":52,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.727941+0000","last_change":"2024-02-13T14:15:14.176872+0000","last_active":"2024-02-13T14:15:53.727941+0000","last_peered":"2024-02-13T14:15:53.727941+0000","last_clean":"2024-02-13T14:15:53.727941+0000","last_became_active":"2024-02-13T14:15:14.176550+0000","last_became_peered":"2024-02-13T14:15:14.176550+0000","last_unstale":"2024-02-13T14:15:53.727941+0000","last_undegraded":"2024-02-13T14:15:53.727941+0000","last_fullsized":"2024-02-13T14:15:53.727941+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:13.158947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:13.158947+0000","last_clean_scrub_stamp":"2024-02-13T14:15:13.158947+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T16:24:43.824185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"70'1","reported_seq":22,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.727510+0000","last_change":"2024-02-13T14:15:18.219104+0000","last_active":"2024-02-13T14:15:53.727510+0000","last_peered":"2024-02-13T14:15:53.727510+0000","last_clean":"2024-02-13T14:15:53.727510+0000","last_became_active":"2024-02-13T14:15:18.218782+0000","last_became_peered":"2024-02-13T14:15:18.218782+0000","last_unstale":"2024-02-13T14:15:53.727510+0000","last_undegraded":"2024-02-13T14:15:53.727510+0000","last_fullsized":"2024-02-13T14:15:53.727510+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:17.185096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:17.185096+0000","last_clean_scrub_stamp":"2024-02-13T14:15:17.185096+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T14:39:58.906671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"70'8","reported_seq":52,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:16:22.467829+0000","last_change":"2024-02-13T14:15:16.201480+0000","last_active":"2024-02-13T14:16:22.467829+0000","last_peered":"2024-02-13T14:16:22.467829+0000","last_clean":"2024-02-13T14:16:22.467829+0000","last_became_active":"2024-02-13T14:15:16.200392+0000","last_became_peered":"2024-02-13T14:15:16.200392+0000","last_unstale":"2024-02-13T14:16:22.467829+0000","last_undegraded":"2024-02-13T14:16:22.467829+0000","last_fullsized":"2024-02-13T14:16:22.467829+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:15.174019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:15.174019+0000","last_clean_scrub_stamp":"2024-02-13T14:15:15.174019+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T14:41:31.533705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"63'1","reported_seq":34,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:54.381915+0000","last_change":"2024-02-13T14:15:12.167170+0000","last_active":"2024-02-13T14:15:54.381915+0000","last_peered":"2024-02-13T14:15:54.381915+0000","last_clean":"2024-02-13T14:15:54.381915+0000","last_became_active":"2024-02-13T14:15:12.166788+0000","last_became_peered":"2024-02-13T14:15:12.166788+0000","last_unstale":"2024-02-13T14:15:54.381915+0000","last_undegraded":"2024-02-13T14:15:54.381915+0000","last_fullsized":"2024-02-13T14:15:54.381915+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:11.150966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:11.150966+0000","last_clean_scrub_stamp":"2024-02-13T14:15:11.150966+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T19:28:25.682466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"70'10","reported_seq":44,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:54.772065+0000","last_change":"2024-02-13T14:15:14.333051+0000","last_active":"2024-02-13T14:15:54.772065+0000","last_peered":"2024-02-13T14:15:54.772065+0000","last_clean":"2024-02-13T14:15:54.772065+0000","last_became_active":"2024-02-13T14:15:14.332735+0000","last_became_peered":"2024-02-13T14:15:14.332735+0000","last_unstale":"2024-02-13T14:15:54.772065+0000","last_undegraded":"2024-02-13T14:15:54.772065+0000","last_fullsized":"2024-02-13T14:15:54.772065+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:13.158947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:13.158947+0000","last_clean_scrub_stamp":"2024-02-13T14:15:13.158947+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T15:51:21.504527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:54.382509+0000","last_change":"2024-02-13T14:15:18.214477+0000","last_active":"2024-02-13T14:15:54.382509+0000","last_peered":"2024-02-13T14:15:54.382509+0000","last_clean":"2024-02-13T14:15:54.382509+0000","last_became_active":"2024-02-13T14:15:18.214036+0000","last_became_peered":"2024-02-13T14:15:18.214036+0000","last_unstale":"2024-02-13T14:15:54.382509+0000","last_undegraded":"2024-02-13T14:15:54.382509+0000","last_fullsized":"2024-02-13T14:15:54.382509+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:17.185096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:17.185096+0000","last_clean_scrub_stamp":"2024-02-13T14:15:17.185096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T19:01:01.536932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:54.772769+0000","last_change":"2024-02-13T14:15:16.201752+0000","last_active":"2024-02-13T14:15:54.772769+0000","last_peered":"2024-02-13T14:15:54.772769+0000","last_clean":"2024-02-13T14:15:54.772769+0000","last_became_active":"2024-02-13T14:15:16.201317+0000","last_became_peered":"2024-02-13T14:15:16.201317+0000","last_unstale":"2024-02-13T14:15:54.772769+0000","last_undegraded":"2024-02-13T14:15:54.772769+0000","last_fullsized":"2024-02-13T14:15:54.772769+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:15.174019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:15.174019+0000","last_clean_scrub_stamp":"2024-02-13T14:15:15.174019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T23:23:53.006168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.726540+0000","last_change":"2024-02-13T14:15:12.171305+0000","last_active":"2024-02-13T14:15:53.726540+0000","last_peered":"2024-02-13T14:15:53.726540+0000","last_clean":"2024-02-13T14:15:53.726540+0000","last_became_active":"2024-02-13T14:15:12.170518+0000","last_became_peered":"2024-02-13T14:15:12.170518+0000","last_unstale":"2024-02-13T14:15:53.726540+0000","last_undegraded":"2024-02-13T14:15:53.726540+0000","last_fullsized":"2024-02-13T14:15:53.726540+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:11.150966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:11.150966+0000","last_clean_scrub_stamp":"2024-02-13T14:15:11.150966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T15:37:46.154891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"70'11","reported_seq":48,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.730891+0000","last_change":"2024-02-13T14:15:14.180906+0000","last_active":"2024-02-13T14:15:53.730891+0000","last_peered":"2024-02-13T14:15:53.730891+0000","last_clean":"2024-02-13T14:15:53.730891+0000","last_became_active":"2024-02-13T14:15:14.180481+0000","last_became_peered":"2024-02-13T14:15:14.180481+0000","last_unstale":"2024-02-13T14:15:53.730891+0000","last_undegraded":"2024-02-13T14:15:53.730891+0000","last_fullsized":"2024-02-13T14:15:53.730891+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:13.158947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:13.158947+0000","last_clean_scrub_stamp":"2024-02-13T14:15:13.158947+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T01:36:05.342954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.727598+0000","last_change":"2024-02-13T14:15:18.219071+0000","last_active":"2024-02-13T14:15:53.727598+0000","last_peered":"2024-02-13T14:15:53.727598+0000","last_clean":"2024-02-13T14:15:53.727598+0000","last_became_active":"2024-02-13T14:15:18.218760+0000","last_became_peered":"2024-02-13T14:15:18.218760+0000","last_unstale":"2024-02-13T14:15:53.727598+0000","last_undegraded":"2024-02-13T14:15:53.727598+0000","last_fullsized":"2024-02-13T14:15:53.727598+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:17.185096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:17.185096+0000","last_clean_scrub_stamp":"2024-02-13T14:15:17.185096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T00:29:40.070824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.728909+0000","last_change":"2024-02-13T14:15:16.199663+0000","last_active":"2024-02-13T14:15:53.728909+0000","last_peered":"2024-02-13T14:15:53.728909+0000","last_clean":"2024-02-13T14:15:53.728909+0000","last_became_active":"2024-02-13T14:15:16.199341+0000","last_became_peered":"2024-02-13T14:15:16.199341+0000","last_unstale":"2024-02-13T14:15:53.728909+0000","last_undegraded":"2024-02-13T14:15:53.728909+0000","last_fullsized":"2024-02-13T14:15:53.728909+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:15.174019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:15.174019+0000","last_clean_scrub_stamp":"2024-02-13T14:15:15.174019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T15:41:09.625967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:54.772621+0000","last_change":"2024-02-13T14:15:12.173489+0000","last_active":"2024-02-13T14:15:54.772621+0000","last_peered":"2024-02-13T14:15:54.772621+0000","last_clean":"2024-02-13T14:15:54.772621+0000","last_became_active":"2024-02-13T14:15:12.173203+0000","last_became_peered":"2024-02-13T14:15:12.173203+0000","last_unstale":"2024-02-13T14:15:54.772621+0000","last_undegraded":"2024-02-13T14:15:54.772621+0000","last_fullsized":"2024-02-13T14:15:54.772621+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:11.150966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:11.150966+0000","last_clean_scrub_stamp":"2024-02-13T14:15:11.150966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T22:59:34.210780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"70'4","reported_seq":35,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.726572+0000","last_change":"2024-02-13T14:15:14.180694+0000","last_active":"2024-02-13T14:15:53.726572+0000","last_peered":"2024-02-13T14:15:53.726572+0000","last_clean":"2024-02-13T14:15:53.726572+0000","last_became_active":"2024-02-13T14:15:14.180386+0000","last_became_peered":"2024-02-13T14:15:14.180386+0000","last_unstale":"2024-02-13T14:15:53.726572+0000","last_undegraded":"2024-02-13T14:15:53.726572+0000","last_fullsized":"2024-02-13T14:15:53.726572+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:13.158947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:13.158947+0000","last_clean_scrub_stamp":"2024-02-13T14:15:13.158947+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T01:29:28.346535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.727438+0000","last_change":"2024-02-13T14:15:18.875994+0000","last_active":"2024-02-13T14:15:53.727438+0000","last_peered":"2024-02-13T14:15:53.727438+0000","last_clean":"2024-02-13T14:15:53.727438+0000","last_became_active":"2024-02-13T14:15:18.875321+0000","last_became_peered":"2024-02-13T14:15:18.875321+0000","last_unstale":"2024-02-13T14:15:53.727438+0000","last_undegraded":"2024-02-13T14:15:53.727438+0000","last_fullsized":"2024-02-13T14:15:53.727438+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:17.185096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:17.185096+0000","last_clean_scrub_stamp":"2024-02-13T14:15:17.185096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T20:04:22.089282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.727408+0000","last_change":"2024-02-13T14:15:16.199984+0000","last_active":"2024-02-13T14:15:53.727408+0000","last_peered":"2024-02-13T14:15:53.727408+0000","last_clean":"2024-02-13T14:15:53.727408+0000","last_became_active":"2024-02-13T14:15:16.199493+0000","last_became_peered":"2024-02-13T14:15:16.199493+0000","last_unstale":"2024-02-13T14:15:53.727408+0000","last_undegraded":"2024-02-13T14:15:53.727408+0000","last_fullsized":"2024-02-13T14:15:53.727408+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:15.174019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:15.174019+0000","last_clean_scrub_stamp":"2024-02-13T14:15:15.174019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T23:45:53.157045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.832910+0000","last_change":"2024-02-13T14:15:12.173488+0000","last_active":"2024-02-13T14:15:53.832910+0000","last_peered":"2024-02-13T14:15:53.832910+0000","last_clean":"2024-02-13T14:15:53.832910+0000","last_became_active":"2024-02-13T14:15:12.173205+0000","last_became_peered":"2024-02-13T14:15:12.173205+0000","last_unstale":"2024-02-13T14:15:53.832910+0000","last_undegraded":"2024-02-13T14:15:53.832910+0000","last_fullsized":"2024-02-13T14:15:53.832910+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:11.150966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:11.150966+0000","last_clean_scrub_stamp":"2024-02-13T14:15:11.150966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T22:00:07.329240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"70'11","reported_seq":48,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.727644+0000","last_change":"2024-02-13T14:15:14.333637+0000","last_active":"2024-02-13T14:15:53.727644+0000","last_peered":"2024-02-13T14:15:53.727644+0000","last_clean":"2024-02-13T14:15:53.727644+0000","last_became_active":"2024-02-13T14:15:14.333072+0000","last_became_peered":"2024-02-13T14:15:14.333072+0000","last_unstale":"2024-02-13T14:15:53.727644+0000","last_undegraded":"2024-02-13T14:15:53.727644+0000","last_fullsized":"2024-02-13T14:15:53.727644+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:13.158947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:13.158947+0000","last_clean_scrub_stamp":"2024-02-13T14:15:13.158947+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T16:19:33.936306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:54.771971+0000","last_change":"2024-02-13T14:15:18.218795+0000","last_active":"2024-02-13T14:15:54.771971+0000","last_peered":"2024-02-13T14:15:54.771971+0000","last_clean":"2024-02-13T14:15:54.771971+0000","last_became_active":"2024-02-13T14:15:18.218491+0000","last_became_peered":"2024-02-13T14:15:18.218491+0000","last_unstale":"2024-02-13T14:15:54.771971+0000","last_undegraded":"2024-02-13T14:15:54.771971+0000","last_fullsized":"2024-02-13T14:15:54.771971+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:17.185096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:17.185096+0000","last_clean_scrub_stamp":"2024-02-13T14:15:17.185096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T15:12:15.731109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.727650+0000","last_change":"2024-02-13T14:15:16.197039+0000","last_active":"2024-02-13T14:15:53.727650+0000","last_peered":"2024-02-13T14:15:53.727650+0000","last_clean":"2024-02-13T14:15:53.727650+0000","last_became_active":"2024-02-13T14:15:16.196673+0000","last_became_peered":"2024-02-13T14:15:16.196673+0000","last_unstale":"2024-02-13T14:15:53.727650+0000","last_undegraded":"2024-02-13T14:15:53.727650+0000","last_fullsized":"2024-02-13T14:15:53.727650+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:15.174019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:15.174019+0000","last_clean_scrub_stamp":"2024-02-13T14:15:15.174019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T17:45:32.446887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.727509+0000","last_change":"2024-02-13T14:15:12.177327+0000","last_active":"2024-02-13T14:15:53.727509+0000","last_peered":"2024-02-13T14:15:53.727509+0000","last_clean":"2024-02-13T14:15:53.727509+0000","last_became_active":"2024-02-13T14:15:12.177032+0000","last_became_peered":"2024-02-13T14:15:12.177032+0000","last_unstale":"2024-02-13T14:15:53.727509+0000","last_undegraded":"2024-02-13T14:15:53.727509+0000","last_fullsized":"2024-02-13T14:15:53.727509+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:11.150966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:11.150966+0000","last_clean_scrub_stamp":"2024-02-13T14:15:11.150966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T17:44:29.588326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"70'9","reported_seq":45,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:54.381099+0000","last_change":"2024-02-13T14:15:14.179255+0000","last_active":"2024-02-13T14:15:54.381099+0000","last_peered":"2024-02-13T14:15:54.381099+0000","last_clean":"2024-02-13T14:15:54.381099+0000","last_became_active":"2024-02-13T14:15:14.178956+0000","last_became_peered":"2024-02-13T14:15:14.178956+0000","last_unstale":"2024-02-13T14:15:54.381099+0000","last_undegraded":"2024-02-13T14:15:54.381099+0000","last_fullsized":"2024-02-13T14:15:54.381099+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:13.158947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:13.158947+0000","last_clean_scrub_stamp":"2024-02-13T14:15:13.158947+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T16:59:48.021624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.727458+0000","last_change":"2024-02-13T14:15:18.211825+0000","last_active":"2024-02-13T14:15:53.727458+0000","last_peered":"2024-02-13T14:15:53.727458+0000","last_clean":"2024-02-13T14:15:53.727458+0000","last_became_active":"2024-02-13T14:15:18.211498+0000","last_became_peered":"2024-02-13T14:15:18.211498+0000","last_unstale":"2024-02-13T14:15:53.727458+0000","last_undegraded":"2024-02-13T14:15:53.727458+0000","last_fullsized":"2024-02-13T14:15:53.727458+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:17.185096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:17.185096+0000","last_clean_scrub_stamp":"2024-02-13T14:15:17.185096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T15:01:47.997499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:54.382198+0000","last_change":"2024-02-13T14:15:16.200746+0000","last_active":"2024-02-13T14:15:54.382198+0000","last_peered":"2024-02-13T14:15:54.382198+0000","last_clean":"2024-02-13T14:15:54.382198+0000","last_became_active":"2024-02-13T14:15:16.200284+0000","last_became_peered":"2024-02-13T14:15:16.200284+0000","last_unstale":"2024-02-13T14:15:54.382198+0000","last_undegraded":"2024-02-13T14:15:54.382198+0000","last_fullsized":"2024-02-13T14:15:54.382198+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:15.174019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:15.174019+0000","last_clean_scrub_stamp":"2024-02-13T14:15:15.174019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T16:47:01.666512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:54.276944+0000","last_change":"2024-02-13T14:15:12.178954+0000","last_active":"2024-02-13T14:15:54.276944+0000","last_peered":"2024-02-13T14:15:54.276944+0000","last_clean":"2024-02-13T14:15:54.276944+0000","last_became_active":"2024-02-13T14:15:12.178663+0000","last_became_peered":"2024-02-13T14:15:12.178663+0000","last_unstale":"2024-02-13T14:15:54.276944+0000","last_undegraded":"2024-02-13T14:15:54.276944+0000","last_fullsized":"2024-02-13T14:15:54.276944+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:11.150966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:11.150966+0000","last_clean_scrub_stamp":"2024-02-13T14:15:11.150966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T23:20:12.238787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"70'11","reported_seq":48,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:54.276085+0000","last_change":"2024-02-13T14:15:14.176562+0000","last_active":"2024-02-13T14:15:54.276085+0000","last_peered":"2024-02-13T14:15:54.276085+0000","last_clean":"2024-02-13T14:15:54.276085+0000","last_became_active":"2024-02-13T14:15:14.176172+0000","last_became_peered":"2024-02-13T14:15:14.176172+0000","last_unstale":"2024-02-13T14:15:54.276085+0000","last_undegraded":"2024-02-13T14:15:54.276085+0000","last_fullsized":"2024-02-13T14:15:54.276085+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:13.158947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:13.158947+0000","last_clean_scrub_stamp":"2024-02-13T14:15:13.158947+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T00:04:18.752904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:54.772402+0000","last_change":"2024-02-13T14:15:18.211512+0000","last_active":"2024-02-13T14:15:54.772402+0000","last_peered":"2024-02-13T14:15:54.772402+0000","last_clean":"2024-02-13T14:15:54.772402+0000","last_became_active":"2024-02-13T14:15:18.211176+0000","last_became_peered":"2024-02-13T14:15:18.211176+0000","last_unstale":"2024-02-13T14:15:54.772402+0000","last_undegraded":"2024-02-13T14:15:54.772402+0000","last_fullsized":"2024-02-13T14:15:54.772402+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:17.185096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:17.185096+0000","last_clean_scrub_stamp":"2024-02-13T14:15:17.185096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T21:55:58.483380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"70'8","reported_seq":52,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:16:22.467815+0000","last_change":"2024-02-13T14:15:16.199356+0000","last_active":"2024-02-13T14:16:22.467815+0000","last_peered":"2024-02-13T14:16:22.467815+0000","last_clean":"2024-02-13T14:16:22.467815+0000","last_became_active":"2024-02-13T14:15:16.199090+0000","last_became_peered":"2024-02-13T14:15:16.199090+0000","last_unstale":"2024-02-13T14:16:22.467815+0000","last_undegraded":"2024-02-13T14:16:22.467815+0000","last_fullsized":"2024-02-13T14:16:22.467815+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:15.174019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:15.174019+0000","last_clean_scrub_stamp":"2024-02-13T14:15:15.174019+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T22:57:22.420105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.727651+0000","last_change":"2024-02-13T14:15:12.417131+0000","last_active":"2024-02-13T14:15:53.727651+0000","last_peered":"2024-02-13T14:15:53.727651+0000","last_clean":"2024-02-13T14:15:53.727651+0000","last_became_active":"2024-02-13T14:15:12.416631+0000","last_became_peered":"2024-02-13T14:15:12.416631+0000","last_unstale":"2024-02-13T14:15:53.727651+0000","last_undegraded":"2024-02-13T14:15:53.727651+0000","last_fullsized":"2024-02-13T14:15:53.727651+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:11.150966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:11.150966+0000","last_clean_scrub_stamp":"2024-02-13T14:15:11.150966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T20:20:28.320751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"70'10","reported_seq":62,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.728017+0000","last_change":"2024-02-13T14:15:30.019823+0000","last_active":"2024-02-13T14:15:53.728017+0000","last_peered":"2024-02-13T14:15:53.728017+0000","last_clean":"2024-02-13T14:15:53.728017+0000","last_became_active":"2024-02-13T14:15:30.011725+0000","last_became_peered":"2024-02-13T14:15:30.011725+0000","last_unstale":"2024-02-13T14:15:53.728017+0000","last_undegraded":"2024-02-13T14:15:53.728017+0000","last_fullsized":"2024-02-13T14:15:53.728017+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:13.158947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:13.158947+0000","last_clean_scrub_stamp":"2024-02-13T14:15:13.158947+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T02:05:15.487998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7],"acting":[3,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:54.277318+0000","last_change":"2024-02-13T14:15:18.215446+0000","last_active":"2024-02-13T14:15:54.277318+0000","last_peered":"2024-02-13T14:15:54.277318+0000","last_clean":"2024-02-13T14:15:54.277318+0000","last_became_active":"2024-02-13T14:15:18.215110+0000","last_became_peered":"2024-02-13T14:15:18.215110+0000","last_unstale":"2024-02-13T14:15:54.277318+0000","last_undegraded":"2024-02-13T14:15:54.277318+0000","last_fullsized":"2024-02-13T14:15:54.277318+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:17.185096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:17.185096+0000","last_clean_scrub_stamp":"2024-02-13T14:15:17.185096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T21:38:04.553920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"70'8","reported_seq":49,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:16:22.467542+0000","last_change":"2024-02-13T14:15:16.203356+0000","last_active":"2024-02-13T14:16:22.467542+0000","last_peered":"2024-02-13T14:16:22.467542+0000","last_clean":"2024-02-13T14:16:22.467542+0000","last_became_active":"2024-02-13T14:15:16.203006+0000","last_became_peered":"2024-02-13T14:15:16.203006+0000","last_unstale":"2024-02-13T14:16:22.467542+0000","last_undegraded":"2024-02-13T14:16:22.467542+0000","last_fullsized":"2024-02-13T14:16:22.467542+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:15.174019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:15.174019+0000","last_clean_scrub_stamp":"2024-02-13T14:15:15.174019+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T19:58:33.733272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"63'1","reported_seq":39,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:54.772241+0000","last_change":"2024-02-13T14:15:12.173688+0000","last_active":"2024-02-13T14:15:54.772241+0000","last_peered":"2024-02-13T14:15:54.772241+0000","last_clean":"2024-02-13T14:15:54.772241+0000","last_became_active":"2024-02-13T14:15:12.172963+0000","last_became_peered":"2024-02-13T14:15:12.172963+0000","last_unstale":"2024-02-13T14:15:54.772241+0000","last_undegraded":"2024-02-13T14:15:54.772241+0000","last_fullsized":"2024-02-13T14:15:54.772241+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:11.150966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:11.150966+0000","last_clean_scrub_stamp":"2024-02-13T14:15:11.150966+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T00:30:50.366918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":398,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"70'9","reported_seq":45,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.833108+0000","last_change":"2024-02-13T14:15:14.333209+0000","last_active":"2024-02-13T14:15:53.833108+0000","last_peered":"2024-02-13T14:15:53.833108+0000","last_clean":"2024-02-13T14:15:53.833108+0000","last_became_active":"2024-02-13T14:15:14.332864+0000","last_became_peered":"2024-02-13T14:15:14.332864+0000","last_unstale":"2024-02-13T14:15:53.833108+0000","last_undegraded":"2024-02-13T14:15:53.833108+0000","last_fullsized":"2024-02-13T14:15:53.833108+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:13.158947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:13.158947+0000","last_clean_scrub_stamp":"2024-02-13T14:15:13.158947+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T01:00:35.182771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.838631+0000","last_change":"2024-02-13T14:15:18.217693+0000","last_active":"2024-02-13T14:15:53.838631+0000","last_peered":"2024-02-13T14:15:53.838631+0000","last_clean":"2024-02-13T14:15:53.838631+0000","last_became_active":"2024-02-13T14:15:18.216672+0000","last_became_peered":"2024-02-13T14:15:18.216672+0000","last_unstale":"2024-02-13T14:15:53.838631+0000","last_undegraded":"2024-02-13T14:15:53.838631+0000","last_fullsized":"2024-02-13T14:15:53.838631+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:17.185096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:17.185096+0000","last_clean_scrub_stamp":"2024-02-13T14:15:17.185096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T21:29:02.134774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.728611+0000","last_change":"2024-02-13T14:15:16.198284+0000","last_active":"2024-02-13T14:15:53.728611+0000","last_peered":"2024-02-13T14:15:53.728611+0000","last_clean":"2024-02-13T14:15:53.728611+0000","last_became_active":"2024-02-13T14:15:16.197873+0000","last_became_peered":"2024-02-13T14:15:16.197873+0000","last_unstale":"2024-02-13T14:15:53.728611+0000","last_undegraded":"2024-02-13T14:15:53.728611+0000","last_fullsized":"2024-02-13T14:15:53.728611+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:15.174019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:15.174019+0000","last_clean_scrub_stamp":"2024-02-13T14:15:15.174019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T19:22:19.615613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.727457+0000","last_change":"2024-02-13T14:15:12.175577+0000","last_active":"2024-02-13T14:15:53.727457+0000","last_peered":"2024-02-13T14:15:53.727457+0000","last_clean":"2024-02-13T14:15:53.727457+0000","last_became_active":"2024-02-13T14:15:12.175265+0000","last_became_peered":"2024-02-13T14:15:12.175265+0000","last_unstale":"2024-02-13T14:15:53.727457+0000","last_undegraded":"2024-02-13T14:15:53.727457+0000","last_fullsized":"2024-02-13T14:15:53.727457+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:11.150966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:11.150966+0000","last_clean_scrub_stamp":"2024-02-13T14:15:11.150966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T15:45:30.799142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"70'9","reported_seq":45,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:54.771842+0000","last_change":"2024-02-13T14:15:14.332529+0000","last_active":"2024-02-13T14:15:54.771842+0000","last_peered":"2024-02-13T14:15:54.771842+0000","last_clean":"2024-02-13T14:15:54.771842+0000","last_became_active":"2024-02-13T14:15:14.332186+0000","last_became_peered":"2024-02-13T14:15:14.332186+0000","last_unstale":"2024-02-13T14:15:54.771842+0000","last_undegraded":"2024-02-13T14:15:54.771842+0000","last_fullsized":"2024-02-13T14:15:54.771842+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:13.158947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:13.158947+0000","last_clean_scrub_stamp":"2024-02-13T14:15:13.158947+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T18:03:39.979457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.727342+0000","last_change":"2024-02-13T14:15:18.876696+0000","last_active":"2024-02-13T14:15:53.727342+0000","last_peered":"2024-02-13T14:15:53.727342+0000","last_clean":"2024-02-13T14:15:53.727342+0000","last_became_active":"2024-02-13T14:15:18.876391+0000","last_became_peered":"2024-02-13T14:15:18.876391+0000","last_unstale":"2024-02-13T14:15:53.727342+0000","last_undegraded":"2024-02-13T14:15:53.727342+0000","last_fullsized":"2024-02-13T14:15:53.727342+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:17.185096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:17.185096+0000","last_clean_scrub_stamp":"2024-02-13T14:15:17.185096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T00:24:21.216608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.832943+0000","last_change":"2024-02-13T14:15:16.199438+0000","last_active":"2024-02-13T14:15:53.832943+0000","last_peered":"2024-02-13T14:15:53.832943+0000","last_clean":"2024-02-13T14:15:53.832943+0000","last_became_active":"2024-02-13T14:15:16.199179+0000","last_became_peered":"2024-02-13T14:15:16.199179+0000","last_unstale":"2024-02-13T14:15:53.832943+0000","last_undegraded":"2024-02-13T14:15:53.832943+0000","last_fullsized":"2024-02-13T14:15:53.832943+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:15.174019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:15.174019+0000","last_clean_scrub_stamp":"2024-02-13T14:15:15.174019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T21:43:27.215796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.728424+0000","last_change":"2024-02-13T14:15:12.175949+0000","last_active":"2024-02-13T14:15:53.728424+0000","last_peered":"2024-02-13T14:15:53.728424+0000","last_clean":"2024-02-13T14:15:53.728424+0000","last_became_active":"2024-02-13T14:15:12.175701+0000","last_became_peered":"2024-02-13T14:15:12.175701+0000","last_unstale":"2024-02-13T14:15:53.728424+0000","last_undegraded":"2024-02-13T14:15:53.728424+0000","last_fullsized":"2024-02-13T14:15:53.728424+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:11.150966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:11.150966+0000","last_clean_scrub_stamp":"2024-02-13T14:15:11.150966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T00:26:32.094209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"70'6","reported_seq":38,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.728697+0000","last_change":"2024-02-13T14:15:14.334485+0000","last_active":"2024-02-13T14:15:53.728697+0000","last_peered":"2024-02-13T14:15:53.728697+0000","last_clean":"2024-02-13T14:15:53.728697+0000","last_became_active":"2024-02-13T14:15:14.333615+0000","last_became_peered":"2024-02-13T14:15:14.333615+0000","last_unstale":"2024-02-13T14:15:53.728697+0000","last_undegraded":"2024-02-13T14:15:53.728697+0000","last_fullsized":"2024-02-13T14:15:53.728697+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:13.158947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:13.158947+0000","last_clean_scrub_stamp":"2024-02-13T14:15:13.158947+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T18:09:09.842222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.832755+0000","last_change":"2024-02-13T14:15:18.874574+0000","last_active":"2024-02-13T14:15:53.832755+0000","last_peered":"2024-02-13T14:15:53.832755+0000","last_clean":"2024-02-13T14:15:53.832755+0000","last_became_active":"2024-02-13T14:15:18.874296+0000","last_became_peered":"2024-02-13T14:15:18.874296+0000","last_unstale":"2024-02-13T14:15:53.832755+0000","last_undegraded":"2024-02-13T14:15:53.832755+0000","last_fullsized":"2024-02-13T14:15:53.832755+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:17.185096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:17.185096+0000","last_clean_scrub_stamp":"2024-02-13T14:15:17.185096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T23:56:12.985471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"63'2","reported_seq":45,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.728360+0000","last_change":"2024-02-13T14:15:12.179215+0000","last_active":"2024-02-13T14:15:53.728360+0000","last_peered":"2024-02-13T14:15:53.728360+0000","last_clean":"2024-02-13T14:15:53.728360+0000","last_became_active":"2024-02-13T14:15:12.178854+0000","last_became_peered":"2024-02-13T14:15:12.178854+0000","last_unstale":"2024-02-13T14:15:53.728360+0000","last_undegraded":"2024-02-13T14:15:53.728360+0000","last_fullsized":"2024-02-13T14:15:53.728360+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:11.150966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:11.150966+0000","last_clean_scrub_stamp":"2024-02-13T14:15:11.150966+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T17:06:18.808053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"70'15","reported_seq":54,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.832370+0000","last_change":"2024-02-13T14:15:14.334377+0000","last_active":"2024-02-13T14:15:53.832370+0000","last_peered":"2024-02-13T14:15:53.832370+0000","last_clean":"2024-02-13T14:15:53.832370+0000","last_became_active":"2024-02-13T14:15:14.334034+0000","last_became_peered":"2024-02-13T14:15:14.334034+0000","last_unstale":"2024-02-13T14:15:53.832370+0000","last_undegraded":"2024-02-13T14:15:53.832370+0000","last_fullsized":"2024-02-13T14:15:53.832370+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:13.158947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:13.158947+0000","last_clean_scrub_stamp":"2024-02-13T14:15:13.158947+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T23:45:24.518339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"70'8","reported_seq":49,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:16:22.467823+0000","last_change":"2024-02-13T14:15:16.202296+0000","last_active":"2024-02-13T14:16:22.467823+0000","last_peered":"2024-02-13T14:16:22.467823+0000","last_clean":"2024-02-13T14:16:22.467823+0000","last_became_active":"2024-02-13T14:15:16.201999+0000","last_became_peered":"2024-02-13T14:15:16.201999+0000","last_unstale":"2024-02-13T14:16:22.467823+0000","last_undegraded":"2024-02-13T14:16:22.467823+0000","last_fullsized":"2024-02-13T14:16:22.467823+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:15.174019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:15.174019+0000","last_clean_scrub_stamp":"2024-02-13T14:15:15.174019+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T01:20:12.563989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.727526+0000","last_change":"2024-02-13T14:15:18.212654+0000","last_active":"2024-02-13T14:15:53.727526+0000","last_peered":"2024-02-13T14:15:53.727526+0000","last_clean":"2024-02-13T14:15:53.727526+0000","last_became_active":"2024-02-13T14:15:18.212174+0000","last_became_peered":"2024-02-13T14:15:18.212174+0000","last_unstale":"2024-02-13T14:15:53.727526+0000","last_undegraded":"2024-02-13T14:15:53.727526+0000","last_fullsized":"2024-02-13T14:15:53.727526+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:17.185096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:17.185096+0000","last_clean_scrub_stamp":"2024-02-13T14:15:17.185096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T00:26:19.949680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"63'1","reported_seq":39,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.728179+0000","last_change":"2024-02-13T14:15:12.177686+0000","last_active":"2024-02-13T14:15:53.728179+0000","last_peered":"2024-02-13T14:15:53.728179+0000","last_clean":"2024-02-13T14:15:53.728179+0000","last_became_active":"2024-02-13T14:15:12.177010+0000","last_became_peered":"2024-02-13T14:15:12.177010+0000","last_unstale":"2024-02-13T14:15:53.728179+0000","last_undegraded":"2024-02-13T14:15:53.728179+0000","last_fullsized":"2024-02-13T14:15:53.728179+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:11.150966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:11.150966+0000","last_clean_scrub_stamp":"2024-02-13T14:15:11.150966+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T18:55:17.145494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"70'12","reported_seq":52,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:54.276057+0000","last_change":"2024-02-13T14:15:14.175855+0000","last_active":"2024-02-13T14:15:54.276057+0000","last_peered":"2024-02-13T14:15:54.276057+0000","last_clean":"2024-02-13T14:15:54.276057+0000","last_became_active":"2024-02-13T14:15:14.175509+0000","last_became_peered":"2024-02-13T14:15:14.175509+0000","last_unstale":"2024-02-13T14:15:54.276057+0000","last_undegraded":"2024-02-13T14:15:54.276057+0000","last_fullsized":"2024-02-13T14:15:54.276057+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:13.158947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:13.158947+0000","last_clean_scrub_stamp":"2024-02-13T14:15:13.158947+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T21:14:45.163180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.837851+0000","last_change":"2024-02-13T14:15:16.202484+0000","last_active":"2024-02-13T14:15:53.837851+0000","last_peered":"2024-02-13T14:15:53.837851+0000","last_clean":"2024-02-13T14:15:53.837851+0000","last_became_active":"2024-02-13T14:15:16.202162+0000","last_became_peered":"2024-02-13T14:15:16.202162+0000","last_unstale":"2024-02-13T14:15:53.837851+0000","last_undegraded":"2024-02-13T14:15:53.837851+0000","last_fullsized":"2024-02-13T14:15:53.837851+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:15.174019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:15.174019+0000","last_clean_scrub_stamp":"2024-02-13T14:15:15.174019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T19:38:04.299989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.727028+0000","last_change":"2024-02-13T14:15:18.213094+0000","last_active":"2024-02-13T14:15:53.727028+0000","last_peered":"2024-02-13T14:15:53.727028+0000","last_clean":"2024-02-13T14:15:53.727028+0000","last_became_active":"2024-02-13T14:15:18.212735+0000","last_became_peered":"2024-02-13T14:15:18.212735+0000","last_unstale":"2024-02-13T14:15:53.727028+0000","last_undegraded":"2024-02-13T14:15:53.727028+0000","last_fullsized":"2024-02-13T14:15:53.727028+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:17.185096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:17.185096+0000","last_clean_scrub_stamp":"2024-02-13T14:15:17.185096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T16:13:34.092833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.838067+0000","last_change":"2024-02-13T14:15:16.203161+0000","last_active":"2024-02-13T14:15:53.838067+0000","last_peered":"2024-02-13T14:15:53.838067+0000","last_clean":"2024-02-13T14:15:53.838067+0000","last_became_active":"2024-02-13T14:15:16.202838+0000","last_became_peered":"2024-02-13T14:15:16.202838+0000","last_unstale":"2024-02-13T14:15:53.838067+0000","last_undegraded":"2024-02-13T14:15:53.838067+0000","last_fullsized":"2024-02-13T14:15:53.838067+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:15.174019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:15.174019+0000","last_clean_scrub_stamp":"2024-02-13T14:15:15.174019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T19:46:00.376646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.727791+0000","last_change":"2024-02-13T14:15:12.175398+0000","last_active":"2024-02-13T14:15:53.727791+0000","last_peered":"2024-02-13T14:15:53.727791+0000","last_clean":"2024-02-13T14:15:53.727791+0000","last_became_active":"2024-02-13T14:15:12.175010+0000","last_became_peered":"2024-02-13T14:15:12.175010+0000","last_unstale":"2024-02-13T14:15:53.727791+0000","last_undegraded":"2024-02-13T14:15:53.727791+0000","last_fullsized":"2024-02-13T14:15:53.727791+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:11.150966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:11.150966+0000","last_clean_scrub_stamp":"2024-02-13T14:15:11.150966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T17:53:35.123529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"70'19","reported_seq":60,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.729697+0000","last_change":"2024-02-13T14:15:14.332776+0000","last_active":"2024-02-13T14:15:53.729697+0000","last_peered":"2024-02-13T14:15:53.729697+0000","last_clean":"2024-02-13T14:15:53.729697+0000","last_became_active":"2024-02-13T14:15:14.332410+0000","last_became_peered":"2024-02-13T14:15:14.332410+0000","last_unstale":"2024-02-13T14:15:53.729697+0000","last_undegraded":"2024-02-13T14:15:53.729697+0000","last_fullsized":"2024-02-13T14:15:53.729697+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:13.158947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:13.158947+0000","last_clean_scrub_stamp":"2024-02-13T14:15:13.158947+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T19:44:38.097073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:54.771992+0000","last_change":"2024-02-13T14:15:18.219177+0000","last_active":"2024-02-13T14:15:54.771992+0000","last_peered":"2024-02-13T14:15:54.771992+0000","last_clean":"2024-02-13T14:15:54.771992+0000","last_became_active":"2024-02-13T14:15:18.218789+0000","last_became_peered":"2024-02-13T14:15:18.218789+0000","last_unstale":"2024-02-13T14:15:54.771992+0000","last_undegraded":"2024-02-13T14:15:54.771992+0000","last_fullsized":"2024-02-13T14:15:54.771992+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:17.185096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:17.185096+0000","last_clean_scrub_stamp":"2024-02-13T14:15:17.185096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T00:40:33.532538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.838226+0000","last_change":"2024-02-13T14:15:16.199129+0000","last_active":"2024-02-13T14:15:53.838226+0000","last_peered":"2024-02-13T14:15:53.838226+0000","last_clean":"2024-02-13T14:15:53.838226+0000","last_became_active":"2024-02-13T14:15:16.198816+0000","last_became_peered":"2024-02-13T14:15:16.198816+0000","last_unstale":"2024-02-13T14:15:53.838226+0000","last_undegraded":"2024-02-13T14:15:53.838226+0000","last_fullsized":"2024-02-13T14:15:53.838226+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:15.174019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:15.174019+0000","last_clean_scrub_stamp":"2024-02-13T14:15:15.174019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T20:54:40.657639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.832762+0000","last_change":"2024-02-13T14:15:12.174353+0000","last_active":"2024-02-13T14:15:53.832762+0000","last_peered":"2024-02-13T14:15:53.832762+0000","last_clean":"2024-02-13T14:15:53.832762+0000","last_became_active":"2024-02-13T14:15:12.174074+0000","last_became_peered":"2024-02-13T14:15:12.174074+0000","last_unstale":"2024-02-13T14:15:53.832762+0000","last_undegraded":"2024-02-13T14:15:53.832762+0000","last_fullsized":"2024-02-13T14:15:53.832762+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:11.150966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:11.150966+0000","last_clean_scrub_stamp":"2024-02-13T14:15:11.150966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T01:06:47.861915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"70'9","reported_seq":45,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:54.772682+0000","last_change":"2024-02-13T14:15:14.178044+0000","last_active":"2024-02-13T14:15:54.772682+0000","last_peered":"2024-02-13T14:15:54.772682+0000","last_clean":"2024-02-13T14:15:54.772682+0000","last_became_active":"2024-02-13T14:15:14.177712+0000","last_became_peered":"2024-02-13T14:15:14.177712+0000","last_unstale":"2024-02-13T14:15:54.772682+0000","last_undegraded":"2024-02-13T14:15:54.772682+0000","last_fullsized":"2024-02-13T14:15:54.772682+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:13.158947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:13.158947+0000","last_clean_scrub_stamp":"2024-02-13T14:15:13.158947+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T20:31:59.447363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.728149+0000","last_change":"2024-02-13T14:15:18.209719+0000","last_active":"2024-02-13T14:15:53.728149+0000","last_peered":"2024-02-13T14:15:53.728149+0000","last_clean":"2024-02-13T14:15:53.728149+0000","last_became_active":"2024-02-13T14:15:18.209420+0000","last_became_peered":"2024-02-13T14:15:18.209420+0000","last_unstale":"2024-02-13T14:15:53.728149+0000","last_undegraded":"2024-02-13T14:15:53.728149+0000","last_fullsized":"2024-02-13T14:15:53.728149+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:17.185096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:17.185096+0000","last_clean_scrub_stamp":"2024-02-13T14:15:17.185096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T20:39:09.520608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"66'2","reported_seq":44,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.832515+0000","last_change":"2024-02-13T14:15:15.191712+0000","last_active":"2024-02-13T14:15:53.832515+0000","last_peered":"2024-02-13T14:15:53.832515+0000","last_clean":"2024-02-13T14:15:53.832515+0000","last_became_active":"2024-02-13T14:15:11.164778+0000","last_became_peered":"2024-02-13T14:15:11.164778+0000","last_unstale":"2024-02-13T14:15:53.832515+0000","last_undegraded":"2024-02-13T14:15:53.832515+0000","last_fullsized":"2024-02-13T14:15:53.832515+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:10.145476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:10.145476+0000","last_clean_scrub_stamp":"2024-02-13T14:15:10.145476+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T23:29:11.986752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014959350000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:54.276701+0000","last_change":"2024-02-13T14:15:12.174683+0000","last_active":"2024-02-13T14:15:54.276701+0000","last_peered":"2024-02-13T14:15:54.276701+0000","last_clean":"2024-02-13T14:15:54.276701+0000","last_became_active":"2024-02-13T14:15:12.174393+0000","last_became_peered":"2024-02-13T14:15:12.174393+0000","last_unstale":"2024-02-13T14:15:54.276701+0000","last_undegraded":"2024-02-13T14:15:54.276701+0000","last_fullsized":"2024-02-13T14:15:54.276701+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:11.150966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:11.150966+0000","last_clean_scrub_stamp":"2024-02-13T14:15:11.150966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T15:55:43.825228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"70'28","reported_seq":93,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:16:22.467542+0000","last_change":"2024-02-13T14:15:14.182362+0000","last_active":"2024-02-13T14:16:22.467542+0000","last_peered":"2024-02-13T14:16:22.467542+0000","last_clean":"2024-02-13T14:16:22.467542+0000","last_became_active":"2024-02-13T14:15:14.181840+0000","last_became_peered":"2024-02-13T14:15:14.181840+0000","last_unstale":"2024-02-13T14:16:22.467542+0000","last_undegraded":"2024-02-13T14:16:22.467542+0000","last_fullsized":"2024-02-13T14:16:22.467542+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:13.158947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:13.158947+0000","last_clean_scrub_stamp":"2024-02-13T14:15:13.158947+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T02:09:12.661626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:54.772481+0000","last_change":"2024-02-13T14:15:16.199472+0000","last_active":"2024-02-13T14:15:54.772481+0000","last_peered":"2024-02-13T14:15:54.772481+0000","last_clean":"2024-02-13T14:15:54.772481+0000","last_became_active":"2024-02-13T14:15:16.199115+0000","last_became_peered":"2024-02-13T14:15:16.199115+0000","last_unstale":"2024-02-13T14:15:54.772481+0000","last_undegraded":"2024-02-13T14:15:54.772481+0000","last_fullsized":"2024-02-13T14:15:54.772481+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:15.174019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:15.174019+0000","last_clean_scrub_stamp":"2024-02-13T14:15:15.174019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T23:21:36.799685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.728042+0000","last_change":"2024-02-13T14:15:18.873642+0000","last_active":"2024-02-13T14:15:53.728042+0000","last_peered":"2024-02-13T14:15:53.728042+0000","last_clean":"2024-02-13T14:15:53.728042+0000","last_became_active":"2024-02-13T14:15:18.873286+0000","last_became_peered":"2024-02-13T14:15:18.873286+0000","last_unstale":"2024-02-13T14:15:53.728042+0000","last_undegraded":"2024-02-13T14:15:53.728042+0000","last_fullsized":"2024-02-13T14:15:53.728042+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:17.185096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:17.185096+0000","last_clean_scrub_stamp":"2024-02-13T14:15:17.185096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T16:23:57.985044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:54.381847+0000","last_change":"2024-02-13T14:15:12.417576+0000","last_active":"2024-02-13T14:15:54.381847+0000","last_peered":"2024-02-13T14:15:54.381847+0000","last_clean":"2024-02-13T14:15:54.381847+0000","last_became_active":"2024-02-13T14:15:12.417254+0000","last_became_peered":"2024-02-13T14:15:12.417254+0000","last_unstale":"2024-02-13T14:15:54.381847+0000","last_undegraded":"2024-02-13T14:15:54.381847+0000","last_fullsized":"2024-02-13T14:15:54.381847+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:11.150966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:11.150966+0000","last_clean_scrub_stamp":"2024-02-13T14:15:11.150966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T18:36:24.228485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":42,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.837952+0000","last_change":"2024-02-13T14:15:15.190228+0000","last_active":"2024-02-13T14:15:53.837952+0000","last_peered":"2024-02-13T14:15:53.837952+0000","last_clean":"2024-02-13T14:15:53.837952+0000","last_became_active":"2024-02-13T14:15:11.165902+0000","last_became_peered":"2024-02-13T14:15:11.165902+0000","last_unstale":"2024-02-13T14:15:53.837952+0000","last_undegraded":"2024-02-13T14:15:53.837952+0000","last_fullsized":"2024-02-13T14:15:53.837952+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:10.145476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:10.145476+0000","last_clean_scrub_stamp":"2024-02-13T14:15:10.145476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T14:23:13.696445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001038976,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"70'13","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.838476+0000","last_change":"2024-02-13T14:15:30.020442+0000","last_active":"2024-02-13T14:15:53.838476+0000","last_peered":"2024-02-13T14:15:53.838476+0000","last_clean":"2024-02-13T14:15:53.838476+0000","last_became_active":"2024-02-13T14:15:30.013757+0000","last_became_peered":"2024-02-13T14:15:30.013757+0000","last_unstale":"2024-02-13T14:15:53.838476+0000","last_undegraded":"2024-02-13T14:15:53.838476+0000","last_fullsized":"2024-02-13T14:15:53.838476+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:13.158947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:13.158947+0000","last_clean_scrub_stamp":"2024-02-13T14:15:13.158947+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T23:04:07.873775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":18,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.838303+0000","last_change":"2024-02-13T14:15:16.199037+0000","last_active":"2024-02-13T14:15:53.838303+0000","last_peered":"2024-02-13T14:15:53.838303+0000","last_clean":"2024-02-13T14:15:53.838303+0000","last_became_active":"2024-02-13T14:15:16.198699+0000","last_became_peered":"2024-02-13T14:15:16.198699+0000","last_unstale":"2024-02-13T14:15:53.838303+0000","last_undegraded":"2024-02-13T14:15:53.838303+0000","last_fullsized":"2024-02-13T14:15:53.838303+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:15.174019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:15.174019+0000","last_clean_scrub_stamp":"2024-02-13T14:15:15.174019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T18:09:45.520620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.832943+0000","last_change":"2024-02-13T14:15:18.211319+0000","last_active":"2024-02-13T14:15:53.832943+0000","last_peered":"2024-02-13T14:15:53.832943+0000","last_clean":"2024-02-13T14:15:53.832943+0000","last_became_active":"2024-02-13T14:15:18.211032+0000","last_became_peered":"2024-02-13T14:15:18.211032+0000","last_unstale":"2024-02-13T14:15:53.832943+0000","last_undegraded":"2024-02-13T14:15:53.832943+0000","last_fullsized":"2024-02-13T14:15:53.832943+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:17.185096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:17.185096+0000","last_clean_scrub_stamp":"2024-02-13T14:15:17.185096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T00:28:22.214175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"74'128","reported_seq":279,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:55.825590+0000","last_change":"2024-02-13T14:15:01.785257+0000","last_active":"2024-02-13T14:15:55.825590+0000","last_peered":"2024-02-13T14:15:55.825590+0000","last_clean":"2024-02-13T14:15:55.825590+0000","last_became_active":"2024-02-13T14:15:01.784776+0000","last_became_peered":"2024-02-13T14:15:01.784776+0000","last_unstale":"2024-02-13T14:15:55.825590+0000","last_undegraded":"2024-02-13T14:15:55.825590+0000","last_fullsized":"2024-02-13T14:15:55.825590+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:12:53.266772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:12:53.266772+0000","last_clean_scrub_stamp":"2024-02-13T14:12:53.266772+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T22:39:57.646723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.727273+0000","last_change":"2024-02-13T14:15:12.177331+0000","last_active":"2024-02-13T14:15:53.727273+0000","last_peered":"2024-02-13T14:15:53.727273+0000","last_clean":"2024-02-13T14:15:53.727273+0000","last_became_active":"2024-02-13T14:15:12.177054+0000","last_became_peered":"2024-02-13T14:15:12.177054+0000","last_unstale":"2024-02-13T14:15:53.727273+0000","last_undegraded":"2024-02-13T14:15:53.727273+0000","last_fullsized":"2024-02-13T14:15:53.727273+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:11.150966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:11.150966+0000","last_clean_scrub_stamp":"2024-02-13T14:15:11.150966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T01:10:11.610740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"70'16","reported_seq":67,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:16:22.468124+0000","last_change":"2024-02-13T14:15:14.332926+0000","last_active":"2024-02-13T14:16:22.468124+0000","last_peered":"2024-02-13T14:16:22.468124+0000","last_clean":"2024-02-13T14:16:22.468124+0000","last_became_active":"2024-02-13T14:15:14.332516+0000","last_became_peered":"2024-02-13T14:15:14.332516+0000","last_unstale":"2024-02-13T14:16:22.468124+0000","last_undegraded":"2024-02-13T14:16:22.468124+0000","last_fullsized":"2024-02-13T14:16:22.468124+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:13.158947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:13.158947+0000","last_clean_scrub_stamp":"2024-02-13T14:15:13.158947+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T01:10:23.819027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.727688+0000","last_change":"2024-02-13T14:15:16.203769+0000","last_active":"2024-02-13T14:15:53.727688+0000","last_peered":"2024-02-13T14:15:53.727688+0000","last_clean":"2024-02-13T14:15:53.727688+0000","last_became_active":"2024-02-13T14:15:16.203491+0000","last_became_peered":"2024-02-13T14:15:16.203491+0000","last_unstale":"2024-02-13T14:15:53.727688+0000","last_undegraded":"2024-02-13T14:15:53.727688+0000","last_fullsized":"2024-02-13T14:15:53.727688+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:15.174019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:15.174019+0000","last_clean_scrub_stamp":"2024-02-13T14:15:15.174019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T19:21:03.840446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:54.382087+0000","last_change":"2024-02-13T14:15:18.213244+0000","last_active":"2024-02-13T14:15:54.382087+0000","last_peered":"2024-02-13T14:15:54.382087+0000","last_clean":"2024-02-13T14:15:54.382087+0000","last_became_active":"2024-02-13T14:15:18.212865+0000","last_became_peered":"2024-02-13T14:15:18.212865+0000","last_unstale":"2024-02-13T14:15:54.382087+0000","last_undegraded":"2024-02-13T14:15:54.382087+0000","last_fullsized":"2024-02-13T14:15:54.382087+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:17.185096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:17.185096+0000","last_clean_scrub_stamp":"2024-02-13T14:15:17.185096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T21:34:30.826293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:54.772111+0000","last_change":"2024-02-13T14:15:12.172581+0000","last_active":"2024-02-13T14:15:54.772111+0000","last_peered":"2024-02-13T14:15:54.772111+0000","last_clean":"2024-02-13T14:15:54.772111+0000","last_became_active":"2024-02-13T14:15:12.172281+0000","last_became_peered":"2024-02-13T14:15:12.172281+0000","last_unstale":"2024-02-13T14:15:54.772111+0000","last_undegraded":"2024-02-13T14:15:54.772111+0000","last_fullsized":"2024-02-13T14:15:54.772111+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:11.150966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:11.150966+0000","last_clean_scrub_stamp":"2024-02-13T14:15:11.150966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T15:39:01.051588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"71'5","reported_seq":117,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:16:28.715407+0000","last_change":"2024-02-13T14:15:15.189543+0000","last_active":"2024-02-13T14:16:28.715407+0000","last_peered":"2024-02-13T14:16:28.715407+0000","last_clean":"2024-02-13T14:16:28.715407+0000","last_became_active":"2024-02-13T14:15:11.165645+0000","last_became_peered":"2024-02-13T14:15:11.165645+0000","last_unstale":"2024-02-13T14:16:28.715407+0000","last_undegraded":"2024-02-13T14:16:28.715407+0000","last_fullsized":"2024-02-13T14:16:28.715407+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:10.145476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:10.145476+0000","last_clean_scrub_stamp":"2024-02-13T14:15:10.145476+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T20:35:16.455198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015406700000000001,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":72,"num_read_kb":67,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"70'12","reported_seq":47,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:54.772378+0000","last_change":"2024-02-13T14:15:14.179313+0000","last_active":"2024-02-13T14:15:54.772378+0000","last_peered":"2024-02-13T14:15:54.772378+0000","last_clean":"2024-02-13T14:15:54.772378+0000","last_became_active":"2024-02-13T14:15:14.178947+0000","last_became_peered":"2024-02-13T14:15:14.178947+0000","last_unstale":"2024-02-13T14:15:54.772378+0000","last_undegraded":"2024-02-13T14:15:54.772378+0000","last_fullsized":"2024-02-13T14:15:54.772378+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:13.158947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:13.158947+0000","last_clean_scrub_stamp":"2024-02-13T14:15:13.158947+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T01:24:26.394723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.833173+0000","last_change":"2024-02-13T14:15:16.200320+0000","last_active":"2024-02-13T14:15:53.833173+0000","last_peered":"2024-02-13T14:15:53.833173+0000","last_clean":"2024-02-13T14:15:53.833173+0000","last_became_active":"2024-02-13T14:15:16.200037+0000","last_became_peered":"2024-02-13T14:15:16.200037+0000","last_unstale":"2024-02-13T14:15:53.833173+0000","last_undegraded":"2024-02-13T14:15:53.833173+0000","last_fullsized":"2024-02-13T14:15:53.833173+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:15.174019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:15.174019+0000","last_clean_scrub_stamp":"2024-02-13T14:15:15.174019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T02:09:47.494344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:54.382097+0000","last_change":"2024-02-13T14:15:18.873555+0000","last_active":"2024-02-13T14:15:54.382097+0000","last_peered":"2024-02-13T14:15:54.382097+0000","last_clean":"2024-02-13T14:15:54.382097+0000","last_became_active":"2024-02-13T14:15:18.873196+0000","last_became_peered":"2024-02-13T14:15:18.873196+0000","last_unstale":"2024-02-13T14:15:54.382097+0000","last_undegraded":"2024-02-13T14:15:54.382097+0000","last_fullsized":"2024-02-13T14:15:54.382097+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:17.185096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:17.185096+0000","last_clean_scrub_stamp":"2024-02-13T14:15:17.185096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T21:21:53.402901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:54.382287+0000","last_change":"2024-02-13T14:15:12.416907+0000","last_active":"2024-02-13T14:15:54.382287+0000","last_peered":"2024-02-13T14:15:54.382287+0000","last_clean":"2024-02-13T14:15:54.382287+0000","last_became_active":"2024-02-13T14:15:12.416562+0000","last_became_peered":"2024-02-13T14:15:12.416562+0000","last_unstale":"2024-02-13T14:15:54.382287+0000","last_undegraded":"2024-02-13T14:15:54.382287+0000","last_fullsized":"2024-02-13T14:15:54.382287+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:11.150966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:11.150966+0000","last_clean_scrub_stamp":"2024-02-13T14:15:11.150966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T19:38:18.643483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"70'19","reported_seq":65,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:54.771840+0000","last_change":"2024-02-13T14:15:14.332634+0000","last_active":"2024-02-13T14:15:54.771840+0000","last_peered":"2024-02-13T14:15:54.771840+0000","last_clean":"2024-02-13T14:15:54.771840+0000","last_became_active":"2024-02-13T14:15:14.332188+0000","last_became_peered":"2024-02-13T14:15:14.332188+0000","last_unstale":"2024-02-13T14:15:54.771840+0000","last_undegraded":"2024-02-13T14:15:54.771840+0000","last_fullsized":"2024-02-13T14:15:54.771840+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:13.158947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:13.158947+0000","last_clean_scrub_stamp":"2024-02-13T14:15:13.158947+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T17:01:48.908838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.729869+0000","last_change":"2024-02-13T14:15:16.201964+0000","last_active":"2024-02-13T14:15:53.729869+0000","last_peered":"2024-02-13T14:15:53.729869+0000","last_clean":"2024-02-13T14:15:53.729869+0000","last_became_active":"2024-02-13T14:15:16.201600+0000","last_became_peered":"2024-02-13T14:15:16.201600+0000","last_unstale":"2024-02-13T14:15:53.729869+0000","last_undegraded":"2024-02-13T14:15:53.729869+0000","last_fullsized":"2024-02-13T14:15:53.729869+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:15.174019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:15.174019+0000","last_clean_scrub_stamp":"2024-02-13T14:15:15.174019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T15:17:01.382222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:54.772021+0000","last_change":"2024-02-13T14:15:18.220066+0000","last_active":"2024-02-13T14:15:54.772021+0000","last_peered":"2024-02-13T14:15:54.772021+0000","last_clean":"2024-02-13T14:15:54.772021+0000","last_became_active":"2024-02-13T14:15:18.219655+0000","last_became_peered":"2024-02-13T14:15:18.219655+0000","last_unstale":"2024-02-13T14:15:54.772021+0000","last_undegraded":"2024-02-13T14:15:54.772021+0000","last_fullsized":"2024-02-13T14:15:54.772021+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:17.185096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:17.185096+0000","last_clean_scrub_stamp":"2024-02-13T14:15:17.185096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T23:56:38.790923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.833028+0000","last_change":"2024-02-13T14:15:12.416955+0000","last_active":"2024-02-13T14:15:53.833028+0000","last_peered":"2024-02-13T14:15:53.833028+0000","last_clean":"2024-02-13T14:15:53.833028+0000","last_became_active":"2024-02-13T14:15:12.416595+0000","last_became_peered":"2024-02-13T14:15:12.416595+0000","last_unstale":"2024-02-13T14:15:53.833028+0000","last_undegraded":"2024-02-13T14:15:53.833028+0000","last_fullsized":"2024-02-13T14:15:53.833028+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:11.150966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:11.150966+0000","last_clean_scrub_stamp":"2024-02-13T14:15:11.150966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T23:33:58.171584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"70'10","reported_seq":44,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:54.381261+0000","last_change":"2024-02-13T14:15:14.182775+0000","last_active":"2024-02-13T14:15:54.381261+0000","last_peered":"2024-02-13T14:15:54.381261+0000","last_clean":"2024-02-13T14:15:54.381261+0000","last_became_active":"2024-02-13T14:15:14.182403+0000","last_became_peered":"2024-02-13T14:15:14.182403+0000","last_unstale":"2024-02-13T14:15:54.381261+0000","last_undegraded":"2024-02-13T14:15:54.381261+0000","last_fullsized":"2024-02-13T14:15:54.381261+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:13.158947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:13.158947+0000","last_clean_scrub_stamp":"2024-02-13T14:15:13.158947+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T19:35:01.071596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"70'8","reported_seq":49,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:16:22.467501+0000","last_change":"2024-02-13T14:15:16.202939+0000","last_active":"2024-02-13T14:16:22.467501+0000","last_peered":"2024-02-13T14:16:22.467501+0000","last_clean":"2024-02-13T14:16:22.467501+0000","last_became_active":"2024-02-13T14:15:16.202633+0000","last_became_peered":"2024-02-13T14:15:16.202633+0000","last_unstale":"2024-02-13T14:16:22.467501+0000","last_undegraded":"2024-02-13T14:16:22.467501+0000","last_fullsized":"2024-02-13T14:16:22.467501+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:15.174019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:15.174019+0000","last_clean_scrub_stamp":"2024-02-13T14:15:15.174019+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T18:21:29.990547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.727264+0000","last_change":"2024-02-13T14:15:18.876763+0000","last_active":"2024-02-13T14:15:53.727264+0000","last_peered":"2024-02-13T14:15:53.727264+0000","last_clean":"2024-02-13T14:15:53.727264+0000","last_became_active":"2024-02-13T14:15:18.876450+0000","last_became_peered":"2024-02-13T14:15:18.876450+0000","last_unstale":"2024-02-13T14:15:53.727264+0000","last_undegraded":"2024-02-13T14:15:53.727264+0000","last_fullsized":"2024-02-13T14:15:53.727264+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:17.185096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:17.185096+0000","last_clean_scrub_stamp":"2024-02-13T14:15:17.185096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T19:18:44.250701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.6","version":"63'1","reported_seq":34,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:54.772577+0000","last_change":"2024-02-13T14:15:12.174096+0000","last_active":"2024-02-13T14:15:54.772577+0000","last_peered":"2024-02-13T14:15:54.772577+0000","last_clean":"2024-02-13T14:15:54.772577+0000","last_became_active":"2024-02-13T14:15:12.173051+0000","last_became_peered":"2024-02-13T14:15:12.173051+0000","last_unstale":"2024-02-13T14:15:54.772577+0000","last_undegraded":"2024-02-13T14:15:54.772577+0000","last_fullsized":"2024-02-13T14:15:54.772577+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:11.150966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:11.150966+0000","last_clean_scrub_stamp":"2024-02-13T14:15:11.150966+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T14:34:17.943660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"70'14","reported_seq":50,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:54.276066+0000","last_change":"2024-02-13T14:15:14.172193+0000","last_active":"2024-02-13T14:15:54.276066+0000","last_peered":"2024-02-13T14:15:54.276066+0000","last_clean":"2024-02-13T14:15:54.276066+0000","last_became_active":"2024-02-13T14:15:14.171816+0000","last_became_peered":"2024-02-13T14:15:14.171816+0000","last_unstale":"2024-02-13T14:15:54.276066+0000","last_undegraded":"2024-02-13T14:15:54.276066+0000","last_fullsized":"2024-02-13T14:15:54.276066+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:13.158947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:13.158947+0000","last_clean_scrub_stamp":"2024-02-13T14:15:13.158947+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T00:22:12.121393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"70'8","reported_seq":49,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:16:22.467497+0000","last_change":"2024-02-13T14:15:16.195776+0000","last_active":"2024-02-13T14:16:22.467497+0000","last_peered":"2024-02-13T14:16:22.467497+0000","last_clean":"2024-02-13T14:16:22.467497+0000","last_became_active":"2024-02-13T14:15:16.195481+0000","last_became_peered":"2024-02-13T14:15:16.195481+0000","last_unstale":"2024-02-13T14:16:22.467497+0000","last_undegraded":"2024-02-13T14:16:22.467497+0000","last_fullsized":"2024-02-13T14:16:22.467497+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:15.174019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:15.174019+0000","last_clean_scrub_stamp":"2024-02-13T14:15:15.174019+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T20:08:34.562080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:54.276643+0000","last_change":"2024-02-13T14:15:18.217626+0000","last_active":"2024-02-13T14:15:54.276643+0000","last_peered":"2024-02-13T14:15:54.276643+0000","last_clean":"2024-02-13T14:15:54.276643+0000","last_became_active":"2024-02-13T14:15:18.217389+0000","last_became_peered":"2024-02-13T14:15:18.217389+0000","last_unstale":"2024-02-13T14:15:54.276643+0000","last_undegraded":"2024-02-13T14:15:54.276643+0000","last_fullsized":"2024-02-13T14:15:54.276643+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:17.185096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:17.185096+0000","last_clean_scrub_stamp":"2024-02-13T14:15:17.185096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T01:17:15.759600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.728455+0000","last_change":"2024-02-13T14:15:12.171708+0000","last_active":"2024-02-13T14:15:53.728455+0000","last_peered":"2024-02-13T14:15:53.728455+0000","last_clean":"2024-02-13T14:15:53.728455+0000","last_became_active":"2024-02-13T14:15:12.171371+0000","last_became_peered":"2024-02-13T14:15:12.171371+0000","last_unstale":"2024-02-13T14:15:53.728455+0000","last_undegraded":"2024-02-13T14:15:53.728455+0000","last_fullsized":"2024-02-13T14:15:53.728455+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:11.150966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:11.150966+0000","last_clean_scrub_stamp":"2024-02-13T14:15:11.150966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T20:40:53.640720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"70'18","reported_seq":61,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.726688+0000","last_change":"2024-02-13T14:15:14.333793+0000","last_active":"2024-02-13T14:15:53.726688+0000","last_peered":"2024-02-13T14:15:53.726688+0000","last_clean":"2024-02-13T14:15:53.726688+0000","last_became_active":"2024-02-13T14:15:14.333478+0000","last_became_peered":"2024-02-13T14:15:14.333478+0000","last_unstale":"2024-02-13T14:15:53.726688+0000","last_undegraded":"2024-02-13T14:15:53.726688+0000","last_fullsized":"2024-02-13T14:15:53.726688+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:13.158947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:13.158947+0000","last_clean_scrub_stamp":"2024-02-13T14:15:13.158947+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T20:44:05.328601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:54.276494+0000","last_change":"2024-02-13T14:15:16.198595+0000","last_active":"2024-02-13T14:15:54.276494+0000","last_peered":"2024-02-13T14:15:54.276494+0000","last_clean":"2024-02-13T14:15:54.276494+0000","last_became_active":"2024-02-13T14:15:16.198291+0000","last_became_peered":"2024-02-13T14:15:16.198291+0000","last_unstale":"2024-02-13T14:15:54.276494+0000","last_undegraded":"2024-02-13T14:15:54.276494+0000","last_fullsized":"2024-02-13T14:15:54.276494+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:15.174019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:15.174019+0000","last_clean_scrub_stamp":"2024-02-13T14:15:15.174019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T23:53:39.694426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.832756+0000","last_change":"2024-02-13T14:15:18.213650+0000","last_active":"2024-02-13T14:15:53.832756+0000","last_peered":"2024-02-13T14:15:53.832756+0000","last_clean":"2024-02-13T14:15:53.832756+0000","last_became_active":"2024-02-13T14:15:18.213375+0000","last_became_peered":"2024-02-13T14:15:18.213375+0000","last_unstale":"2024-02-13T14:15:53.832756+0000","last_undegraded":"2024-02-13T14:15:53.832756+0000","last_fullsized":"2024-02-13T14:15:53.832756+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:17.185096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:17.185096+0000","last_clean_scrub_stamp":"2024-02-13T14:15:17.185096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T17:03:41.344035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"70'8","reported_seq":49,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:16:22.467820+0000","last_change":"2024-02-13T14:15:16.196368+0000","last_active":"2024-02-13T14:16:22.467820+0000","last_peered":"2024-02-13T14:16:22.467820+0000","last_clean":"2024-02-13T14:16:22.467820+0000","last_became_active":"2024-02-13T14:15:16.196057+0000","last_became_peered":"2024-02-13T14:15:16.196057+0000","last_unstale":"2024-02-13T14:16:22.467820+0000","last_undegraded":"2024-02-13T14:16:22.467820+0000","last_fullsized":"2024-02-13T14:16:22.467820+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:15.174019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:15.174019+0000","last_clean_scrub_stamp":"2024-02-13T14:15:15.174019+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T00:14:51.809112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.726618+0000","last_change":"2024-02-13T14:15:12.168180+0000","last_active":"2024-02-13T14:15:53.726618+0000","last_peered":"2024-02-13T14:15:53.726618+0000","last_clean":"2024-02-13T14:15:53.726618+0000","last_became_active":"2024-02-13T14:15:12.167830+0000","last_became_peered":"2024-02-13T14:15:12.167830+0000","last_unstale":"2024-02-13T14:15:53.726618+0000","last_undegraded":"2024-02-13T14:15:53.726618+0000","last_fullsized":"2024-02-13T14:15:53.726618+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:11.150966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:11.150966+0000","last_clean_scrub_stamp":"2024-02-13T14:15:11.150966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T00:40:38.722769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"70'15","reported_seq":54,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:54.381211+0000","last_change":"2024-02-13T14:15:14.181534+0000","last_active":"2024-02-13T14:15:54.381211+0000","last_peered":"2024-02-13T14:15:54.381211+0000","last_clean":"2024-02-13T14:15:54.381211+0000","last_became_active":"2024-02-13T14:15:14.180668+0000","last_became_peered":"2024-02-13T14:15:14.180668+0000","last_unstale":"2024-02-13T14:15:54.381211+0000","last_undegraded":"2024-02-13T14:15:54.381211+0000","last_fullsized":"2024-02-13T14:15:54.381211+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:13.158947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:13.158947+0000","last_clean_scrub_stamp":"2024-02-13T14:15:13.158947+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T23:47:18.317400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.728082+0000","last_change":"2024-02-13T14:15:18.873633+0000","last_active":"2024-02-13T14:15:53.728082+0000","last_peered":"2024-02-13T14:15:53.728082+0000","last_clean":"2024-02-13T14:15:53.728082+0000","last_became_active":"2024-02-13T14:15:18.873283+0000","last_became_peered":"2024-02-13T14:15:18.873283+0000","last_unstale":"2024-02-13T14:15:53.728082+0000","last_undegraded":"2024-02-13T14:15:53.728082+0000","last_fullsized":"2024-02-13T14:15:53.728082+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:17.185096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:17.185096+0000","last_clean_scrub_stamp":"2024-02-13T14:15:17.185096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T18:27:27.484930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.833154+0000","last_change":"2024-02-13T14:15:16.197082+0000","last_active":"2024-02-13T14:15:53.833154+0000","last_peered":"2024-02-13T14:15:53.833154+0000","last_clean":"2024-02-13T14:15:53.833154+0000","last_became_active":"2024-02-13T14:15:16.196293+0000","last_became_peered":"2024-02-13T14:15:16.196293+0000","last_unstale":"2024-02-13T14:15:53.833154+0000","last_undegraded":"2024-02-13T14:15:53.833154+0000","last_fullsized":"2024-02-13T14:15:53.833154+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:15.174019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:15.174019+0000","last_clean_scrub_stamp":"2024-02-13T14:15:15.174019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T17:00:46.766784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:54.276539+0000","last_change":"2024-02-13T14:15:12.417704+0000","last_active":"2024-02-13T14:15:54.276539+0000","last_peered":"2024-02-13T14:15:54.276539+0000","last_clean":"2024-02-13T14:15:54.276539+0000","last_became_active":"2024-02-13T14:15:12.416567+0000","last_became_peered":"2024-02-13T14:15:12.416567+0000","last_unstale":"2024-02-13T14:15:54.276539+0000","last_undegraded":"2024-02-13T14:15:54.276539+0000","last_fullsized":"2024-02-13T14:15:54.276539+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:11.150966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:11.150966+0000","last_clean_scrub_stamp":"2024-02-13T14:15:11.150966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T01:59:35.106234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"70'11","reported_seq":48,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:54.276353+0000","last_change":"2024-02-13T14:15:14.176817+0000","last_active":"2024-02-13T14:15:54.276353+0000","last_peered":"2024-02-13T14:15:54.276353+0000","last_clean":"2024-02-13T14:15:54.276353+0000","last_became_active":"2024-02-13T14:15:14.176123+0000","last_became_peered":"2024-02-13T14:15:14.176123+0000","last_unstale":"2024-02-13T14:15:54.276353+0000","last_undegraded":"2024-02-13T14:15:54.276353+0000","last_fullsized":"2024-02-13T14:15:54.276353+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:13.158947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:13.158947+0000","last_clean_scrub_stamp":"2024-02-13T14:15:13.158947+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T00:01:54.818359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.838606+0000","last_change":"2024-02-13T14:15:18.215227+0000","last_active":"2024-02-13T14:15:53.838606+0000","last_peered":"2024-02-13T14:15:53.838606+0000","last_clean":"2024-02-13T14:15:53.838606+0000","last_became_active":"2024-02-13T14:15:18.214947+0000","last_became_peered":"2024-02-13T14:15:18.214947+0000","last_unstale":"2024-02-13T14:15:53.838606+0000","last_undegraded":"2024-02-13T14:15:53.838606+0000","last_fullsized":"2024-02-13T14:15:53.838606+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:17.185096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:17.185096+0000","last_clean_scrub_stamp":"2024-02-13T14:15:17.185096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T17:37:51.731285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:54.382328+0000","last_change":"2024-02-13T14:15:16.198099+0000","last_active":"2024-02-13T14:15:54.382328+0000","last_peered":"2024-02-13T14:15:54.382328+0000","last_clean":"2024-02-13T14:15:54.382328+0000","last_became_active":"2024-02-13T14:15:16.197809+0000","last_became_peered":"2024-02-13T14:15:16.197809+0000","last_unstale":"2024-02-13T14:15:54.382328+0000","last_undegraded":"2024-02-13T14:15:54.382328+0000","last_fullsized":"2024-02-13T14:15:54.382328+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:15.174019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:15.174019+0000","last_clean_scrub_stamp":"2024-02-13T14:15:15.174019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T17:04:20.778378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.729914+0000","last_change":"2024-02-13T14:15:12.173685+0000","last_active":"2024-02-13T14:15:53.729914+0000","last_peered":"2024-02-13T14:15:53.729914+0000","last_clean":"2024-02-13T14:15:53.729914+0000","last_became_active":"2024-02-13T14:15:12.173336+0000","last_became_peered":"2024-02-13T14:15:12.173336+0000","last_unstale":"2024-02-13T14:15:53.729914+0000","last_undegraded":"2024-02-13T14:15:53.729914+0000","last_fullsized":"2024-02-13T14:15:53.729914+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:11.150966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:11.150966+0000","last_clean_scrub_stamp":"2024-02-13T14:15:11.150966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T01:47:42.700019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"70'17","reported_seq":81,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:54.276323+0000","last_change":"2024-02-13T14:15:30.027281+0000","last_active":"2024-02-13T14:15:54.276323+0000","last_peered":"2024-02-13T14:15:54.276323+0000","last_clean":"2024-02-13T14:15:54.276323+0000","last_became_active":"2024-02-13T14:15:30.013179+0000","last_became_peered":"2024-02-13T14:15:30.013179+0000","last_unstale":"2024-02-13T14:15:54.276323+0000","last_undegraded":"2024-02-13T14:15:54.276323+0000","last_fullsized":"2024-02-13T14:15:54.276323+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:13.158947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:13.158947+0000","last_clean_scrub_stamp":"2024-02-13T14:15:13.158947+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T23:17:54.679510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"70'1","reported_seq":22,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:54.277242+0000","last_change":"2024-02-13T14:15:18.215668+0000","last_active":"2024-02-13T14:15:54.277242+0000","last_peered":"2024-02-13T14:15:54.277242+0000","last_clean":"2024-02-13T14:15:54.277242+0000","last_became_active":"2024-02-13T14:15:18.215173+0000","last_became_peered":"2024-02-13T14:15:18.215173+0000","last_unstale":"2024-02-13T14:15:54.277242+0000","last_undegraded":"2024-02-13T14:15:54.277242+0000","last_fullsized":"2024-02-13T14:15:54.277242+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:17.185096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:17.185096+0000","last_clean_scrub_stamp":"2024-02-13T14:15:17.185096+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T18:29:41.390078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"70'8","reported_seq":49,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:16:22.467582+0000","last_change":"2024-02-13T14:15:16.199835+0000","last_active":"2024-02-13T14:16:22.467582+0000","last_peered":"2024-02-13T14:16:22.467582+0000","last_clean":"2024-02-13T14:16:22.467582+0000","last_became_active":"2024-02-13T14:15:16.199319+0000","last_became_peered":"2024-02-13T14:15:16.199319+0000","last_unstale":"2024-02-13T14:16:22.467582+0000","last_undegraded":"2024-02-13T14:16:22.467582+0000","last_fullsized":"2024-02-13T14:16:22.467582+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:15.174019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:15.174019+0000","last_clean_scrub_stamp":"2024-02-13T14:15:15.174019+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T00:45:12.263270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.727694+0000","last_change":"2024-02-13T14:15:12.174951+0000","last_active":"2024-02-13T14:15:53.727694+0000","last_peered":"2024-02-13T14:15:53.727694+0000","last_clean":"2024-02-13T14:15:53.727694+0000","last_became_active":"2024-02-13T14:15:12.174462+0000","last_became_peered":"2024-02-13T14:15:12.174462+0000","last_unstale":"2024-02-13T14:15:53.727694+0000","last_undegraded":"2024-02-13T14:15:53.727694+0000","last_fullsized":"2024-02-13T14:15:53.727694+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:11.150966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:11.150966+0000","last_clean_scrub_stamp":"2024-02-13T14:15:11.150966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T14:42:22.960402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"70'10","reported_seq":44,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:54.276135+0000","last_change":"2024-02-13T14:15:14.176111+0000","last_active":"2024-02-13T14:15:54.276135+0000","last_peered":"2024-02-13T14:15:54.276135+0000","last_clean":"2024-02-13T14:15:54.276135+0000","last_became_active":"2024-02-13T14:15:14.174639+0000","last_became_peered":"2024-02-13T14:15:14.174639+0000","last_unstale":"2024-02-13T14:15:54.276135+0000","last_undegraded":"2024-02-13T14:15:54.276135+0000","last_fullsized":"2024-02-13T14:15:54.276135+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:13.158947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:13.158947+0000","last_clean_scrub_stamp":"2024-02-13T14:15:13.158947+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T16:07:46.515640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":697,"num_read_kb":454,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":393,"ondisk_log_size":393,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1456,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":72,"num_read_kb":67,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":9,"acting":9,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":56,"seq":240518168596,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32572,"kb_used_data":1396,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712580,"statfs":{"total":95995035648,"available":95961681920,"internally_reserved":0,"allocated":1429504,"data_stored":937896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Feb 13 14:15:58 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.79200000000000004}]},{"osd":1,"last update":"Tue Feb 13 14:15:58 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.77400000000000002}]},{"osd":2,"last update":"Tue Feb 13 14:15:58 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"last":0.71799999999999997}]},{"osd":3,"last update":"Tue Feb 13 14:15:58 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.745}]},{"osd":4,"last update":"Tue Feb 13 14:15:58 2024","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.629,"5min":0.629,"15min":0.629},"last":0.59699999999999998}]},{"osd":5,"last update":"Tue Feb 13 14:15:58 2024","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.626,"5min":0.626,"15min":0.626},"last":0.63100000000000001}]},{"osd":6,"last update":"Tue Feb 13 14:15:58 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.67700000000000005}]}]},{"osd":6,"up_from":48,"seq":206158430234,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32544,"kb_used_data":1368,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712608,"statfs":{"total":95995035648,"available":95961710592,"internally_reserved":0,"allocated":1400832,"data_stored":935573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Feb 13 14:15:36 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.36699999999999999}]},{"osd":1,"last update":"Tue Feb 13 14:15:36 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.38800000000000001}]},{"osd":2,"last update":"Tue Feb 13 14:15:36 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.41299999999999998}]},{"osd":3,"last update":"Tue Feb 13 14:15:36 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.501}]},{"osd":4,"last update":"Tue Feb 13 14:15:36 2024","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.626}]},{"osd":5,"last update":"Tue Feb 13 14:15:36 2024","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.434,"15min":0.434},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.58099999999999996}]},{"osd":7,"last update":"Tue Feb 13 14:16:00 2024","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.64100000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574900,"num_pgs":54,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27864,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717288,"statfs":{"total":95995035648,"available":95966502912,"internally_reserved":0,"allocated":864256,"data_stored":347437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Feb 13 14:15:31 2024","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.32100000000000001,"15min":0.29599999999999999},"min":{"1min":0.24199999999999999,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.33700000000000002,"15min":0.311},"min":{"1min":0.22800000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.51000000000000001,"5min":0.54800000000000004,"15min":0.54800000000000004},"last":0.30599999999999999}]},{"osd":2,"last update":"Tue Feb 13 14:16:01 2024","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.377,"15min":0.375},"min":{"1min":0.19800000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.372},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.40200000000000002,"15min":0.39200000000000002},"min":{"1min":0.27900000000000003,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.622,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.28799999999999998}]},{"osd":3,"last update":"Tue Feb 13 14:16:21 2024","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.439,"15min":0.45100000000000001},"min":{"1min":0.29299999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.624,"5min":0.63,"15min":0.63},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.46999999999999997,"15min":0.47399999999999998},"min":{"1min":0.32100000000000001,"5min":0.309,"15min":0.309},"max":{"1min":0.67800000000000005,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.35799999999999998}]},{"osd":4,"last update":"Tue Feb 13 14:15:41 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.56599999999999995,"15min":0.56799999999999995},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.57999999999999996}]},{"osd":5,"last update":"Tue Feb 13 14:16:09 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.60499999999999998,"15min":0.60699999999999998},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.81699999999999995,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.59799999999999998,"15min":0.59699999999999998},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.79200000000000004,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.65100000000000002}]},{"osd":6,"last update":"Tue Feb 13 14:15:36 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.59499999999999997}]},{"osd":7,"last update":"Tue Feb 13 14:16:01 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.625}]}]},{"osd":0,"up_from":8,"seq":34359738425,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32576,"kb_used_data":1396,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712576,"statfs":{"total":95995035648,"available":95961677824,"internally_reserved":0,"allocated":1429504,"data_stored":936765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Feb 13 14:15:32 2024","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.33200000000000002,"15min":0.30599999999999999},"min":{"1min":0.245,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.52400000000000002,"5min":0.61299999999999999,"15min":0.61299999999999999},"last":0.30599999999999999},{"interface":"front","average":{"1min":0.41199999999999998,"5min":0.34899999999999998,"15min":0.31900000000000001},"min":{"1min":0.22500000000000001,"5min":0.223,"15min":0.223},"max":{"1min":0.60299999999999998,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.246}]},{"osd":2,"last update":"Tue Feb 13 14:15:58 2024","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.39300000000000002,"15min":0.379},"min":{"1min":0.27200000000000002,"5min":0.246,"15min":0.246},"max":{"1min":0.49299999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"last":0.29099999999999998},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.40100000000000002,"15min":0.39300000000000002},"min":{"1min":0.26300000000000001,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"last":0.26600000000000001}]},{"osd":3,"last update":"Tue Feb 13 14:16:21 2024","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.45800000000000002,"15min":0.46600000000000003},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.57299999999999995,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.432,"15min":0.436},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.56499999999999995,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.32500000000000001}]},{"osd":4,"last update":"Tue Feb 13 14:15:46 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.59399999999999997,"15min":0.59799999999999998},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.70499999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.625,"5min":0.59499999999999997,"15min":0.58999999999999997},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.50700000000000001}]},{"osd":5,"last update":"Tue Feb 13 14:16:14 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.59899999999999998,"15min":0.60099999999999998},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.76400000000000001,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.58399999999999996,"15min":0.58099999999999996},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.52500000000000002}]},{"osd":6,"last update":"Tue Feb 13 14:15:32 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.55800000000000005}]},{"osd":7,"last update":"Tue Feb 13 14:15:58 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.49299999999999999}]}]},{"osd":2,"up_from":18,"seq":77309411374,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27820,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717332,"statfs":{"total":95995035648,"available":95966547968,"internally_reserved":0,"allocated":819200,"data_stored":345662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Feb 13 14:15:59 2024","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.35399999999999998,"15min":0.34399999999999997},"min":{"1min":0.219,"5min":0.217,"15min":0.217},"max":{"1min":0.47399999999999998,"5min":0.503,"15min":0.503},"last":0.27300000000000002},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.375,"15min":0.35799999999999998},"min":{"1min":0.23999999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.55900000000000005,"5min":0.57899999999999996,"15min":0.57899999999999996},"last":0.29099999999999998}]},{"osd":1,"last update":"Tue Feb 13 14:15:59 2024","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.36099999999999999,"15min":0.34399999999999997},"min":{"1min":0.27800000000000002,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.51500000000000001,"5min":0.58399999999999996,"15min":0.58399999999999996},"last":0.20999999999999999},{"interface":"front","average":{"1min":0.435,"5min":0.39400000000000002,"15min":0.36899999999999999},"min":{"1min":0.30199999999999999,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.33100000000000002}]},{"osd":3,"last update":"Tue Feb 13 14:16:20 2024","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.47499999999999998,"15min":0.48099999999999998},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.39600000000000002,"5min":0.44600000000000001,"15min":0.45400000000000001},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.57699999999999996,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.23699999999999999}]},{"osd":4,"last update":"Tue Feb 13 14:15:47 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.55700000000000005,"15min":0.55600000000000005},"min":{"1min":0.42699999999999999,"5min":0.371,"15min":0.371},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.59599999999999997,"15min":0.59399999999999997},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.76600000000000001,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.45200000000000001}]},{"osd":5,"last update":"Tue Feb 13 14:16:11 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.63400000000000001,"15min":0.64000000000000001},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.60399999999999998,"15min":0.60699999999999998},"min":{"1min":0.40200000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.437}]},{"osd":6,"last update":"Tue Feb 13 14:15:34 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.48199999999999998}]},{"osd":7,"last update":"Tue Feb 13 14:15:59 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.42399999999999999}]}]},{"osd":3,"up_from":26,"seq":111669149737,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32008,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713144,"statfs":{"total":95995035648,"available":95962259456,"internally_reserved":0,"allocated":847872,"data_stored":346494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Feb 13 14:16:21 2024","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.38200000000000001,"15min":0.38400000000000001},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.58999999999999997,"5min":0.63,"15min":0.63},"last":0.27700000000000002},{"interface":"front","average":{"1min":0.33800000000000002,"5min":0.40799999999999997,"15min":0.42699999999999999},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.45200000000000001,"5min":0.60699999999999998,"15min":0.60699999999999998},"last":0.33500000000000002}]},{"osd":1,"last update":"Tue Feb 13 14:16:21 2024","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.42599999999999999,"15min":0.439},"min":{"1min":0.26200000000000001,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.48299999999999998,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.29999999999999999},{"interface":"front","average":{"1min":0.35899999999999999,"5min":0.41199999999999998,"15min":0.42599999999999999},"min":{"1min":0.25600000000000001,"5min":0.216,"15min":0.216},"max":{"1min":0.56100000000000005,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.31900000000000001}]},{"osd":2,"last update":"Tue Feb 13 14:16:21 2024","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.44600000000000001,"15min":0.46200000000000002},"min":{"1min":0.28100000000000003,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.5,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.372},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.41999999999999998,"15min":0.42799999999999999},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.52400000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.34699999999999998}]},{"osd":4,"last update":"Tue Feb 13 14:15:41 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.54700000000000004,"15min":0.54900000000000004},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.40799999999999997}]},{"osd":5,"last update":"Tue Feb 13 14:16:08 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.57399999999999995,"15min":0.57599999999999996},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.71299999999999997,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.58399999999999996,"15min":0.59199999999999997},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.51700000000000002}]},{"osd":6,"last update":"Tue Feb 13 14:15:32 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.44}]},{"osd":7,"last update":"Tue Feb 13 14:16:00 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.42899999999999999}]}]},{"osd":4,"up_from":31,"seq":133143986212,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32004,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713148,"statfs":{"total":95995035648,"available":95962263552,"internally_reserved":0,"allocated":843776,"data_stored":346697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Feb 13 14:15:42 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.58599999999999997,"15min":0.58999999999999997},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.57799999999999996}]},{"osd":1,"last update":"Tue Feb 13 14:15:42 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.57299999999999995,"15min":0.57699999999999996},"min":{"1min":0.379,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.57099999999999995,"15min":0.57299999999999995},"min":{"1min":0.39900000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.77000000000000002,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.623}]},{"osd":2,"last update":"Tue Feb 13 14:15:42 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.56699999999999995,"15min":0.56899999999999995},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.56699999999999995,"15min":0.57099999999999995},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.54900000000000004}]},{"osd":3,"last update":"Tue Feb 13 14:15:42 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.57799999999999996,"15min":0.58199999999999996},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56000000000000005,"15min":0.55800000000000005},"min":{"1min":0.41699999999999998,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.79000000000000004,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.65600000000000003}]},{"osd":5,"last update":"Tue Feb 13 14:16:11 2024","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.42799999999999999,"15min":0.42499999999999999},"min":{"1min":0.29499999999999998,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.59399999999999997,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.433,"5min":0.433,"15min":0.433},"min":{"1min":0.32000000000000001,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.622,"5min":0.749,"15min":0.749},"last":0.71999999999999997}]},{"osd":6,"last update":"Tue Feb 13 14:15:33 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.495},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.74199999999999999}]},{"osd":7,"last update":"Tue Feb 13 14:15:58 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.47299999999999998}]}]},{"osd":5,"up_from":39,"seq":167503724574,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31976,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713176,"statfs":{"total":95995035648,"available":95962292224,"internally_reserved":0,"allocated":815104,"data_stored":345640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Feb 13 14:16:12 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.52400000000000002,"15min":0.52000000000000002},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.58999999999999997}]},{"osd":1,"last update":"Tue Feb 13 14:16:12 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.52300000000000002,"15min":0.52100000000000002},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.54400000000000004,"15min":0.54200000000000004},"min":{"1min":0.39800000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.63200000000000001}]},{"osd":2,"last update":"Tue Feb 13 14:16:12 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.53800000000000003,"15min":0.54000000000000004},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.80400000000000005,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.33900000000000002}]},{"osd":3,"last update":"Tue Feb 13 14:16:12 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.53100000000000003,"15min":0.53200000000000003},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.755,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.432},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.57499999999999996,"15min":0.57699999999999996},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.45300000000000001}]},{"osd":4,"last update":"Tue Feb 13 14:16:12 2024","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.371,"15min":0.36399999999999999},"min":{"1min":0.28000000000000003,"5min":0.214,"15min":0.214},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.38400000000000001,"15min":0.378},"min":{"1min":0.26400000000000001,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.54300000000000004}]},{"osd":6,"last update":"Tue Feb 13 14:15:32 2024","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.50800000000000001}]},{"osd":7,"last update":"Tue Feb 13 14:15:58 2024","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.61199999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-13T14:16:31.732 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662 shell --fsid 5c04b074-ca79-11ee-95b9-87774f69a715 -- ceph pg dump --format=json 2024-02-13T14:16:31.973 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/mon.a/config 2024-02-13T14:16:31.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:16:31 smithi087 ceph-mon[27372]: from='client.14706 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-13T14:16:31.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:16:31 smithi087 ceph-mon[39120]: from='client.14706 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-13T14:16:32.118 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:16:31 smithi148 ceph-mon[31198]: from='client.14706 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-13T14:16:33.118 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:16:32 smithi148 ceph-mon[31198]: pgmap v22: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-13T14:16:33.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:16:32 smithi087 ceph-mon[27372]: pgmap v22: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-13T14:16:33.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:16:32 smithi087 ceph-mon[39120]: pgmap v22: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-13T14:16:34.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:16:34 smithi087 ceph-mon[27372]: pgmap v23: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-13T14:16:34.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:16:34 smithi087 ceph-mon[39120]: pgmap v23: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-13T14:16:35.118 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:16:34 smithi148 ceph-mon[31198]: pgmap v23: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-13T14:16:35.119 INFO:journalctl@ceph.iscsi.iscsi.a.smithi148.stdout:Feb 13 14:16:34 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-iscsi-iscsi-a[59644]: debug there is no tcmu-runner data available 2024-02-13T14:16:35.991 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:16:35 smithi087 ceph-mon[27372]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:16:35.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:16:35 smithi087 ceph-mon[39120]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:16:36.118 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:16:35 smithi148 ceph-mon[31198]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:16:36.772 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-02-13T14:16:36.776 INFO:teuthology.orchestra.run.smithi087.stderr:dumped all 2024-02-13T14:16:36.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:16:36 smithi087 ceph-mon[27372]: pgmap v24: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:16:36.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:16:36 smithi087 ceph-mon[39120]: pgmap v24: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:16:37.118 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:16:36 smithi148 ceph-mon[31198]: pgmap v24: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:16:37.453 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:16:37 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: ::ffff:172.21.15.148 - - [13/Feb/2024:14:16:37] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-13T14:16:37.481 INFO:teuthology.orchestra.run.smithi087.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-02-13T14:16:35.745233+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596316,"num_objects":198,"num_object_clones":0,"num_object_copies":594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":198,"num_whiteouts":0,"num_read":1010,"num_read_kb":856,"num_write":615,"num_write_kb":2461,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":1771544,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":600,"ondisk_log_size":600,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":249364,"kb_used_data":8252,"kb_used_omap":12,"kb_used_meta":241011,"kb_avail":749711852,"statfs":{"total":767960285184,"available":767704936448,"internally_reserved":0,"allocated":8450048,"data_stored":4542164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":246795824},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002765"},"pg_stats":[{"pgid":"6.1a","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:54.276605+0000","last_change":"2024-02-13T14:15:18.215385+0000","last_active":"2024-02-13T14:15:54.276605+0000","last_peered":"2024-02-13T14:15:54.276605+0000","last_clean":"2024-02-13T14:15:54.276605+0000","last_became_active":"2024-02-13T14:15:18.215083+0000","last_became_peered":"2024-02-13T14:15:18.215083+0000","last_unstale":"2024-02-13T14:15:54.276605+0000","last_undegraded":"2024-02-13T14:15:54.276605+0000","last_fullsized":"2024-02-13T14:15:54.276605+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:17.185096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:17.185096+0000","last_clean_scrub_stamp":"2024-02-13T14:15:17.185096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T01:51:24.997993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:54.381794+0000","last_change":"2024-02-13T14:15:16.197449+0000","last_active":"2024-02-13T14:15:54.381794+0000","last_peered":"2024-02-13T14:15:54.381794+0000","last_clean":"2024-02-13T14:15:54.381794+0000","last_became_active":"2024-02-13T14:15:16.197108+0000","last_became_peered":"2024-02-13T14:15:16.197108+0000","last_unstale":"2024-02-13T14:15:54.381794+0000","last_undegraded":"2024-02-13T14:15:54.381794+0000","last_fullsized":"2024-02-13T14:15:54.381794+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:15.174019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:15.174019+0000","last_clean_scrub_stamp":"2024-02-13T14:15:15.174019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T15:09:12.659558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:54.772654+0000","last_change":"2024-02-13T14:15:12.416660+0000","last_active":"2024-02-13T14:15:54.772654+0000","last_peered":"2024-02-13T14:15:54.772654+0000","last_clean":"2024-02-13T14:15:54.772654+0000","last_became_active":"2024-02-13T14:15:12.416368+0000","last_became_peered":"2024-02-13T14:15:12.416368+0000","last_unstale":"2024-02-13T14:15:54.772654+0000","last_undegraded":"2024-02-13T14:15:54.772654+0000","last_fullsized":"2024-02-13T14:15:54.772654+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:11.150966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:11.150966+0000","last_clean_scrub_stamp":"2024-02-13T14:15:11.150966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T21:46:08.382574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"70'9","reported_seq":45,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:54.276057+0000","last_change":"2024-02-13T14:15:14.182625+0000","last_active":"2024-02-13T14:15:54.276057+0000","last_peered":"2024-02-13T14:15:54.276057+0000","last_clean":"2024-02-13T14:15:54.276057+0000","last_became_active":"2024-02-13T14:15:14.182282+0000","last_became_peered":"2024-02-13T14:15:14.182282+0000","last_unstale":"2024-02-13T14:15:54.276057+0000","last_undegraded":"2024-02-13T14:15:54.276057+0000","last_fullsized":"2024-02-13T14:15:54.276057+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:13.158947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:13.158947+0000","last_clean_scrub_stamp":"2024-02-13T14:15:13.158947+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T17:52:21.022729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.727607+0000","last_change":"2024-02-13T14:15:18.875889+0000","last_active":"2024-02-13T14:15:53.727607+0000","last_peered":"2024-02-13T14:15:53.727607+0000","last_clean":"2024-02-13T14:15:53.727607+0000","last_became_active":"2024-02-13T14:15:18.875348+0000","last_became_peered":"2024-02-13T14:15:18.875348+0000","last_unstale":"2024-02-13T14:15:53.727607+0000","last_undegraded":"2024-02-13T14:15:53.727607+0000","last_fullsized":"2024-02-13T14:15:53.727607+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:17.185096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:17.185096+0000","last_clean_scrub_stamp":"2024-02-13T14:15:17.185096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T18:21:06.075573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:54.276453+0000","last_change":"2024-02-13T14:15:16.197696+0000","last_active":"2024-02-13T14:15:54.276453+0000","last_peered":"2024-02-13T14:15:54.276453+0000","last_clean":"2024-02-13T14:15:54.276453+0000","last_became_active":"2024-02-13T14:15:16.197348+0000","last_became_peered":"2024-02-13T14:15:16.197348+0000","last_unstale":"2024-02-13T14:15:54.276453+0000","last_undegraded":"2024-02-13T14:15:54.276453+0000","last_fullsized":"2024-02-13T14:15:54.276453+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:15.174019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:15.174019+0000","last_clean_scrub_stamp":"2024-02-13T14:15:15.174019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T00:05:58.719527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.728384+0000","last_change":"2024-02-13T14:15:12.417522+0000","last_active":"2024-02-13T14:15:53.728384+0000","last_peered":"2024-02-13T14:15:53.728384+0000","last_clean":"2024-02-13T14:15:53.728384+0000","last_became_active":"2024-02-13T14:15:12.416419+0000","last_became_peered":"2024-02-13T14:15:12.416419+0000","last_unstale":"2024-02-13T14:15:53.728384+0000","last_undegraded":"2024-02-13T14:15:53.728384+0000","last_fullsized":"2024-02-13T14:15:53.728384+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:11.150966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:11.150966+0000","last_clean_scrub_stamp":"2024-02-13T14:15:11.150966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T18:01:22.555152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"70'15","reported_seq":54,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.727784+0000","last_change":"2024-02-13T14:15:14.173684+0000","last_active":"2024-02-13T14:15:53.727784+0000","last_peered":"2024-02-13T14:15:53.727784+0000","last_clean":"2024-02-13T14:15:53.727784+0000","last_became_active":"2024-02-13T14:15:14.173365+0000","last_became_peered":"2024-02-13T14:15:14.173365+0000","last_unstale":"2024-02-13T14:15:53.727784+0000","last_undegraded":"2024-02-13T14:15:53.727784+0000","last_fullsized":"2024-02-13T14:15:53.727784+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:13.158947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:13.158947+0000","last_clean_scrub_stamp":"2024-02-13T14:15:13.158947+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T20:34:45.824272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:54.772164+0000","last_change":"2024-02-13T14:15:18.215613+0000","last_active":"2024-02-13T14:15:54.772164+0000","last_peered":"2024-02-13T14:15:54.772164+0000","last_clean":"2024-02-13T14:15:54.772164+0000","last_became_active":"2024-02-13T14:15:18.215306+0000","last_became_peered":"2024-02-13T14:15:18.215306+0000","last_unstale":"2024-02-13T14:15:54.772164+0000","last_undegraded":"2024-02-13T14:15:54.772164+0000","last_fullsized":"2024-02-13T14:15:54.772164+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:17.185096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:17.185096+0000","last_clean_scrub_stamp":"2024-02-13T14:15:17.185096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T14:25:01.251773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.832598+0000","last_change":"2024-02-13T14:15:16.204071+0000","last_active":"2024-02-13T14:15:53.832598+0000","last_peered":"2024-02-13T14:15:53.832598+0000","last_clean":"2024-02-13T14:15:53.832598+0000","last_became_active":"2024-02-13T14:15:16.203759+0000","last_became_peered":"2024-02-13T14:15:16.203759+0000","last_unstale":"2024-02-13T14:15:53.832598+0000","last_undegraded":"2024-02-13T14:15:53.832598+0000","last_fullsized":"2024-02-13T14:15:53.832598+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:15.174019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:15.174019+0000","last_clean_scrub_stamp":"2024-02-13T14:15:15.174019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T19:28:19.698492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.832942+0000","last_change":"2024-02-13T14:15:12.175280+0000","last_active":"2024-02-13T14:15:53.832942+0000","last_peered":"2024-02-13T14:15:53.832942+0000","last_clean":"2024-02-13T14:15:53.832942+0000","last_became_active":"2024-02-13T14:15:12.174975+0000","last_became_peered":"2024-02-13T14:15:12.174975+0000","last_unstale":"2024-02-13T14:15:53.832942+0000","last_undegraded":"2024-02-13T14:15:53.832942+0000","last_fullsized":"2024-02-13T14:15:53.832942+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:11.150966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:11.150966+0000","last_clean_scrub_stamp":"2024-02-13T14:15:11.150966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T20:13:22.230505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"70'9","reported_seq":45,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:54.276060+0000","last_change":"2024-02-13T14:15:14.175228+0000","last_active":"2024-02-13T14:15:54.276060+0000","last_peered":"2024-02-13T14:15:54.276060+0000","last_clean":"2024-02-13T14:15:54.276060+0000","last_became_active":"2024-02-13T14:15:14.174677+0000","last_became_peered":"2024-02-13T14:15:14.174677+0000","last_unstale":"2024-02-13T14:15:54.276060+0000","last_undegraded":"2024-02-13T14:15:54.276060+0000","last_fullsized":"2024-02-13T14:15:54.276060+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:13.158947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:13.158947+0000","last_clean_scrub_stamp":"2024-02-13T14:15:13.158947+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T18:49:31.417441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.832366+0000","last_change":"2024-02-13T14:15:18.213356+0000","last_active":"2024-02-13T14:15:53.832366+0000","last_peered":"2024-02-13T14:15:53.832366+0000","last_clean":"2024-02-13T14:15:53.832366+0000","last_became_active":"2024-02-13T14:15:18.212835+0000","last_became_peered":"2024-02-13T14:15:18.212835+0000","last_unstale":"2024-02-13T14:15:53.832366+0000","last_undegraded":"2024-02-13T14:15:53.832366+0000","last_fullsized":"2024-02-13T14:15:53.832366+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:17.185096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:17.185096+0000","last_clean_scrub_stamp":"2024-02-13T14:15:17.185096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T22:35:59.324232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.727257+0000","last_change":"2024-02-13T14:15:16.200284+0000","last_active":"2024-02-13T14:15:53.727257+0000","last_peered":"2024-02-13T14:15:53.727257+0000","last_clean":"2024-02-13T14:15:53.727257+0000","last_became_active":"2024-02-13T14:15:16.199987+0000","last_became_peered":"2024-02-13T14:15:16.199987+0000","last_unstale":"2024-02-13T14:15:53.727257+0000","last_undegraded":"2024-02-13T14:15:53.727257+0000","last_fullsized":"2024-02-13T14:15:53.727257+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:15.174019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:15.174019+0000","last_clean_scrub_stamp":"2024-02-13T14:15:15.174019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T00:02:15.562600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.832679+0000","last_change":"2024-02-13T14:15:12.171453+0000","last_active":"2024-02-13T14:15:53.832679+0000","last_peered":"2024-02-13T14:15:53.832679+0000","last_clean":"2024-02-13T14:15:53.832679+0000","last_became_active":"2024-02-13T14:15:12.171075+0000","last_became_peered":"2024-02-13T14:15:12.171075+0000","last_unstale":"2024-02-13T14:15:53.832679+0000","last_undegraded":"2024-02-13T14:15:53.832679+0000","last_fullsized":"2024-02-13T14:15:53.832679+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:11.150966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:11.150966+0000","last_clean_scrub_stamp":"2024-02-13T14:15:11.150966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T19:41:07.694427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"70'5","reported_seq":39,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:54.276134+0000","last_change":"2024-02-13T14:15:14.174951+0000","last_active":"2024-02-13T14:15:54.276134+0000","last_peered":"2024-02-13T14:15:54.276134+0000","last_clean":"2024-02-13T14:15:54.276134+0000","last_became_active":"2024-02-13T14:15:14.174622+0000","last_became_peered":"2024-02-13T14:15:14.174622+0000","last_unstale":"2024-02-13T14:15:54.276134+0000","last_undegraded":"2024-02-13T14:15:54.276134+0000","last_fullsized":"2024-02-13T14:15:54.276134+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:13.158947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:13.158947+0000","last_clean_scrub_stamp":"2024-02-13T14:15:13.158947+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T23:02:13.770652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:54.277087+0000","last_change":"2024-02-13T14:15:18.874263+0000","last_active":"2024-02-13T14:15:54.277087+0000","last_peered":"2024-02-13T14:15:54.277087+0000","last_clean":"2024-02-13T14:15:54.277087+0000","last_became_active":"2024-02-13T14:15:18.873241+0000","last_became_peered":"2024-02-13T14:15:18.873241+0000","last_unstale":"2024-02-13T14:15:54.277087+0000","last_undegraded":"2024-02-13T14:15:54.277087+0000","last_fullsized":"2024-02-13T14:15:54.277087+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:17.185096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:17.185096+0000","last_clean_scrub_stamp":"2024-02-13T14:15:17.185096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T01:26:39.592611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:54.382447+0000","last_change":"2024-02-13T14:15:16.194648+0000","last_active":"2024-02-13T14:15:54.382447+0000","last_peered":"2024-02-13T14:15:54.382447+0000","last_clean":"2024-02-13T14:15:54.382447+0000","last_became_active":"2024-02-13T14:15:16.193624+0000","last_became_peered":"2024-02-13T14:15:16.193624+0000","last_unstale":"2024-02-13T14:15:54.382447+0000","last_undegraded":"2024-02-13T14:15:54.382447+0000","last_fullsized":"2024-02-13T14:15:54.382447+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:15.174019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:15.174019+0000","last_clean_scrub_stamp":"2024-02-13T14:15:15.174019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T16:13:17.644055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:54.772619+0000","last_change":"2024-02-13T14:15:12.175084+0000","last_active":"2024-02-13T14:15:54.772619+0000","last_peered":"2024-02-13T14:15:54.772619+0000","last_clean":"2024-02-13T14:15:54.772619+0000","last_became_active":"2024-02-13T14:15:12.174565+0000","last_became_peered":"2024-02-13T14:15:12.174565+0000","last_unstale":"2024-02-13T14:15:54.772619+0000","last_undegraded":"2024-02-13T14:15:54.772619+0000","last_fullsized":"2024-02-13T14:15:54.772619+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:11.150966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:11.150966+0000","last_clean_scrub_stamp":"2024-02-13T14:15:11.150966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T17:36:04.928340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"70'15","reported_seq":54,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.838258+0000","last_change":"2024-02-13T14:15:14.176409+0000","last_active":"2024-02-13T14:15:53.838258+0000","last_peered":"2024-02-13T14:15:53.838258+0000","last_clean":"2024-02-13T14:15:53.838258+0000","last_became_active":"2024-02-13T14:15:14.176071+0000","last_became_peered":"2024-02-13T14:15:14.176071+0000","last_unstale":"2024-02-13T14:15:53.838258+0000","last_undegraded":"2024-02-13T14:15:53.838258+0000","last_fullsized":"2024-02-13T14:15:53.838258+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:13.158947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:13.158947+0000","last_clean_scrub_stamp":"2024-02-13T14:15:13.158947+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T15:58:59.517593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.728628+0000","last_change":"2024-02-13T14:15:18.873756+0000","last_active":"2024-02-13T14:15:53.728628+0000","last_peered":"2024-02-13T14:15:53.728628+0000","last_clean":"2024-02-13T14:15:53.728628+0000","last_became_active":"2024-02-13T14:15:18.873280+0000","last_became_peered":"2024-02-13T14:15:18.873280+0000","last_unstale":"2024-02-13T14:15:53.728628+0000","last_undegraded":"2024-02-13T14:15:53.728628+0000","last_fullsized":"2024-02-13T14:15:53.728628+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:17.185096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:17.185096+0000","last_clean_scrub_stamp":"2024-02-13T14:15:17.185096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T01:44:10.218859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:54.276823+0000","last_change":"2024-02-13T14:15:16.199416+0000","last_active":"2024-02-13T14:15:54.276823+0000","last_peered":"2024-02-13T14:15:54.276823+0000","last_clean":"2024-02-13T14:15:54.276823+0000","last_became_active":"2024-02-13T14:15:16.199144+0000","last_became_peered":"2024-02-13T14:15:16.199144+0000","last_unstale":"2024-02-13T14:15:54.276823+0000","last_undegraded":"2024-02-13T14:15:54.276823+0000","last_fullsized":"2024-02-13T14:15:54.276823+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:15.174019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:15.174019+0000","last_clean_scrub_stamp":"2024-02-13T14:15:15.174019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T17:20:20.840702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:54.276761+0000","last_change":"2024-02-13T14:15:12.417462+0000","last_active":"2024-02-13T14:15:54.276761+0000","last_peered":"2024-02-13T14:15:54.276761+0000","last_clean":"2024-02-13T14:15:54.276761+0000","last_became_active":"2024-02-13T14:15:12.416542+0000","last_became_peered":"2024-02-13T14:15:12.416542+0000","last_unstale":"2024-02-13T14:15:54.276761+0000","last_undegraded":"2024-02-13T14:15:54.276761+0000","last_fullsized":"2024-02-13T14:15:54.276761+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:11.150966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:11.150966+0000","last_clean_scrub_stamp":"2024-02-13T14:15:11.150966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T01:29:09.612554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"70'12","reported_seq":52,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.727941+0000","last_change":"2024-02-13T14:15:14.176872+0000","last_active":"2024-02-13T14:15:53.727941+0000","last_peered":"2024-02-13T14:15:53.727941+0000","last_clean":"2024-02-13T14:15:53.727941+0000","last_became_active":"2024-02-13T14:15:14.176550+0000","last_became_peered":"2024-02-13T14:15:14.176550+0000","last_unstale":"2024-02-13T14:15:53.727941+0000","last_undegraded":"2024-02-13T14:15:53.727941+0000","last_fullsized":"2024-02-13T14:15:53.727941+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:13.158947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:13.158947+0000","last_clean_scrub_stamp":"2024-02-13T14:15:13.158947+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T16:24:43.824185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"70'1","reported_seq":22,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.727510+0000","last_change":"2024-02-13T14:15:18.219104+0000","last_active":"2024-02-13T14:15:53.727510+0000","last_peered":"2024-02-13T14:15:53.727510+0000","last_clean":"2024-02-13T14:15:53.727510+0000","last_became_active":"2024-02-13T14:15:18.218782+0000","last_became_peered":"2024-02-13T14:15:18.218782+0000","last_unstale":"2024-02-13T14:15:53.727510+0000","last_undegraded":"2024-02-13T14:15:53.727510+0000","last_fullsized":"2024-02-13T14:15:53.727510+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:17.185096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:17.185096+0000","last_clean_scrub_stamp":"2024-02-13T14:15:17.185096+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T14:39:58.906671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"70'8","reported_seq":53,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:16:27.468567+0000","last_change":"2024-02-13T14:15:16.201480+0000","last_active":"2024-02-13T14:16:27.468567+0000","last_peered":"2024-02-13T14:16:27.468567+0000","last_clean":"2024-02-13T14:16:27.468567+0000","last_became_active":"2024-02-13T14:15:16.200392+0000","last_became_peered":"2024-02-13T14:15:16.200392+0000","last_unstale":"2024-02-13T14:16:27.468567+0000","last_undegraded":"2024-02-13T14:16:27.468567+0000","last_fullsized":"2024-02-13T14:16:27.468567+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:15.174019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:15.174019+0000","last_clean_scrub_stamp":"2024-02-13T14:15:15.174019+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T14:41:31.533705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"63'1","reported_seq":34,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:54.381915+0000","last_change":"2024-02-13T14:15:12.167170+0000","last_active":"2024-02-13T14:15:54.381915+0000","last_peered":"2024-02-13T14:15:54.381915+0000","last_clean":"2024-02-13T14:15:54.381915+0000","last_became_active":"2024-02-13T14:15:12.166788+0000","last_became_peered":"2024-02-13T14:15:12.166788+0000","last_unstale":"2024-02-13T14:15:54.381915+0000","last_undegraded":"2024-02-13T14:15:54.381915+0000","last_fullsized":"2024-02-13T14:15:54.381915+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:11.150966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:11.150966+0000","last_clean_scrub_stamp":"2024-02-13T14:15:11.150966+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T19:28:25.682466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"70'10","reported_seq":44,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:54.772065+0000","last_change":"2024-02-13T14:15:14.333051+0000","last_active":"2024-02-13T14:15:54.772065+0000","last_peered":"2024-02-13T14:15:54.772065+0000","last_clean":"2024-02-13T14:15:54.772065+0000","last_became_active":"2024-02-13T14:15:14.332735+0000","last_became_peered":"2024-02-13T14:15:14.332735+0000","last_unstale":"2024-02-13T14:15:54.772065+0000","last_undegraded":"2024-02-13T14:15:54.772065+0000","last_fullsized":"2024-02-13T14:15:54.772065+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:13.158947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:13.158947+0000","last_clean_scrub_stamp":"2024-02-13T14:15:13.158947+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T15:51:21.504527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:54.382509+0000","last_change":"2024-02-13T14:15:18.214477+0000","last_active":"2024-02-13T14:15:54.382509+0000","last_peered":"2024-02-13T14:15:54.382509+0000","last_clean":"2024-02-13T14:15:54.382509+0000","last_became_active":"2024-02-13T14:15:18.214036+0000","last_became_peered":"2024-02-13T14:15:18.214036+0000","last_unstale":"2024-02-13T14:15:54.382509+0000","last_undegraded":"2024-02-13T14:15:54.382509+0000","last_fullsized":"2024-02-13T14:15:54.382509+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:17.185096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:17.185096+0000","last_clean_scrub_stamp":"2024-02-13T14:15:17.185096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T19:01:01.536932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:54.772769+0000","last_change":"2024-02-13T14:15:16.201752+0000","last_active":"2024-02-13T14:15:54.772769+0000","last_peered":"2024-02-13T14:15:54.772769+0000","last_clean":"2024-02-13T14:15:54.772769+0000","last_became_active":"2024-02-13T14:15:16.201317+0000","last_became_peered":"2024-02-13T14:15:16.201317+0000","last_unstale":"2024-02-13T14:15:54.772769+0000","last_undegraded":"2024-02-13T14:15:54.772769+0000","last_fullsized":"2024-02-13T14:15:54.772769+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:15.174019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:15.174019+0000","last_clean_scrub_stamp":"2024-02-13T14:15:15.174019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T23:23:53.006168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.726540+0000","last_change":"2024-02-13T14:15:12.171305+0000","last_active":"2024-02-13T14:15:53.726540+0000","last_peered":"2024-02-13T14:15:53.726540+0000","last_clean":"2024-02-13T14:15:53.726540+0000","last_became_active":"2024-02-13T14:15:12.170518+0000","last_became_peered":"2024-02-13T14:15:12.170518+0000","last_unstale":"2024-02-13T14:15:53.726540+0000","last_undegraded":"2024-02-13T14:15:53.726540+0000","last_fullsized":"2024-02-13T14:15:53.726540+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:11.150966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:11.150966+0000","last_clean_scrub_stamp":"2024-02-13T14:15:11.150966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T15:37:46.154891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"70'11","reported_seq":48,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.730891+0000","last_change":"2024-02-13T14:15:14.180906+0000","last_active":"2024-02-13T14:15:53.730891+0000","last_peered":"2024-02-13T14:15:53.730891+0000","last_clean":"2024-02-13T14:15:53.730891+0000","last_became_active":"2024-02-13T14:15:14.180481+0000","last_became_peered":"2024-02-13T14:15:14.180481+0000","last_unstale":"2024-02-13T14:15:53.730891+0000","last_undegraded":"2024-02-13T14:15:53.730891+0000","last_fullsized":"2024-02-13T14:15:53.730891+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:13.158947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:13.158947+0000","last_clean_scrub_stamp":"2024-02-13T14:15:13.158947+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T01:36:05.342954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.727598+0000","last_change":"2024-02-13T14:15:18.219071+0000","last_active":"2024-02-13T14:15:53.727598+0000","last_peered":"2024-02-13T14:15:53.727598+0000","last_clean":"2024-02-13T14:15:53.727598+0000","last_became_active":"2024-02-13T14:15:18.218760+0000","last_became_peered":"2024-02-13T14:15:18.218760+0000","last_unstale":"2024-02-13T14:15:53.727598+0000","last_undegraded":"2024-02-13T14:15:53.727598+0000","last_fullsized":"2024-02-13T14:15:53.727598+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:17.185096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:17.185096+0000","last_clean_scrub_stamp":"2024-02-13T14:15:17.185096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T00:29:40.070824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.728909+0000","last_change":"2024-02-13T14:15:16.199663+0000","last_active":"2024-02-13T14:15:53.728909+0000","last_peered":"2024-02-13T14:15:53.728909+0000","last_clean":"2024-02-13T14:15:53.728909+0000","last_became_active":"2024-02-13T14:15:16.199341+0000","last_became_peered":"2024-02-13T14:15:16.199341+0000","last_unstale":"2024-02-13T14:15:53.728909+0000","last_undegraded":"2024-02-13T14:15:53.728909+0000","last_fullsized":"2024-02-13T14:15:53.728909+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:15.174019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:15.174019+0000","last_clean_scrub_stamp":"2024-02-13T14:15:15.174019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T15:41:09.625967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:54.772621+0000","last_change":"2024-02-13T14:15:12.173489+0000","last_active":"2024-02-13T14:15:54.772621+0000","last_peered":"2024-02-13T14:15:54.772621+0000","last_clean":"2024-02-13T14:15:54.772621+0000","last_became_active":"2024-02-13T14:15:12.173203+0000","last_became_peered":"2024-02-13T14:15:12.173203+0000","last_unstale":"2024-02-13T14:15:54.772621+0000","last_undegraded":"2024-02-13T14:15:54.772621+0000","last_fullsized":"2024-02-13T14:15:54.772621+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:11.150966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:11.150966+0000","last_clean_scrub_stamp":"2024-02-13T14:15:11.150966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T22:59:34.210780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"70'4","reported_seq":35,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.726572+0000","last_change":"2024-02-13T14:15:14.180694+0000","last_active":"2024-02-13T14:15:53.726572+0000","last_peered":"2024-02-13T14:15:53.726572+0000","last_clean":"2024-02-13T14:15:53.726572+0000","last_became_active":"2024-02-13T14:15:14.180386+0000","last_became_peered":"2024-02-13T14:15:14.180386+0000","last_unstale":"2024-02-13T14:15:53.726572+0000","last_undegraded":"2024-02-13T14:15:53.726572+0000","last_fullsized":"2024-02-13T14:15:53.726572+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:13.158947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:13.158947+0000","last_clean_scrub_stamp":"2024-02-13T14:15:13.158947+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T01:29:28.346535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.727438+0000","last_change":"2024-02-13T14:15:18.875994+0000","last_active":"2024-02-13T14:15:53.727438+0000","last_peered":"2024-02-13T14:15:53.727438+0000","last_clean":"2024-02-13T14:15:53.727438+0000","last_became_active":"2024-02-13T14:15:18.875321+0000","last_became_peered":"2024-02-13T14:15:18.875321+0000","last_unstale":"2024-02-13T14:15:53.727438+0000","last_undegraded":"2024-02-13T14:15:53.727438+0000","last_fullsized":"2024-02-13T14:15:53.727438+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:17.185096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:17.185096+0000","last_clean_scrub_stamp":"2024-02-13T14:15:17.185096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T20:04:22.089282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.727408+0000","last_change":"2024-02-13T14:15:16.199984+0000","last_active":"2024-02-13T14:15:53.727408+0000","last_peered":"2024-02-13T14:15:53.727408+0000","last_clean":"2024-02-13T14:15:53.727408+0000","last_became_active":"2024-02-13T14:15:16.199493+0000","last_became_peered":"2024-02-13T14:15:16.199493+0000","last_unstale":"2024-02-13T14:15:53.727408+0000","last_undegraded":"2024-02-13T14:15:53.727408+0000","last_fullsized":"2024-02-13T14:15:53.727408+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:15.174019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:15.174019+0000","last_clean_scrub_stamp":"2024-02-13T14:15:15.174019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T23:45:53.157045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.832910+0000","last_change":"2024-02-13T14:15:12.173488+0000","last_active":"2024-02-13T14:15:53.832910+0000","last_peered":"2024-02-13T14:15:53.832910+0000","last_clean":"2024-02-13T14:15:53.832910+0000","last_became_active":"2024-02-13T14:15:12.173205+0000","last_became_peered":"2024-02-13T14:15:12.173205+0000","last_unstale":"2024-02-13T14:15:53.832910+0000","last_undegraded":"2024-02-13T14:15:53.832910+0000","last_fullsized":"2024-02-13T14:15:53.832910+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:11.150966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:11.150966+0000","last_clean_scrub_stamp":"2024-02-13T14:15:11.150966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T22:00:07.329240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"70'11","reported_seq":48,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.727644+0000","last_change":"2024-02-13T14:15:14.333637+0000","last_active":"2024-02-13T14:15:53.727644+0000","last_peered":"2024-02-13T14:15:53.727644+0000","last_clean":"2024-02-13T14:15:53.727644+0000","last_became_active":"2024-02-13T14:15:14.333072+0000","last_became_peered":"2024-02-13T14:15:14.333072+0000","last_unstale":"2024-02-13T14:15:53.727644+0000","last_undegraded":"2024-02-13T14:15:53.727644+0000","last_fullsized":"2024-02-13T14:15:53.727644+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:13.158947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:13.158947+0000","last_clean_scrub_stamp":"2024-02-13T14:15:13.158947+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T16:19:33.936306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:54.771971+0000","last_change":"2024-02-13T14:15:18.218795+0000","last_active":"2024-02-13T14:15:54.771971+0000","last_peered":"2024-02-13T14:15:54.771971+0000","last_clean":"2024-02-13T14:15:54.771971+0000","last_became_active":"2024-02-13T14:15:18.218491+0000","last_became_peered":"2024-02-13T14:15:18.218491+0000","last_unstale":"2024-02-13T14:15:54.771971+0000","last_undegraded":"2024-02-13T14:15:54.771971+0000","last_fullsized":"2024-02-13T14:15:54.771971+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:17.185096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:17.185096+0000","last_clean_scrub_stamp":"2024-02-13T14:15:17.185096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T15:12:15.731109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.727650+0000","last_change":"2024-02-13T14:15:16.197039+0000","last_active":"2024-02-13T14:15:53.727650+0000","last_peered":"2024-02-13T14:15:53.727650+0000","last_clean":"2024-02-13T14:15:53.727650+0000","last_became_active":"2024-02-13T14:15:16.196673+0000","last_became_peered":"2024-02-13T14:15:16.196673+0000","last_unstale":"2024-02-13T14:15:53.727650+0000","last_undegraded":"2024-02-13T14:15:53.727650+0000","last_fullsized":"2024-02-13T14:15:53.727650+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:15.174019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:15.174019+0000","last_clean_scrub_stamp":"2024-02-13T14:15:15.174019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T17:45:32.446887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.727509+0000","last_change":"2024-02-13T14:15:12.177327+0000","last_active":"2024-02-13T14:15:53.727509+0000","last_peered":"2024-02-13T14:15:53.727509+0000","last_clean":"2024-02-13T14:15:53.727509+0000","last_became_active":"2024-02-13T14:15:12.177032+0000","last_became_peered":"2024-02-13T14:15:12.177032+0000","last_unstale":"2024-02-13T14:15:53.727509+0000","last_undegraded":"2024-02-13T14:15:53.727509+0000","last_fullsized":"2024-02-13T14:15:53.727509+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:11.150966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:11.150966+0000","last_clean_scrub_stamp":"2024-02-13T14:15:11.150966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T17:44:29.588326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"70'9","reported_seq":45,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:54.381099+0000","last_change":"2024-02-13T14:15:14.179255+0000","last_active":"2024-02-13T14:15:54.381099+0000","last_peered":"2024-02-13T14:15:54.381099+0000","last_clean":"2024-02-13T14:15:54.381099+0000","last_became_active":"2024-02-13T14:15:14.178956+0000","last_became_peered":"2024-02-13T14:15:14.178956+0000","last_unstale":"2024-02-13T14:15:54.381099+0000","last_undegraded":"2024-02-13T14:15:54.381099+0000","last_fullsized":"2024-02-13T14:15:54.381099+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:13.158947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:13.158947+0000","last_clean_scrub_stamp":"2024-02-13T14:15:13.158947+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T16:59:48.021624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.727458+0000","last_change":"2024-02-13T14:15:18.211825+0000","last_active":"2024-02-13T14:15:53.727458+0000","last_peered":"2024-02-13T14:15:53.727458+0000","last_clean":"2024-02-13T14:15:53.727458+0000","last_became_active":"2024-02-13T14:15:18.211498+0000","last_became_peered":"2024-02-13T14:15:18.211498+0000","last_unstale":"2024-02-13T14:15:53.727458+0000","last_undegraded":"2024-02-13T14:15:53.727458+0000","last_fullsized":"2024-02-13T14:15:53.727458+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:17.185096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:17.185096+0000","last_clean_scrub_stamp":"2024-02-13T14:15:17.185096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T15:01:47.997499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:54.382198+0000","last_change":"2024-02-13T14:15:16.200746+0000","last_active":"2024-02-13T14:15:54.382198+0000","last_peered":"2024-02-13T14:15:54.382198+0000","last_clean":"2024-02-13T14:15:54.382198+0000","last_became_active":"2024-02-13T14:15:16.200284+0000","last_became_peered":"2024-02-13T14:15:16.200284+0000","last_unstale":"2024-02-13T14:15:54.382198+0000","last_undegraded":"2024-02-13T14:15:54.382198+0000","last_fullsized":"2024-02-13T14:15:54.382198+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:15.174019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:15.174019+0000","last_clean_scrub_stamp":"2024-02-13T14:15:15.174019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T16:47:01.666512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:54.276944+0000","last_change":"2024-02-13T14:15:12.178954+0000","last_active":"2024-02-13T14:15:54.276944+0000","last_peered":"2024-02-13T14:15:54.276944+0000","last_clean":"2024-02-13T14:15:54.276944+0000","last_became_active":"2024-02-13T14:15:12.178663+0000","last_became_peered":"2024-02-13T14:15:12.178663+0000","last_unstale":"2024-02-13T14:15:54.276944+0000","last_undegraded":"2024-02-13T14:15:54.276944+0000","last_fullsized":"2024-02-13T14:15:54.276944+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:11.150966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:11.150966+0000","last_clean_scrub_stamp":"2024-02-13T14:15:11.150966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T23:20:12.238787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"70'11","reported_seq":48,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:54.276085+0000","last_change":"2024-02-13T14:15:14.176562+0000","last_active":"2024-02-13T14:15:54.276085+0000","last_peered":"2024-02-13T14:15:54.276085+0000","last_clean":"2024-02-13T14:15:54.276085+0000","last_became_active":"2024-02-13T14:15:14.176172+0000","last_became_peered":"2024-02-13T14:15:14.176172+0000","last_unstale":"2024-02-13T14:15:54.276085+0000","last_undegraded":"2024-02-13T14:15:54.276085+0000","last_fullsized":"2024-02-13T14:15:54.276085+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:13.158947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:13.158947+0000","last_clean_scrub_stamp":"2024-02-13T14:15:13.158947+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T00:04:18.752904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:54.772402+0000","last_change":"2024-02-13T14:15:18.211512+0000","last_active":"2024-02-13T14:15:54.772402+0000","last_peered":"2024-02-13T14:15:54.772402+0000","last_clean":"2024-02-13T14:15:54.772402+0000","last_became_active":"2024-02-13T14:15:18.211176+0000","last_became_peered":"2024-02-13T14:15:18.211176+0000","last_unstale":"2024-02-13T14:15:54.772402+0000","last_undegraded":"2024-02-13T14:15:54.772402+0000","last_fullsized":"2024-02-13T14:15:54.772402+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:17.185096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:17.185096+0000","last_clean_scrub_stamp":"2024-02-13T14:15:17.185096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T21:55:58.483380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"70'8","reported_seq":53,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:16:27.468568+0000","last_change":"2024-02-13T14:15:16.199356+0000","last_active":"2024-02-13T14:16:27.468568+0000","last_peered":"2024-02-13T14:16:27.468568+0000","last_clean":"2024-02-13T14:16:27.468568+0000","last_became_active":"2024-02-13T14:15:16.199090+0000","last_became_peered":"2024-02-13T14:15:16.199090+0000","last_unstale":"2024-02-13T14:16:27.468568+0000","last_undegraded":"2024-02-13T14:16:27.468568+0000","last_fullsized":"2024-02-13T14:16:27.468568+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:15.174019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:15.174019+0000","last_clean_scrub_stamp":"2024-02-13T14:15:15.174019+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T22:57:22.420105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.727651+0000","last_change":"2024-02-13T14:15:12.417131+0000","last_active":"2024-02-13T14:15:53.727651+0000","last_peered":"2024-02-13T14:15:53.727651+0000","last_clean":"2024-02-13T14:15:53.727651+0000","last_became_active":"2024-02-13T14:15:12.416631+0000","last_became_peered":"2024-02-13T14:15:12.416631+0000","last_unstale":"2024-02-13T14:15:53.727651+0000","last_undegraded":"2024-02-13T14:15:53.727651+0000","last_fullsized":"2024-02-13T14:15:53.727651+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:11.150966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:11.150966+0000","last_clean_scrub_stamp":"2024-02-13T14:15:11.150966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T20:20:28.320751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"70'10","reported_seq":62,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.728017+0000","last_change":"2024-02-13T14:15:30.019823+0000","last_active":"2024-02-13T14:15:53.728017+0000","last_peered":"2024-02-13T14:15:53.728017+0000","last_clean":"2024-02-13T14:15:53.728017+0000","last_became_active":"2024-02-13T14:15:30.011725+0000","last_became_peered":"2024-02-13T14:15:30.011725+0000","last_unstale":"2024-02-13T14:15:53.728017+0000","last_undegraded":"2024-02-13T14:15:53.728017+0000","last_fullsized":"2024-02-13T14:15:53.728017+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:13.158947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:13.158947+0000","last_clean_scrub_stamp":"2024-02-13T14:15:13.158947+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T02:05:15.487998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7],"acting":[3,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:54.277318+0000","last_change":"2024-02-13T14:15:18.215446+0000","last_active":"2024-02-13T14:15:54.277318+0000","last_peered":"2024-02-13T14:15:54.277318+0000","last_clean":"2024-02-13T14:15:54.277318+0000","last_became_active":"2024-02-13T14:15:18.215110+0000","last_became_peered":"2024-02-13T14:15:18.215110+0000","last_unstale":"2024-02-13T14:15:54.277318+0000","last_undegraded":"2024-02-13T14:15:54.277318+0000","last_fullsized":"2024-02-13T14:15:54.277318+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:17.185096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:17.185096+0000","last_clean_scrub_stamp":"2024-02-13T14:15:17.185096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T21:38:04.553920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"70'8","reported_seq":50,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:16:27.468545+0000","last_change":"2024-02-13T14:15:16.203356+0000","last_active":"2024-02-13T14:16:27.468545+0000","last_peered":"2024-02-13T14:16:27.468545+0000","last_clean":"2024-02-13T14:16:27.468545+0000","last_became_active":"2024-02-13T14:15:16.203006+0000","last_became_peered":"2024-02-13T14:15:16.203006+0000","last_unstale":"2024-02-13T14:16:27.468545+0000","last_undegraded":"2024-02-13T14:16:27.468545+0000","last_fullsized":"2024-02-13T14:16:27.468545+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:15.174019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:15.174019+0000","last_clean_scrub_stamp":"2024-02-13T14:15:15.174019+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T19:58:33.733272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"63'1","reported_seq":39,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:54.772241+0000","last_change":"2024-02-13T14:15:12.173688+0000","last_active":"2024-02-13T14:15:54.772241+0000","last_peered":"2024-02-13T14:15:54.772241+0000","last_clean":"2024-02-13T14:15:54.772241+0000","last_became_active":"2024-02-13T14:15:12.172963+0000","last_became_peered":"2024-02-13T14:15:12.172963+0000","last_unstale":"2024-02-13T14:15:54.772241+0000","last_undegraded":"2024-02-13T14:15:54.772241+0000","last_fullsized":"2024-02-13T14:15:54.772241+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:11.150966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:11.150966+0000","last_clean_scrub_stamp":"2024-02-13T14:15:11.150966+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T00:30:50.366918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":398,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"70'9","reported_seq":45,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.833108+0000","last_change":"2024-02-13T14:15:14.333209+0000","last_active":"2024-02-13T14:15:53.833108+0000","last_peered":"2024-02-13T14:15:53.833108+0000","last_clean":"2024-02-13T14:15:53.833108+0000","last_became_active":"2024-02-13T14:15:14.332864+0000","last_became_peered":"2024-02-13T14:15:14.332864+0000","last_unstale":"2024-02-13T14:15:53.833108+0000","last_undegraded":"2024-02-13T14:15:53.833108+0000","last_fullsized":"2024-02-13T14:15:53.833108+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:13.158947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:13.158947+0000","last_clean_scrub_stamp":"2024-02-13T14:15:13.158947+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T01:00:35.182771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.838631+0000","last_change":"2024-02-13T14:15:18.217693+0000","last_active":"2024-02-13T14:15:53.838631+0000","last_peered":"2024-02-13T14:15:53.838631+0000","last_clean":"2024-02-13T14:15:53.838631+0000","last_became_active":"2024-02-13T14:15:18.216672+0000","last_became_peered":"2024-02-13T14:15:18.216672+0000","last_unstale":"2024-02-13T14:15:53.838631+0000","last_undegraded":"2024-02-13T14:15:53.838631+0000","last_fullsized":"2024-02-13T14:15:53.838631+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:17.185096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:17.185096+0000","last_clean_scrub_stamp":"2024-02-13T14:15:17.185096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T21:29:02.134774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.728611+0000","last_change":"2024-02-13T14:15:16.198284+0000","last_active":"2024-02-13T14:15:53.728611+0000","last_peered":"2024-02-13T14:15:53.728611+0000","last_clean":"2024-02-13T14:15:53.728611+0000","last_became_active":"2024-02-13T14:15:16.197873+0000","last_became_peered":"2024-02-13T14:15:16.197873+0000","last_unstale":"2024-02-13T14:15:53.728611+0000","last_undegraded":"2024-02-13T14:15:53.728611+0000","last_fullsized":"2024-02-13T14:15:53.728611+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:15.174019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:15.174019+0000","last_clean_scrub_stamp":"2024-02-13T14:15:15.174019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T19:22:19.615613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.727457+0000","last_change":"2024-02-13T14:15:12.175577+0000","last_active":"2024-02-13T14:15:53.727457+0000","last_peered":"2024-02-13T14:15:53.727457+0000","last_clean":"2024-02-13T14:15:53.727457+0000","last_became_active":"2024-02-13T14:15:12.175265+0000","last_became_peered":"2024-02-13T14:15:12.175265+0000","last_unstale":"2024-02-13T14:15:53.727457+0000","last_undegraded":"2024-02-13T14:15:53.727457+0000","last_fullsized":"2024-02-13T14:15:53.727457+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:11.150966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:11.150966+0000","last_clean_scrub_stamp":"2024-02-13T14:15:11.150966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T15:45:30.799142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"70'9","reported_seq":45,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:54.771842+0000","last_change":"2024-02-13T14:15:14.332529+0000","last_active":"2024-02-13T14:15:54.771842+0000","last_peered":"2024-02-13T14:15:54.771842+0000","last_clean":"2024-02-13T14:15:54.771842+0000","last_became_active":"2024-02-13T14:15:14.332186+0000","last_became_peered":"2024-02-13T14:15:14.332186+0000","last_unstale":"2024-02-13T14:15:54.771842+0000","last_undegraded":"2024-02-13T14:15:54.771842+0000","last_fullsized":"2024-02-13T14:15:54.771842+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:13.158947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:13.158947+0000","last_clean_scrub_stamp":"2024-02-13T14:15:13.158947+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T18:03:39.979457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.727342+0000","last_change":"2024-02-13T14:15:18.876696+0000","last_active":"2024-02-13T14:15:53.727342+0000","last_peered":"2024-02-13T14:15:53.727342+0000","last_clean":"2024-02-13T14:15:53.727342+0000","last_became_active":"2024-02-13T14:15:18.876391+0000","last_became_peered":"2024-02-13T14:15:18.876391+0000","last_unstale":"2024-02-13T14:15:53.727342+0000","last_undegraded":"2024-02-13T14:15:53.727342+0000","last_fullsized":"2024-02-13T14:15:53.727342+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:17.185096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:17.185096+0000","last_clean_scrub_stamp":"2024-02-13T14:15:17.185096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T00:24:21.216608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.832943+0000","last_change":"2024-02-13T14:15:16.199438+0000","last_active":"2024-02-13T14:15:53.832943+0000","last_peered":"2024-02-13T14:15:53.832943+0000","last_clean":"2024-02-13T14:15:53.832943+0000","last_became_active":"2024-02-13T14:15:16.199179+0000","last_became_peered":"2024-02-13T14:15:16.199179+0000","last_unstale":"2024-02-13T14:15:53.832943+0000","last_undegraded":"2024-02-13T14:15:53.832943+0000","last_fullsized":"2024-02-13T14:15:53.832943+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:15.174019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:15.174019+0000","last_clean_scrub_stamp":"2024-02-13T14:15:15.174019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T21:43:27.215796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.728424+0000","last_change":"2024-02-13T14:15:12.175949+0000","last_active":"2024-02-13T14:15:53.728424+0000","last_peered":"2024-02-13T14:15:53.728424+0000","last_clean":"2024-02-13T14:15:53.728424+0000","last_became_active":"2024-02-13T14:15:12.175701+0000","last_became_peered":"2024-02-13T14:15:12.175701+0000","last_unstale":"2024-02-13T14:15:53.728424+0000","last_undegraded":"2024-02-13T14:15:53.728424+0000","last_fullsized":"2024-02-13T14:15:53.728424+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:11.150966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:11.150966+0000","last_clean_scrub_stamp":"2024-02-13T14:15:11.150966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T00:26:32.094209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"70'6","reported_seq":38,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.728697+0000","last_change":"2024-02-13T14:15:14.334485+0000","last_active":"2024-02-13T14:15:53.728697+0000","last_peered":"2024-02-13T14:15:53.728697+0000","last_clean":"2024-02-13T14:15:53.728697+0000","last_became_active":"2024-02-13T14:15:14.333615+0000","last_became_peered":"2024-02-13T14:15:14.333615+0000","last_unstale":"2024-02-13T14:15:53.728697+0000","last_undegraded":"2024-02-13T14:15:53.728697+0000","last_fullsized":"2024-02-13T14:15:53.728697+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:13.158947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:13.158947+0000","last_clean_scrub_stamp":"2024-02-13T14:15:13.158947+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T18:09:09.842222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.832755+0000","last_change":"2024-02-13T14:15:18.874574+0000","last_active":"2024-02-13T14:15:53.832755+0000","last_peered":"2024-02-13T14:15:53.832755+0000","last_clean":"2024-02-13T14:15:53.832755+0000","last_became_active":"2024-02-13T14:15:18.874296+0000","last_became_peered":"2024-02-13T14:15:18.874296+0000","last_unstale":"2024-02-13T14:15:53.832755+0000","last_undegraded":"2024-02-13T14:15:53.832755+0000","last_fullsized":"2024-02-13T14:15:53.832755+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:17.185096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:17.185096+0000","last_clean_scrub_stamp":"2024-02-13T14:15:17.185096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T23:56:12.985471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"63'2","reported_seq":45,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.728360+0000","last_change":"2024-02-13T14:15:12.179215+0000","last_active":"2024-02-13T14:15:53.728360+0000","last_peered":"2024-02-13T14:15:53.728360+0000","last_clean":"2024-02-13T14:15:53.728360+0000","last_became_active":"2024-02-13T14:15:12.178854+0000","last_became_peered":"2024-02-13T14:15:12.178854+0000","last_unstale":"2024-02-13T14:15:53.728360+0000","last_undegraded":"2024-02-13T14:15:53.728360+0000","last_fullsized":"2024-02-13T14:15:53.728360+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:11.150966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:11.150966+0000","last_clean_scrub_stamp":"2024-02-13T14:15:11.150966+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T17:06:18.808053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"70'15","reported_seq":54,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.832370+0000","last_change":"2024-02-13T14:15:14.334377+0000","last_active":"2024-02-13T14:15:53.832370+0000","last_peered":"2024-02-13T14:15:53.832370+0000","last_clean":"2024-02-13T14:15:53.832370+0000","last_became_active":"2024-02-13T14:15:14.334034+0000","last_became_peered":"2024-02-13T14:15:14.334034+0000","last_unstale":"2024-02-13T14:15:53.832370+0000","last_undegraded":"2024-02-13T14:15:53.832370+0000","last_fullsized":"2024-02-13T14:15:53.832370+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:13.158947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:13.158947+0000","last_clean_scrub_stamp":"2024-02-13T14:15:13.158947+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T23:45:24.518339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"70'8","reported_seq":50,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:16:27.468589+0000","last_change":"2024-02-13T14:15:16.202296+0000","last_active":"2024-02-13T14:16:27.468589+0000","last_peered":"2024-02-13T14:16:27.468589+0000","last_clean":"2024-02-13T14:16:27.468589+0000","last_became_active":"2024-02-13T14:15:16.201999+0000","last_became_peered":"2024-02-13T14:15:16.201999+0000","last_unstale":"2024-02-13T14:16:27.468589+0000","last_undegraded":"2024-02-13T14:16:27.468589+0000","last_fullsized":"2024-02-13T14:16:27.468589+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:15.174019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:15.174019+0000","last_clean_scrub_stamp":"2024-02-13T14:15:15.174019+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T01:20:12.563989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.727526+0000","last_change":"2024-02-13T14:15:18.212654+0000","last_active":"2024-02-13T14:15:53.727526+0000","last_peered":"2024-02-13T14:15:53.727526+0000","last_clean":"2024-02-13T14:15:53.727526+0000","last_became_active":"2024-02-13T14:15:18.212174+0000","last_became_peered":"2024-02-13T14:15:18.212174+0000","last_unstale":"2024-02-13T14:15:53.727526+0000","last_undegraded":"2024-02-13T14:15:53.727526+0000","last_fullsized":"2024-02-13T14:15:53.727526+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:17.185096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:17.185096+0000","last_clean_scrub_stamp":"2024-02-13T14:15:17.185096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T00:26:19.949680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"63'1","reported_seq":39,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.728179+0000","last_change":"2024-02-13T14:15:12.177686+0000","last_active":"2024-02-13T14:15:53.728179+0000","last_peered":"2024-02-13T14:15:53.728179+0000","last_clean":"2024-02-13T14:15:53.728179+0000","last_became_active":"2024-02-13T14:15:12.177010+0000","last_became_peered":"2024-02-13T14:15:12.177010+0000","last_unstale":"2024-02-13T14:15:53.728179+0000","last_undegraded":"2024-02-13T14:15:53.728179+0000","last_fullsized":"2024-02-13T14:15:53.728179+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:11.150966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:11.150966+0000","last_clean_scrub_stamp":"2024-02-13T14:15:11.150966+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T18:55:17.145494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"70'12","reported_seq":52,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:54.276057+0000","last_change":"2024-02-13T14:15:14.175855+0000","last_active":"2024-02-13T14:15:54.276057+0000","last_peered":"2024-02-13T14:15:54.276057+0000","last_clean":"2024-02-13T14:15:54.276057+0000","last_became_active":"2024-02-13T14:15:14.175509+0000","last_became_peered":"2024-02-13T14:15:14.175509+0000","last_unstale":"2024-02-13T14:15:54.276057+0000","last_undegraded":"2024-02-13T14:15:54.276057+0000","last_fullsized":"2024-02-13T14:15:54.276057+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:13.158947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:13.158947+0000","last_clean_scrub_stamp":"2024-02-13T14:15:13.158947+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T21:14:45.163180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.837851+0000","last_change":"2024-02-13T14:15:16.202484+0000","last_active":"2024-02-13T14:15:53.837851+0000","last_peered":"2024-02-13T14:15:53.837851+0000","last_clean":"2024-02-13T14:15:53.837851+0000","last_became_active":"2024-02-13T14:15:16.202162+0000","last_became_peered":"2024-02-13T14:15:16.202162+0000","last_unstale":"2024-02-13T14:15:53.837851+0000","last_undegraded":"2024-02-13T14:15:53.837851+0000","last_fullsized":"2024-02-13T14:15:53.837851+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:15.174019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:15.174019+0000","last_clean_scrub_stamp":"2024-02-13T14:15:15.174019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T19:38:04.299989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.727028+0000","last_change":"2024-02-13T14:15:18.213094+0000","last_active":"2024-02-13T14:15:53.727028+0000","last_peered":"2024-02-13T14:15:53.727028+0000","last_clean":"2024-02-13T14:15:53.727028+0000","last_became_active":"2024-02-13T14:15:18.212735+0000","last_became_peered":"2024-02-13T14:15:18.212735+0000","last_unstale":"2024-02-13T14:15:53.727028+0000","last_undegraded":"2024-02-13T14:15:53.727028+0000","last_fullsized":"2024-02-13T14:15:53.727028+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:17.185096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:17.185096+0000","last_clean_scrub_stamp":"2024-02-13T14:15:17.185096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T16:13:34.092833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.838067+0000","last_change":"2024-02-13T14:15:16.203161+0000","last_active":"2024-02-13T14:15:53.838067+0000","last_peered":"2024-02-13T14:15:53.838067+0000","last_clean":"2024-02-13T14:15:53.838067+0000","last_became_active":"2024-02-13T14:15:16.202838+0000","last_became_peered":"2024-02-13T14:15:16.202838+0000","last_unstale":"2024-02-13T14:15:53.838067+0000","last_undegraded":"2024-02-13T14:15:53.838067+0000","last_fullsized":"2024-02-13T14:15:53.838067+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:15.174019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:15.174019+0000","last_clean_scrub_stamp":"2024-02-13T14:15:15.174019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T19:46:00.376646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.727791+0000","last_change":"2024-02-13T14:15:12.175398+0000","last_active":"2024-02-13T14:15:53.727791+0000","last_peered":"2024-02-13T14:15:53.727791+0000","last_clean":"2024-02-13T14:15:53.727791+0000","last_became_active":"2024-02-13T14:15:12.175010+0000","last_became_peered":"2024-02-13T14:15:12.175010+0000","last_unstale":"2024-02-13T14:15:53.727791+0000","last_undegraded":"2024-02-13T14:15:53.727791+0000","last_fullsized":"2024-02-13T14:15:53.727791+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:11.150966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:11.150966+0000","last_clean_scrub_stamp":"2024-02-13T14:15:11.150966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T17:53:35.123529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"70'19","reported_seq":60,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.729697+0000","last_change":"2024-02-13T14:15:14.332776+0000","last_active":"2024-02-13T14:15:53.729697+0000","last_peered":"2024-02-13T14:15:53.729697+0000","last_clean":"2024-02-13T14:15:53.729697+0000","last_became_active":"2024-02-13T14:15:14.332410+0000","last_became_peered":"2024-02-13T14:15:14.332410+0000","last_unstale":"2024-02-13T14:15:53.729697+0000","last_undegraded":"2024-02-13T14:15:53.729697+0000","last_fullsized":"2024-02-13T14:15:53.729697+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:13.158947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:13.158947+0000","last_clean_scrub_stamp":"2024-02-13T14:15:13.158947+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T19:44:38.097073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:54.771992+0000","last_change":"2024-02-13T14:15:18.219177+0000","last_active":"2024-02-13T14:15:54.771992+0000","last_peered":"2024-02-13T14:15:54.771992+0000","last_clean":"2024-02-13T14:15:54.771992+0000","last_became_active":"2024-02-13T14:15:18.218789+0000","last_became_peered":"2024-02-13T14:15:18.218789+0000","last_unstale":"2024-02-13T14:15:54.771992+0000","last_undegraded":"2024-02-13T14:15:54.771992+0000","last_fullsized":"2024-02-13T14:15:54.771992+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:17.185096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:17.185096+0000","last_clean_scrub_stamp":"2024-02-13T14:15:17.185096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T00:40:33.532538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.838226+0000","last_change":"2024-02-13T14:15:16.199129+0000","last_active":"2024-02-13T14:15:53.838226+0000","last_peered":"2024-02-13T14:15:53.838226+0000","last_clean":"2024-02-13T14:15:53.838226+0000","last_became_active":"2024-02-13T14:15:16.198816+0000","last_became_peered":"2024-02-13T14:15:16.198816+0000","last_unstale":"2024-02-13T14:15:53.838226+0000","last_undegraded":"2024-02-13T14:15:53.838226+0000","last_fullsized":"2024-02-13T14:15:53.838226+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:15.174019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:15.174019+0000","last_clean_scrub_stamp":"2024-02-13T14:15:15.174019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T20:54:40.657639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.832762+0000","last_change":"2024-02-13T14:15:12.174353+0000","last_active":"2024-02-13T14:15:53.832762+0000","last_peered":"2024-02-13T14:15:53.832762+0000","last_clean":"2024-02-13T14:15:53.832762+0000","last_became_active":"2024-02-13T14:15:12.174074+0000","last_became_peered":"2024-02-13T14:15:12.174074+0000","last_unstale":"2024-02-13T14:15:53.832762+0000","last_undegraded":"2024-02-13T14:15:53.832762+0000","last_fullsized":"2024-02-13T14:15:53.832762+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:11.150966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:11.150966+0000","last_clean_scrub_stamp":"2024-02-13T14:15:11.150966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T01:06:47.861915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"70'9","reported_seq":45,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:54.772682+0000","last_change":"2024-02-13T14:15:14.178044+0000","last_active":"2024-02-13T14:15:54.772682+0000","last_peered":"2024-02-13T14:15:54.772682+0000","last_clean":"2024-02-13T14:15:54.772682+0000","last_became_active":"2024-02-13T14:15:14.177712+0000","last_became_peered":"2024-02-13T14:15:14.177712+0000","last_unstale":"2024-02-13T14:15:54.772682+0000","last_undegraded":"2024-02-13T14:15:54.772682+0000","last_fullsized":"2024-02-13T14:15:54.772682+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:13.158947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:13.158947+0000","last_clean_scrub_stamp":"2024-02-13T14:15:13.158947+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T20:31:59.447363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.728149+0000","last_change":"2024-02-13T14:15:18.209719+0000","last_active":"2024-02-13T14:15:53.728149+0000","last_peered":"2024-02-13T14:15:53.728149+0000","last_clean":"2024-02-13T14:15:53.728149+0000","last_became_active":"2024-02-13T14:15:18.209420+0000","last_became_peered":"2024-02-13T14:15:18.209420+0000","last_unstale":"2024-02-13T14:15:53.728149+0000","last_undegraded":"2024-02-13T14:15:53.728149+0000","last_fullsized":"2024-02-13T14:15:53.728149+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:17.185096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:17.185096+0000","last_clean_scrub_stamp":"2024-02-13T14:15:17.185096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T20:39:09.520608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"66'2","reported_seq":44,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.832515+0000","last_change":"2024-02-13T14:15:15.191712+0000","last_active":"2024-02-13T14:15:53.832515+0000","last_peered":"2024-02-13T14:15:53.832515+0000","last_clean":"2024-02-13T14:15:53.832515+0000","last_became_active":"2024-02-13T14:15:11.164778+0000","last_became_peered":"2024-02-13T14:15:11.164778+0000","last_unstale":"2024-02-13T14:15:53.832515+0000","last_undegraded":"2024-02-13T14:15:53.832515+0000","last_fullsized":"2024-02-13T14:15:53.832515+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:10.145476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:10.145476+0000","last_clean_scrub_stamp":"2024-02-13T14:15:10.145476+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T23:29:11.986752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014959350000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:54.276701+0000","last_change":"2024-02-13T14:15:12.174683+0000","last_active":"2024-02-13T14:15:54.276701+0000","last_peered":"2024-02-13T14:15:54.276701+0000","last_clean":"2024-02-13T14:15:54.276701+0000","last_became_active":"2024-02-13T14:15:12.174393+0000","last_became_peered":"2024-02-13T14:15:12.174393+0000","last_unstale":"2024-02-13T14:15:54.276701+0000","last_undegraded":"2024-02-13T14:15:54.276701+0000","last_fullsized":"2024-02-13T14:15:54.276701+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:11.150966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:11.150966+0000","last_clean_scrub_stamp":"2024-02-13T14:15:11.150966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T15:55:43.825228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"70'28","reported_seq":94,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:16:27.468460+0000","last_change":"2024-02-13T14:15:14.182362+0000","last_active":"2024-02-13T14:16:27.468460+0000","last_peered":"2024-02-13T14:16:27.468460+0000","last_clean":"2024-02-13T14:16:27.468460+0000","last_became_active":"2024-02-13T14:15:14.181840+0000","last_became_peered":"2024-02-13T14:15:14.181840+0000","last_unstale":"2024-02-13T14:16:27.468460+0000","last_undegraded":"2024-02-13T14:16:27.468460+0000","last_fullsized":"2024-02-13T14:16:27.468460+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:13.158947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:13.158947+0000","last_clean_scrub_stamp":"2024-02-13T14:15:13.158947+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T02:09:12.661626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:54.772481+0000","last_change":"2024-02-13T14:15:16.199472+0000","last_active":"2024-02-13T14:15:54.772481+0000","last_peered":"2024-02-13T14:15:54.772481+0000","last_clean":"2024-02-13T14:15:54.772481+0000","last_became_active":"2024-02-13T14:15:16.199115+0000","last_became_peered":"2024-02-13T14:15:16.199115+0000","last_unstale":"2024-02-13T14:15:54.772481+0000","last_undegraded":"2024-02-13T14:15:54.772481+0000","last_fullsized":"2024-02-13T14:15:54.772481+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:15.174019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:15.174019+0000","last_clean_scrub_stamp":"2024-02-13T14:15:15.174019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T23:21:36.799685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.728042+0000","last_change":"2024-02-13T14:15:18.873642+0000","last_active":"2024-02-13T14:15:53.728042+0000","last_peered":"2024-02-13T14:15:53.728042+0000","last_clean":"2024-02-13T14:15:53.728042+0000","last_became_active":"2024-02-13T14:15:18.873286+0000","last_became_peered":"2024-02-13T14:15:18.873286+0000","last_unstale":"2024-02-13T14:15:53.728042+0000","last_undegraded":"2024-02-13T14:15:53.728042+0000","last_fullsized":"2024-02-13T14:15:53.728042+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:17.185096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:17.185096+0000","last_clean_scrub_stamp":"2024-02-13T14:15:17.185096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T16:23:57.985044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:54.381847+0000","last_change":"2024-02-13T14:15:12.417576+0000","last_active":"2024-02-13T14:15:54.381847+0000","last_peered":"2024-02-13T14:15:54.381847+0000","last_clean":"2024-02-13T14:15:54.381847+0000","last_became_active":"2024-02-13T14:15:12.417254+0000","last_became_peered":"2024-02-13T14:15:12.417254+0000","last_unstale":"2024-02-13T14:15:54.381847+0000","last_undegraded":"2024-02-13T14:15:54.381847+0000","last_fullsized":"2024-02-13T14:15:54.381847+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:11.150966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:11.150966+0000","last_clean_scrub_stamp":"2024-02-13T14:15:11.150966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T18:36:24.228485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":42,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.837952+0000","last_change":"2024-02-13T14:15:15.190228+0000","last_active":"2024-02-13T14:15:53.837952+0000","last_peered":"2024-02-13T14:15:53.837952+0000","last_clean":"2024-02-13T14:15:53.837952+0000","last_became_active":"2024-02-13T14:15:11.165902+0000","last_became_peered":"2024-02-13T14:15:11.165902+0000","last_unstale":"2024-02-13T14:15:53.837952+0000","last_undegraded":"2024-02-13T14:15:53.837952+0000","last_fullsized":"2024-02-13T14:15:53.837952+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:10.145476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:10.145476+0000","last_clean_scrub_stamp":"2024-02-13T14:15:10.145476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T14:23:13.696445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001038976,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"70'13","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.838476+0000","last_change":"2024-02-13T14:15:30.020442+0000","last_active":"2024-02-13T14:15:53.838476+0000","last_peered":"2024-02-13T14:15:53.838476+0000","last_clean":"2024-02-13T14:15:53.838476+0000","last_became_active":"2024-02-13T14:15:30.013757+0000","last_became_peered":"2024-02-13T14:15:30.013757+0000","last_unstale":"2024-02-13T14:15:53.838476+0000","last_undegraded":"2024-02-13T14:15:53.838476+0000","last_fullsized":"2024-02-13T14:15:53.838476+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:13.158947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:13.158947+0000","last_clean_scrub_stamp":"2024-02-13T14:15:13.158947+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T23:04:07.873775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":18,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.838303+0000","last_change":"2024-02-13T14:15:16.199037+0000","last_active":"2024-02-13T14:15:53.838303+0000","last_peered":"2024-02-13T14:15:53.838303+0000","last_clean":"2024-02-13T14:15:53.838303+0000","last_became_active":"2024-02-13T14:15:16.198699+0000","last_became_peered":"2024-02-13T14:15:16.198699+0000","last_unstale":"2024-02-13T14:15:53.838303+0000","last_undegraded":"2024-02-13T14:15:53.838303+0000","last_fullsized":"2024-02-13T14:15:53.838303+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:15.174019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:15.174019+0000","last_clean_scrub_stamp":"2024-02-13T14:15:15.174019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T18:09:45.520620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.832943+0000","last_change":"2024-02-13T14:15:18.211319+0000","last_active":"2024-02-13T14:15:53.832943+0000","last_peered":"2024-02-13T14:15:53.832943+0000","last_clean":"2024-02-13T14:15:53.832943+0000","last_became_active":"2024-02-13T14:15:18.211032+0000","last_became_peered":"2024-02-13T14:15:18.211032+0000","last_unstale":"2024-02-13T14:15:53.832943+0000","last_undegraded":"2024-02-13T14:15:53.832943+0000","last_fullsized":"2024-02-13T14:15:53.832943+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:17.185096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:17.185096+0000","last_clean_scrub_stamp":"2024-02-13T14:15:17.185096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T00:28:22.214175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"74'128","reported_seq":279,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:55.825590+0000","last_change":"2024-02-13T14:15:01.785257+0000","last_active":"2024-02-13T14:15:55.825590+0000","last_peered":"2024-02-13T14:15:55.825590+0000","last_clean":"2024-02-13T14:15:55.825590+0000","last_became_active":"2024-02-13T14:15:01.784776+0000","last_became_peered":"2024-02-13T14:15:01.784776+0000","last_unstale":"2024-02-13T14:15:55.825590+0000","last_undegraded":"2024-02-13T14:15:55.825590+0000","last_fullsized":"2024-02-13T14:15:55.825590+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:12:53.266772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:12:53.266772+0000","last_clean_scrub_stamp":"2024-02-13T14:12:53.266772+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T22:39:57.646723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.727273+0000","last_change":"2024-02-13T14:15:12.177331+0000","last_active":"2024-02-13T14:15:53.727273+0000","last_peered":"2024-02-13T14:15:53.727273+0000","last_clean":"2024-02-13T14:15:53.727273+0000","last_became_active":"2024-02-13T14:15:12.177054+0000","last_became_peered":"2024-02-13T14:15:12.177054+0000","last_unstale":"2024-02-13T14:15:53.727273+0000","last_undegraded":"2024-02-13T14:15:53.727273+0000","last_fullsized":"2024-02-13T14:15:53.727273+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:11.150966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:11.150966+0000","last_clean_scrub_stamp":"2024-02-13T14:15:11.150966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T01:10:11.610740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"70'16","reported_seq":68,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:16:27.468571+0000","last_change":"2024-02-13T14:15:14.332926+0000","last_active":"2024-02-13T14:16:27.468571+0000","last_peered":"2024-02-13T14:16:27.468571+0000","last_clean":"2024-02-13T14:16:27.468571+0000","last_became_active":"2024-02-13T14:15:14.332516+0000","last_became_peered":"2024-02-13T14:15:14.332516+0000","last_unstale":"2024-02-13T14:16:27.468571+0000","last_undegraded":"2024-02-13T14:16:27.468571+0000","last_fullsized":"2024-02-13T14:16:27.468571+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:13.158947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:13.158947+0000","last_clean_scrub_stamp":"2024-02-13T14:15:13.158947+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T01:10:23.819027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.727688+0000","last_change":"2024-02-13T14:15:16.203769+0000","last_active":"2024-02-13T14:15:53.727688+0000","last_peered":"2024-02-13T14:15:53.727688+0000","last_clean":"2024-02-13T14:15:53.727688+0000","last_became_active":"2024-02-13T14:15:16.203491+0000","last_became_peered":"2024-02-13T14:15:16.203491+0000","last_unstale":"2024-02-13T14:15:53.727688+0000","last_undegraded":"2024-02-13T14:15:53.727688+0000","last_fullsized":"2024-02-13T14:15:53.727688+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:15.174019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:15.174019+0000","last_clean_scrub_stamp":"2024-02-13T14:15:15.174019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T19:21:03.840446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:54.382087+0000","last_change":"2024-02-13T14:15:18.213244+0000","last_active":"2024-02-13T14:15:54.382087+0000","last_peered":"2024-02-13T14:15:54.382087+0000","last_clean":"2024-02-13T14:15:54.382087+0000","last_became_active":"2024-02-13T14:15:18.212865+0000","last_became_peered":"2024-02-13T14:15:18.212865+0000","last_unstale":"2024-02-13T14:15:54.382087+0000","last_undegraded":"2024-02-13T14:15:54.382087+0000","last_fullsized":"2024-02-13T14:15:54.382087+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:17.185096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:17.185096+0000","last_clean_scrub_stamp":"2024-02-13T14:15:17.185096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T21:34:30.826293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:54.772111+0000","last_change":"2024-02-13T14:15:12.172581+0000","last_active":"2024-02-13T14:15:54.772111+0000","last_peered":"2024-02-13T14:15:54.772111+0000","last_clean":"2024-02-13T14:15:54.772111+0000","last_became_active":"2024-02-13T14:15:12.172281+0000","last_became_peered":"2024-02-13T14:15:12.172281+0000","last_unstale":"2024-02-13T14:15:54.772111+0000","last_undegraded":"2024-02-13T14:15:54.772111+0000","last_fullsized":"2024-02-13T14:15:54.772111+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:11.150966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:11.150966+0000","last_clean_scrub_stamp":"2024-02-13T14:15:11.150966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T15:39:01.051588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"71'5","reported_seq":122,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:16:33.723588+0000","last_change":"2024-02-13T14:15:15.189543+0000","last_active":"2024-02-13T14:16:33.723588+0000","last_peered":"2024-02-13T14:16:33.723588+0000","last_clean":"2024-02-13T14:16:33.723588+0000","last_became_active":"2024-02-13T14:15:11.165645+0000","last_became_peered":"2024-02-13T14:15:11.165645+0000","last_unstale":"2024-02-13T14:16:33.723588+0000","last_undegraded":"2024-02-13T14:16:33.723588+0000","last_fullsized":"2024-02-13T14:16:33.723588+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:10.145476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:10.145476+0000","last_clean_scrub_stamp":"2024-02-13T14:15:10.145476+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T20:35:16.455198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015406700000000001,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":77,"num_read_kb":72,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"70'12","reported_seq":47,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:54.772378+0000","last_change":"2024-02-13T14:15:14.179313+0000","last_active":"2024-02-13T14:15:54.772378+0000","last_peered":"2024-02-13T14:15:54.772378+0000","last_clean":"2024-02-13T14:15:54.772378+0000","last_became_active":"2024-02-13T14:15:14.178947+0000","last_became_peered":"2024-02-13T14:15:14.178947+0000","last_unstale":"2024-02-13T14:15:54.772378+0000","last_undegraded":"2024-02-13T14:15:54.772378+0000","last_fullsized":"2024-02-13T14:15:54.772378+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:13.158947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:13.158947+0000","last_clean_scrub_stamp":"2024-02-13T14:15:13.158947+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T01:24:26.394723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.833173+0000","last_change":"2024-02-13T14:15:16.200320+0000","last_active":"2024-02-13T14:15:53.833173+0000","last_peered":"2024-02-13T14:15:53.833173+0000","last_clean":"2024-02-13T14:15:53.833173+0000","last_became_active":"2024-02-13T14:15:16.200037+0000","last_became_peered":"2024-02-13T14:15:16.200037+0000","last_unstale":"2024-02-13T14:15:53.833173+0000","last_undegraded":"2024-02-13T14:15:53.833173+0000","last_fullsized":"2024-02-13T14:15:53.833173+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:15.174019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:15.174019+0000","last_clean_scrub_stamp":"2024-02-13T14:15:15.174019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T02:09:47.494344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:54.382097+0000","last_change":"2024-02-13T14:15:18.873555+0000","last_active":"2024-02-13T14:15:54.382097+0000","last_peered":"2024-02-13T14:15:54.382097+0000","last_clean":"2024-02-13T14:15:54.382097+0000","last_became_active":"2024-02-13T14:15:18.873196+0000","last_became_peered":"2024-02-13T14:15:18.873196+0000","last_unstale":"2024-02-13T14:15:54.382097+0000","last_undegraded":"2024-02-13T14:15:54.382097+0000","last_fullsized":"2024-02-13T14:15:54.382097+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:17.185096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:17.185096+0000","last_clean_scrub_stamp":"2024-02-13T14:15:17.185096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T21:21:53.402901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:54.382287+0000","last_change":"2024-02-13T14:15:12.416907+0000","last_active":"2024-02-13T14:15:54.382287+0000","last_peered":"2024-02-13T14:15:54.382287+0000","last_clean":"2024-02-13T14:15:54.382287+0000","last_became_active":"2024-02-13T14:15:12.416562+0000","last_became_peered":"2024-02-13T14:15:12.416562+0000","last_unstale":"2024-02-13T14:15:54.382287+0000","last_undegraded":"2024-02-13T14:15:54.382287+0000","last_fullsized":"2024-02-13T14:15:54.382287+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:11.150966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:11.150966+0000","last_clean_scrub_stamp":"2024-02-13T14:15:11.150966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T19:38:18.643483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"70'19","reported_seq":65,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:54.771840+0000","last_change":"2024-02-13T14:15:14.332634+0000","last_active":"2024-02-13T14:15:54.771840+0000","last_peered":"2024-02-13T14:15:54.771840+0000","last_clean":"2024-02-13T14:15:54.771840+0000","last_became_active":"2024-02-13T14:15:14.332188+0000","last_became_peered":"2024-02-13T14:15:14.332188+0000","last_unstale":"2024-02-13T14:15:54.771840+0000","last_undegraded":"2024-02-13T14:15:54.771840+0000","last_fullsized":"2024-02-13T14:15:54.771840+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:13.158947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:13.158947+0000","last_clean_scrub_stamp":"2024-02-13T14:15:13.158947+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T17:01:48.908838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.729869+0000","last_change":"2024-02-13T14:15:16.201964+0000","last_active":"2024-02-13T14:15:53.729869+0000","last_peered":"2024-02-13T14:15:53.729869+0000","last_clean":"2024-02-13T14:15:53.729869+0000","last_became_active":"2024-02-13T14:15:16.201600+0000","last_became_peered":"2024-02-13T14:15:16.201600+0000","last_unstale":"2024-02-13T14:15:53.729869+0000","last_undegraded":"2024-02-13T14:15:53.729869+0000","last_fullsized":"2024-02-13T14:15:53.729869+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:15.174019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:15.174019+0000","last_clean_scrub_stamp":"2024-02-13T14:15:15.174019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T15:17:01.382222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:54.772021+0000","last_change":"2024-02-13T14:15:18.220066+0000","last_active":"2024-02-13T14:15:54.772021+0000","last_peered":"2024-02-13T14:15:54.772021+0000","last_clean":"2024-02-13T14:15:54.772021+0000","last_became_active":"2024-02-13T14:15:18.219655+0000","last_became_peered":"2024-02-13T14:15:18.219655+0000","last_unstale":"2024-02-13T14:15:54.772021+0000","last_undegraded":"2024-02-13T14:15:54.772021+0000","last_fullsized":"2024-02-13T14:15:54.772021+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:17.185096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:17.185096+0000","last_clean_scrub_stamp":"2024-02-13T14:15:17.185096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T23:56:38.790923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.833028+0000","last_change":"2024-02-13T14:15:12.416955+0000","last_active":"2024-02-13T14:15:53.833028+0000","last_peered":"2024-02-13T14:15:53.833028+0000","last_clean":"2024-02-13T14:15:53.833028+0000","last_became_active":"2024-02-13T14:15:12.416595+0000","last_became_peered":"2024-02-13T14:15:12.416595+0000","last_unstale":"2024-02-13T14:15:53.833028+0000","last_undegraded":"2024-02-13T14:15:53.833028+0000","last_fullsized":"2024-02-13T14:15:53.833028+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:11.150966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:11.150966+0000","last_clean_scrub_stamp":"2024-02-13T14:15:11.150966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T23:33:58.171584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"70'10","reported_seq":44,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:54.381261+0000","last_change":"2024-02-13T14:15:14.182775+0000","last_active":"2024-02-13T14:15:54.381261+0000","last_peered":"2024-02-13T14:15:54.381261+0000","last_clean":"2024-02-13T14:15:54.381261+0000","last_became_active":"2024-02-13T14:15:14.182403+0000","last_became_peered":"2024-02-13T14:15:14.182403+0000","last_unstale":"2024-02-13T14:15:54.381261+0000","last_undegraded":"2024-02-13T14:15:54.381261+0000","last_fullsized":"2024-02-13T14:15:54.381261+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:13.158947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:13.158947+0000","last_clean_scrub_stamp":"2024-02-13T14:15:13.158947+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T19:35:01.071596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"70'8","reported_seq":50,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:16:27.468459+0000","last_change":"2024-02-13T14:15:16.202939+0000","last_active":"2024-02-13T14:16:27.468459+0000","last_peered":"2024-02-13T14:16:27.468459+0000","last_clean":"2024-02-13T14:16:27.468459+0000","last_became_active":"2024-02-13T14:15:16.202633+0000","last_became_peered":"2024-02-13T14:15:16.202633+0000","last_unstale":"2024-02-13T14:16:27.468459+0000","last_undegraded":"2024-02-13T14:16:27.468459+0000","last_fullsized":"2024-02-13T14:16:27.468459+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:15.174019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:15.174019+0000","last_clean_scrub_stamp":"2024-02-13T14:15:15.174019+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T18:21:29.990547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.727264+0000","last_change":"2024-02-13T14:15:18.876763+0000","last_active":"2024-02-13T14:15:53.727264+0000","last_peered":"2024-02-13T14:15:53.727264+0000","last_clean":"2024-02-13T14:15:53.727264+0000","last_became_active":"2024-02-13T14:15:18.876450+0000","last_became_peered":"2024-02-13T14:15:18.876450+0000","last_unstale":"2024-02-13T14:15:53.727264+0000","last_undegraded":"2024-02-13T14:15:53.727264+0000","last_fullsized":"2024-02-13T14:15:53.727264+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:17.185096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:17.185096+0000","last_clean_scrub_stamp":"2024-02-13T14:15:17.185096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T19:18:44.250701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.6","version":"63'1","reported_seq":34,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:54.772577+0000","last_change":"2024-02-13T14:15:12.174096+0000","last_active":"2024-02-13T14:15:54.772577+0000","last_peered":"2024-02-13T14:15:54.772577+0000","last_clean":"2024-02-13T14:15:54.772577+0000","last_became_active":"2024-02-13T14:15:12.173051+0000","last_became_peered":"2024-02-13T14:15:12.173051+0000","last_unstale":"2024-02-13T14:15:54.772577+0000","last_undegraded":"2024-02-13T14:15:54.772577+0000","last_fullsized":"2024-02-13T14:15:54.772577+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:11.150966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:11.150966+0000","last_clean_scrub_stamp":"2024-02-13T14:15:11.150966+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T14:34:17.943660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"70'14","reported_seq":50,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:54.276066+0000","last_change":"2024-02-13T14:15:14.172193+0000","last_active":"2024-02-13T14:15:54.276066+0000","last_peered":"2024-02-13T14:15:54.276066+0000","last_clean":"2024-02-13T14:15:54.276066+0000","last_became_active":"2024-02-13T14:15:14.171816+0000","last_became_peered":"2024-02-13T14:15:14.171816+0000","last_unstale":"2024-02-13T14:15:54.276066+0000","last_undegraded":"2024-02-13T14:15:54.276066+0000","last_fullsized":"2024-02-13T14:15:54.276066+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:13.158947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:13.158947+0000","last_clean_scrub_stamp":"2024-02-13T14:15:13.158947+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T00:22:12.121393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"70'8","reported_seq":50,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:16:27.468460+0000","last_change":"2024-02-13T14:15:16.195776+0000","last_active":"2024-02-13T14:16:27.468460+0000","last_peered":"2024-02-13T14:16:27.468460+0000","last_clean":"2024-02-13T14:16:27.468460+0000","last_became_active":"2024-02-13T14:15:16.195481+0000","last_became_peered":"2024-02-13T14:15:16.195481+0000","last_unstale":"2024-02-13T14:16:27.468460+0000","last_undegraded":"2024-02-13T14:16:27.468460+0000","last_fullsized":"2024-02-13T14:16:27.468460+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:15.174019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:15.174019+0000","last_clean_scrub_stamp":"2024-02-13T14:15:15.174019+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T20:08:34.562080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:54.276643+0000","last_change":"2024-02-13T14:15:18.217626+0000","last_active":"2024-02-13T14:15:54.276643+0000","last_peered":"2024-02-13T14:15:54.276643+0000","last_clean":"2024-02-13T14:15:54.276643+0000","last_became_active":"2024-02-13T14:15:18.217389+0000","last_became_peered":"2024-02-13T14:15:18.217389+0000","last_unstale":"2024-02-13T14:15:54.276643+0000","last_undegraded":"2024-02-13T14:15:54.276643+0000","last_fullsized":"2024-02-13T14:15:54.276643+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:17.185096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:17.185096+0000","last_clean_scrub_stamp":"2024-02-13T14:15:17.185096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T01:17:15.759600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.728455+0000","last_change":"2024-02-13T14:15:12.171708+0000","last_active":"2024-02-13T14:15:53.728455+0000","last_peered":"2024-02-13T14:15:53.728455+0000","last_clean":"2024-02-13T14:15:53.728455+0000","last_became_active":"2024-02-13T14:15:12.171371+0000","last_became_peered":"2024-02-13T14:15:12.171371+0000","last_unstale":"2024-02-13T14:15:53.728455+0000","last_undegraded":"2024-02-13T14:15:53.728455+0000","last_fullsized":"2024-02-13T14:15:53.728455+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:11.150966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:11.150966+0000","last_clean_scrub_stamp":"2024-02-13T14:15:11.150966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T20:40:53.640720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"70'18","reported_seq":61,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.726688+0000","last_change":"2024-02-13T14:15:14.333793+0000","last_active":"2024-02-13T14:15:53.726688+0000","last_peered":"2024-02-13T14:15:53.726688+0000","last_clean":"2024-02-13T14:15:53.726688+0000","last_became_active":"2024-02-13T14:15:14.333478+0000","last_became_peered":"2024-02-13T14:15:14.333478+0000","last_unstale":"2024-02-13T14:15:53.726688+0000","last_undegraded":"2024-02-13T14:15:53.726688+0000","last_fullsized":"2024-02-13T14:15:53.726688+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:13.158947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:13.158947+0000","last_clean_scrub_stamp":"2024-02-13T14:15:13.158947+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T20:44:05.328601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:54.276494+0000","last_change":"2024-02-13T14:15:16.198595+0000","last_active":"2024-02-13T14:15:54.276494+0000","last_peered":"2024-02-13T14:15:54.276494+0000","last_clean":"2024-02-13T14:15:54.276494+0000","last_became_active":"2024-02-13T14:15:16.198291+0000","last_became_peered":"2024-02-13T14:15:16.198291+0000","last_unstale":"2024-02-13T14:15:54.276494+0000","last_undegraded":"2024-02-13T14:15:54.276494+0000","last_fullsized":"2024-02-13T14:15:54.276494+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:15.174019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:15.174019+0000","last_clean_scrub_stamp":"2024-02-13T14:15:15.174019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T23:53:39.694426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.832756+0000","last_change":"2024-02-13T14:15:18.213650+0000","last_active":"2024-02-13T14:15:53.832756+0000","last_peered":"2024-02-13T14:15:53.832756+0000","last_clean":"2024-02-13T14:15:53.832756+0000","last_became_active":"2024-02-13T14:15:18.213375+0000","last_became_peered":"2024-02-13T14:15:18.213375+0000","last_unstale":"2024-02-13T14:15:53.832756+0000","last_undegraded":"2024-02-13T14:15:53.832756+0000","last_fullsized":"2024-02-13T14:15:53.832756+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:17.185096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:17.185096+0000","last_clean_scrub_stamp":"2024-02-13T14:15:17.185096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T17:03:41.344035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"70'8","reported_seq":50,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:16:27.468567+0000","last_change":"2024-02-13T14:15:16.196368+0000","last_active":"2024-02-13T14:16:27.468567+0000","last_peered":"2024-02-13T14:16:27.468567+0000","last_clean":"2024-02-13T14:16:27.468567+0000","last_became_active":"2024-02-13T14:15:16.196057+0000","last_became_peered":"2024-02-13T14:15:16.196057+0000","last_unstale":"2024-02-13T14:16:27.468567+0000","last_undegraded":"2024-02-13T14:16:27.468567+0000","last_fullsized":"2024-02-13T14:16:27.468567+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:15.174019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:15.174019+0000","last_clean_scrub_stamp":"2024-02-13T14:15:15.174019+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T00:14:51.809112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.726618+0000","last_change":"2024-02-13T14:15:12.168180+0000","last_active":"2024-02-13T14:15:53.726618+0000","last_peered":"2024-02-13T14:15:53.726618+0000","last_clean":"2024-02-13T14:15:53.726618+0000","last_became_active":"2024-02-13T14:15:12.167830+0000","last_became_peered":"2024-02-13T14:15:12.167830+0000","last_unstale":"2024-02-13T14:15:53.726618+0000","last_undegraded":"2024-02-13T14:15:53.726618+0000","last_fullsized":"2024-02-13T14:15:53.726618+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:11.150966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:11.150966+0000","last_clean_scrub_stamp":"2024-02-13T14:15:11.150966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T00:40:38.722769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"70'15","reported_seq":54,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:54.381211+0000","last_change":"2024-02-13T14:15:14.181534+0000","last_active":"2024-02-13T14:15:54.381211+0000","last_peered":"2024-02-13T14:15:54.381211+0000","last_clean":"2024-02-13T14:15:54.381211+0000","last_became_active":"2024-02-13T14:15:14.180668+0000","last_became_peered":"2024-02-13T14:15:14.180668+0000","last_unstale":"2024-02-13T14:15:54.381211+0000","last_undegraded":"2024-02-13T14:15:54.381211+0000","last_fullsized":"2024-02-13T14:15:54.381211+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:13.158947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:13.158947+0000","last_clean_scrub_stamp":"2024-02-13T14:15:13.158947+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T23:47:18.317400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.728082+0000","last_change":"2024-02-13T14:15:18.873633+0000","last_active":"2024-02-13T14:15:53.728082+0000","last_peered":"2024-02-13T14:15:53.728082+0000","last_clean":"2024-02-13T14:15:53.728082+0000","last_became_active":"2024-02-13T14:15:18.873283+0000","last_became_peered":"2024-02-13T14:15:18.873283+0000","last_unstale":"2024-02-13T14:15:53.728082+0000","last_undegraded":"2024-02-13T14:15:53.728082+0000","last_fullsized":"2024-02-13T14:15:53.728082+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:17.185096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:17.185096+0000","last_clean_scrub_stamp":"2024-02-13T14:15:17.185096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T18:27:27.484930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.833154+0000","last_change":"2024-02-13T14:15:16.197082+0000","last_active":"2024-02-13T14:15:53.833154+0000","last_peered":"2024-02-13T14:15:53.833154+0000","last_clean":"2024-02-13T14:15:53.833154+0000","last_became_active":"2024-02-13T14:15:16.196293+0000","last_became_peered":"2024-02-13T14:15:16.196293+0000","last_unstale":"2024-02-13T14:15:53.833154+0000","last_undegraded":"2024-02-13T14:15:53.833154+0000","last_fullsized":"2024-02-13T14:15:53.833154+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:15.174019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:15.174019+0000","last_clean_scrub_stamp":"2024-02-13T14:15:15.174019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T17:00:46.766784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:54.276539+0000","last_change":"2024-02-13T14:15:12.417704+0000","last_active":"2024-02-13T14:15:54.276539+0000","last_peered":"2024-02-13T14:15:54.276539+0000","last_clean":"2024-02-13T14:15:54.276539+0000","last_became_active":"2024-02-13T14:15:12.416567+0000","last_became_peered":"2024-02-13T14:15:12.416567+0000","last_unstale":"2024-02-13T14:15:54.276539+0000","last_undegraded":"2024-02-13T14:15:54.276539+0000","last_fullsized":"2024-02-13T14:15:54.276539+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:11.150966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:11.150966+0000","last_clean_scrub_stamp":"2024-02-13T14:15:11.150966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T01:59:35.106234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"70'11","reported_seq":48,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:54.276353+0000","last_change":"2024-02-13T14:15:14.176817+0000","last_active":"2024-02-13T14:15:54.276353+0000","last_peered":"2024-02-13T14:15:54.276353+0000","last_clean":"2024-02-13T14:15:54.276353+0000","last_became_active":"2024-02-13T14:15:14.176123+0000","last_became_peered":"2024-02-13T14:15:14.176123+0000","last_unstale":"2024-02-13T14:15:54.276353+0000","last_undegraded":"2024-02-13T14:15:54.276353+0000","last_fullsized":"2024-02-13T14:15:54.276353+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:13.158947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:13.158947+0000","last_clean_scrub_stamp":"2024-02-13T14:15:13.158947+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T00:01:54.818359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.838606+0000","last_change":"2024-02-13T14:15:18.215227+0000","last_active":"2024-02-13T14:15:53.838606+0000","last_peered":"2024-02-13T14:15:53.838606+0000","last_clean":"2024-02-13T14:15:53.838606+0000","last_became_active":"2024-02-13T14:15:18.214947+0000","last_became_peered":"2024-02-13T14:15:18.214947+0000","last_unstale":"2024-02-13T14:15:53.838606+0000","last_undegraded":"2024-02-13T14:15:53.838606+0000","last_fullsized":"2024-02-13T14:15:53.838606+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:17.185096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:17.185096+0000","last_clean_scrub_stamp":"2024-02-13T14:15:17.185096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T17:37:51.731285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:54.382328+0000","last_change":"2024-02-13T14:15:16.198099+0000","last_active":"2024-02-13T14:15:54.382328+0000","last_peered":"2024-02-13T14:15:54.382328+0000","last_clean":"2024-02-13T14:15:54.382328+0000","last_became_active":"2024-02-13T14:15:16.197809+0000","last_became_peered":"2024-02-13T14:15:16.197809+0000","last_unstale":"2024-02-13T14:15:54.382328+0000","last_undegraded":"2024-02-13T14:15:54.382328+0000","last_fullsized":"2024-02-13T14:15:54.382328+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:15.174019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:15.174019+0000","last_clean_scrub_stamp":"2024-02-13T14:15:15.174019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T17:04:20.778378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.729914+0000","last_change":"2024-02-13T14:15:12.173685+0000","last_active":"2024-02-13T14:15:53.729914+0000","last_peered":"2024-02-13T14:15:53.729914+0000","last_clean":"2024-02-13T14:15:53.729914+0000","last_became_active":"2024-02-13T14:15:12.173336+0000","last_became_peered":"2024-02-13T14:15:12.173336+0000","last_unstale":"2024-02-13T14:15:53.729914+0000","last_undegraded":"2024-02-13T14:15:53.729914+0000","last_fullsized":"2024-02-13T14:15:53.729914+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:11.150966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:11.150966+0000","last_clean_scrub_stamp":"2024-02-13T14:15:11.150966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T01:47:42.700019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"70'17","reported_seq":81,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:54.276323+0000","last_change":"2024-02-13T14:15:30.027281+0000","last_active":"2024-02-13T14:15:54.276323+0000","last_peered":"2024-02-13T14:15:54.276323+0000","last_clean":"2024-02-13T14:15:54.276323+0000","last_became_active":"2024-02-13T14:15:30.013179+0000","last_became_peered":"2024-02-13T14:15:30.013179+0000","last_unstale":"2024-02-13T14:15:54.276323+0000","last_undegraded":"2024-02-13T14:15:54.276323+0000","last_fullsized":"2024-02-13T14:15:54.276323+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:13.158947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:13.158947+0000","last_clean_scrub_stamp":"2024-02-13T14:15:13.158947+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T23:17:54.679510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"70'1","reported_seq":22,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:54.277242+0000","last_change":"2024-02-13T14:15:18.215668+0000","last_active":"2024-02-13T14:15:54.277242+0000","last_peered":"2024-02-13T14:15:54.277242+0000","last_clean":"2024-02-13T14:15:54.277242+0000","last_became_active":"2024-02-13T14:15:18.215173+0000","last_became_peered":"2024-02-13T14:15:18.215173+0000","last_unstale":"2024-02-13T14:15:54.277242+0000","last_undegraded":"2024-02-13T14:15:54.277242+0000","last_fullsized":"2024-02-13T14:15:54.277242+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:17.185096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:17.185096+0000","last_clean_scrub_stamp":"2024-02-13T14:15:17.185096+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T18:29:41.390078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"70'8","reported_seq":50,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:16:27.468459+0000","last_change":"2024-02-13T14:15:16.199835+0000","last_active":"2024-02-13T14:16:27.468459+0000","last_peered":"2024-02-13T14:16:27.468459+0000","last_clean":"2024-02-13T14:16:27.468459+0000","last_became_active":"2024-02-13T14:15:16.199319+0000","last_became_peered":"2024-02-13T14:15:16.199319+0000","last_unstale":"2024-02-13T14:16:27.468459+0000","last_undegraded":"2024-02-13T14:16:27.468459+0000","last_fullsized":"2024-02-13T14:16:27.468459+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:15.174019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:15.174019+0000","last_clean_scrub_stamp":"2024-02-13T14:15:15.174019+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T00:45:12.263270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.727694+0000","last_change":"2024-02-13T14:15:12.174951+0000","last_active":"2024-02-13T14:15:53.727694+0000","last_peered":"2024-02-13T14:15:53.727694+0000","last_clean":"2024-02-13T14:15:53.727694+0000","last_became_active":"2024-02-13T14:15:12.174462+0000","last_became_peered":"2024-02-13T14:15:12.174462+0000","last_unstale":"2024-02-13T14:15:53.727694+0000","last_undegraded":"2024-02-13T14:15:53.727694+0000","last_fullsized":"2024-02-13T14:15:53.727694+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:11.150966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:11.150966+0000","last_clean_scrub_stamp":"2024-02-13T14:15:11.150966+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T14:42:22.960402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"70'10","reported_seq":44,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-13T14:15:54.276135+0000","last_change":"2024-02-13T14:15:14.176111+0000","last_active":"2024-02-13T14:15:54.276135+0000","last_peered":"2024-02-13T14:15:54.276135+0000","last_clean":"2024-02-13T14:15:54.276135+0000","last_became_active":"2024-02-13T14:15:14.174639+0000","last_became_peered":"2024-02-13T14:15:14.174639+0000","last_unstale":"2024-02-13T14:15:54.276135+0000","last_undegraded":"2024-02-13T14:15:54.276135+0000","last_fullsized":"2024-02-13T14:15:54.276135+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:13.158947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:13.158947+0000","last_clean_scrub_stamp":"2024-02-13T14:15:13.158947+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T16:07:46.515640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":697,"num_read_kb":454,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":393,"ondisk_log_size":393,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1456,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":77,"num_read_kb":72,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":9,"acting":9,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":56,"seq":240518168597,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32572,"kb_used_data":1396,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712580,"statfs":{"total":95995035648,"available":95961681920,"internally_reserved":0,"allocated":1429504,"data_stored":937896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Feb 13 14:15:58 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.66000000000000003}]},{"osd":1,"last update":"Tue Feb 13 14:15:58 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.64300000000000002}]},{"osd":2,"last update":"Tue Feb 13 14:15:58 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"last":0.60199999999999998}]},{"osd":3,"last update":"Tue Feb 13 14:15:58 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.52600000000000002}]},{"osd":4,"last update":"Tue Feb 13 14:15:58 2024","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.435},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.629,"5min":0.629,"15min":0.629},"last":0.46200000000000002}]},{"osd":5,"last update":"Tue Feb 13 14:15:58 2024","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.626,"5min":0.626,"15min":0.626},"last":0.56999999999999995}]},{"osd":6,"last update":"Tue Feb 13 14:15:58 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.54600000000000004}]}]},{"osd":6,"up_from":48,"seq":206158430235,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32544,"kb_used_data":1368,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712608,"statfs":{"total":95995035648,"available":95961710592,"internally_reserved":0,"allocated":1400832,"data_stored":935573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Feb 13 14:15:36 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.505}]},{"osd":1,"last update":"Tue Feb 13 14:15:36 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.63800000000000001}]},{"osd":2,"last update":"Tue Feb 13 14:15:36 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.55600000000000005}]},{"osd":3,"last update":"Tue Feb 13 14:15:36 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.67900000000000005}]},{"osd":4,"last update":"Tue Feb 13 14:15:36 2024","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.70299999999999996}]},{"osd":5,"last update":"Tue Feb 13 14:15:36 2024","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.434,"15min":0.434},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.624},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.66100000000000003}]},{"osd":7,"last update":"Tue Feb 13 14:16:00 2024","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.749},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.77200000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574901,"num_pgs":54,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27864,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717288,"statfs":{"total":95995035648,"available":95966502912,"internally_reserved":0,"allocated":864256,"data_stored":347437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Feb 13 14:15:31 2024","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.32100000000000001,"15min":0.29599999999999999},"min":{"1min":0.24199999999999999,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"last":0.248},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.33700000000000002,"15min":0.311},"min":{"1min":0.22800000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.51000000000000001,"5min":0.54800000000000004,"15min":0.54800000000000004},"last":0.36599999999999999}]},{"osd":2,"last update":"Tue Feb 13 14:16:01 2024","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.377,"15min":0.375},"min":{"1min":0.19800000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.308},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.40200000000000002,"15min":0.39200000000000002},"min":{"1min":0.27900000000000003,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.622,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.28699999999999998}]},{"osd":3,"last update":"Tue Feb 13 14:16:21 2024","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.439,"15min":0.45100000000000001},"min":{"1min":0.29299999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.624,"5min":0.63,"15min":0.63},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.46999999999999997,"15min":0.47399999999999998},"min":{"1min":0.32100000000000001,"5min":0.309,"15min":0.309},"max":{"1min":0.67800000000000005,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.33700000000000002}]},{"osd":4,"last update":"Tue Feb 13 14:15:41 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.56599999999999995,"15min":0.56799999999999995},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.41999999999999998}]},{"osd":5,"last update":"Tue Feb 13 14:16:09 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.60499999999999998,"15min":0.60699999999999998},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.81699999999999995,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.59799999999999998,"15min":0.59699999999999998},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.79200000000000004,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.53900000000000003}]},{"osd":6,"last update":"Tue Feb 13 14:15:36 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.45300000000000001}]},{"osd":7,"last update":"Tue Feb 13 14:16:01 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.55200000000000005}]}]},{"osd":0,"up_from":8,"seq":34359738426,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32576,"kb_used_data":1396,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712576,"statfs":{"total":95995035648,"available":95961677824,"internally_reserved":0,"allocated":1429504,"data_stored":936765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Feb 13 14:15:32 2024","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.33200000000000002,"15min":0.30599999999999999},"min":{"1min":0.245,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.52400000000000002,"5min":0.61299999999999999,"15min":0.61299999999999999},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.41199999999999998,"5min":0.34899999999999998,"15min":0.31900000000000001},"min":{"1min":0.22500000000000001,"5min":0.223,"15min":0.223},"max":{"1min":0.60299999999999998,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.35599999999999998}]},{"osd":2,"last update":"Tue Feb 13 14:15:58 2024","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.39300000000000002,"15min":0.379},"min":{"1min":0.27200000000000002,"5min":0.246,"15min":0.246},"max":{"1min":0.49299999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"last":0.374},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.40100000000000002,"15min":0.39300000000000002},"min":{"1min":0.26300000000000001,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"last":0.33000000000000002}]},{"osd":3,"last update":"Tue Feb 13 14:16:21 2024","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.45800000000000002,"15min":0.46600000000000003},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.57299999999999995,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.432,"15min":0.436},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.56499999999999995,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.41199999999999998}]},{"osd":4,"last update":"Tue Feb 13 14:15:46 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.59399999999999997,"15min":0.59799999999999998},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.70499999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.625,"5min":0.59499999999999997,"15min":0.58999999999999997},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.60399999999999998}]},{"osd":5,"last update":"Tue Feb 13 14:16:14 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.59899999999999998,"15min":0.60099999999999998},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.76400000000000001,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.58399999999999996,"15min":0.58099999999999996},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.622}]},{"osd":6,"last update":"Tue Feb 13 14:15:32 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.64000000000000001}]},{"osd":7,"last update":"Tue Feb 13 14:15:58 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.58599999999999997}]}]},{"osd":2,"up_from":18,"seq":77309411375,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27820,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717332,"statfs":{"total":95995035648,"available":95966547968,"internally_reserved":0,"allocated":819200,"data_stored":345662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Feb 13 14:15:59 2024","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.35399999999999998,"15min":0.34399999999999997},"min":{"1min":0.219,"5min":0.217,"15min":0.217},"max":{"1min":0.47399999999999998,"5min":0.503,"15min":0.503},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.375,"15min":0.35799999999999998},"min":{"1min":0.23999999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.55900000000000005,"5min":0.57899999999999996,"15min":0.57899999999999996},"last":0.35299999999999998}]},{"osd":1,"last update":"Tue Feb 13 14:15:59 2024","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.36099999999999999,"15min":0.34399999999999997},"min":{"1min":0.27800000000000002,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.51500000000000001,"5min":0.58399999999999996,"15min":0.58399999999999996},"last":0.315},{"interface":"front","average":{"1min":0.435,"5min":0.39400000000000002,"15min":0.36899999999999999},"min":{"1min":0.30199999999999999,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.252}]},{"osd":3,"last update":"Tue Feb 13 14:16:20 2024","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.47499999999999998,"15min":0.48099999999999998},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.29199999999999998},{"interface":"front","average":{"1min":0.39600000000000002,"5min":0.44600000000000001,"15min":0.45400000000000001},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.57699999999999996,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.33200000000000002}]},{"osd":4,"last update":"Tue Feb 13 14:15:47 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.55700000000000005,"15min":0.55600000000000005},"min":{"1min":0.42699999999999999,"5min":0.371,"15min":0.371},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.59599999999999997,"15min":0.59399999999999997},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.76600000000000001,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.44600000000000001}]},{"osd":5,"last update":"Tue Feb 13 14:16:11 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.63400000000000001,"15min":0.64000000000000001},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.501},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.60399999999999998,"15min":0.60699999999999998},"min":{"1min":0.40200000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.46899999999999997}]},{"osd":6,"last update":"Tue Feb 13 14:15:34 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.55200000000000005}]},{"osd":7,"last update":"Tue Feb 13 14:15:59 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.52600000000000002}]}]},{"osd":3,"up_from":26,"seq":111669149738,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32008,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713144,"statfs":{"total":95995035648,"available":95962259456,"internally_reserved":0,"allocated":847872,"data_stored":346494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Feb 13 14:16:21 2024","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.38200000000000001,"15min":0.38400000000000001},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.58999999999999997,"5min":0.63,"15min":0.63},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.33800000000000002,"5min":0.40799999999999997,"15min":0.42699999999999999},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.45200000000000001,"5min":0.60699999999999998,"15min":0.60699999999999998},"last":0.35199999999999998}]},{"osd":1,"last update":"Tue Feb 13 14:16:21 2024","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.42599999999999999,"15min":0.439},"min":{"1min":0.26200000000000001,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.48299999999999998,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.35899999999999999,"5min":0.41199999999999998,"15min":0.42599999999999999},"min":{"1min":0.25600000000000001,"5min":0.216,"15min":0.216},"max":{"1min":0.56100000000000005,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.313}]},{"osd":2,"last update":"Tue Feb 13 14:16:21 2024","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.44600000000000001,"15min":0.46200000000000002},"min":{"1min":0.28100000000000003,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.5,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.41999999999999998,"15min":0.42799999999999999},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.52400000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.33200000000000002}]},{"osd":4,"last update":"Tue Feb 13 14:15:41 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.54700000000000004,"15min":0.54900000000000004},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.66400000000000003}]},{"osd":5,"last update":"Tue Feb 13 14:16:08 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.57399999999999995,"15min":0.57599999999999996},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.71299999999999997,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.58399999999999996,"15min":0.59199999999999997},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.67300000000000004}]},{"osd":6,"last update":"Tue Feb 13 14:16:33 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.60999999999999999,"15min":0.61399999999999999},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.60099999999999998,"15min":0.60699999999999998},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.79900000000000004,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.70599999999999996}]},{"osd":7,"last update":"Tue Feb 13 14:16:00 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.60799999999999998}]}]},{"osd":4,"up_from":31,"seq":133143986213,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32004,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713148,"statfs":{"total":95995035648,"available":95962263552,"internally_reserved":0,"allocated":843776,"data_stored":346697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Feb 13 14:15:42 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.58599999999999997,"15min":0.58999999999999997},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.629}]},{"osd":1,"last update":"Tue Feb 13 14:15:42 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.57299999999999995,"15min":0.57699999999999996},"min":{"1min":0.379,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.57099999999999995,"15min":0.57299999999999995},"min":{"1min":0.39900000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.77000000000000002,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.57999999999999996}]},{"osd":2,"last update":"Tue Feb 13 14:15:42 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.56699999999999995,"15min":0.56899999999999995},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.56699999999999995,"15min":0.57099999999999995},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.59899999999999998}]},{"osd":3,"last update":"Tue Feb 13 14:15:42 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.57799999999999996,"15min":0.58199999999999996},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56000000000000005,"15min":0.55800000000000005},"min":{"1min":0.41699999999999998,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.79000000000000004,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.61399999999999999}]},{"osd":5,"last update":"Tue Feb 13 14:16:11 2024","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.42799999999999999,"15min":0.42499999999999999},"min":{"1min":0.29499999999999998,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.59399999999999997,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.503},{"interface":"front","average":{"1min":0.433,"5min":0.433,"15min":0.433},"min":{"1min":0.32000000000000001,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.622,"5min":0.749,"15min":0.749},"last":0.40600000000000003}]},{"osd":6,"last update":"Tue Feb 13 14:15:33 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.42799999999999999}]},{"osd":7,"last update":"Tue Feb 13 14:15:58 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.44500000000000001}]}]},{"osd":5,"up_from":39,"seq":167503724575,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31976,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713176,"statfs":{"total":95995035648,"available":95962292224,"internally_reserved":0,"allocated":815104,"data_stored":345640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Feb 13 14:16:12 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.52400000000000002,"15min":0.52000000000000002},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.46600000000000003}]},{"osd":1,"last update":"Tue Feb 13 14:16:12 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.52300000000000002,"15min":0.52100000000000002},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.54400000000000004,"15min":0.54200000000000004},"min":{"1min":0.39800000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.54000000000000004}]},{"osd":2,"last update":"Tue Feb 13 14:16:12 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.53800000000000003,"15min":0.54000000000000004},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.80400000000000005,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.70599999999999996}]},{"osd":3,"last update":"Tue Feb 13 14:16:12 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.53100000000000003,"15min":0.53200000000000003},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.755,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.57499999999999996,"15min":0.57699999999999996},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.76600000000000001}]},{"osd":4,"last update":"Tue Feb 13 14:16:12 2024","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.371,"15min":0.36399999999999999},"min":{"1min":0.28000000000000003,"5min":0.214,"15min":0.214},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.38400000000000001,"15min":0.378},"min":{"1min":0.26400000000000001,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.78300000000000003}]},{"osd":6,"last update":"Tue Feb 13 14:15:32 2024","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.81299999999999994}]},{"osd":7,"last update":"Tue Feb 13 14:15:58 2024","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.754},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.82999999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-13T14:16:37.487 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-02-13T14:16:37.487 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-02-13T14:16:37.487 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-02-13T14:16:37.487 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662 shell --fsid 5c04b074-ca79-11ee-95b9-87774f69a715 -- ceph health --format=json 2024-02-13T14:16:37.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:16:37 smithi087 ceph-mon[27372]: from='client.24620 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-13T14:16:37.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:16:37 smithi087 ceph-mon[39120]: from='client.24620 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-13T14:16:37.992 INFO:journalctl@ceph.alertmanager.a.smithi087.stdout:Feb 13 14:16:37 smithi087 podman[76631]: 2024-02-13 14:16:37.83730333 +0000 UTC m=+0.229420431 container create 689561ab5d2d12cfe733ccf3a3d77c114b936f8b532c2b8c5fde4bdc05b6f4fe (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-02-13T14:16:38.118 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:16:37 smithi148 ceph-mon[31198]: from='client.24620 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-13T14:16:38.434 INFO:journalctl@ceph.alertmanager.a.smithi087.stdout:Feb 13 14:16:38 smithi087 podman[76631]: 2024-02-13 14:16:38.123339236 +0000 UTC m=+0.515456337 container init 689561ab5d2d12cfe733ccf3a3d77c114b936f8b532c2b8c5fde4bdc05b6f4fe (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-02-13T14:16:38.435 INFO:journalctl@ceph.alertmanager.a.smithi087.stdout:Feb 13 14:16:38 smithi087 podman[76631]: 2024-02-13 14:16:38.128629119 +0000 UTC m=+0.520746222 container start 689561ab5d2d12cfe733ccf3a3d77c114b936f8b532c2b8c5fde4bdc05b6f4fe (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-02-13T14:16:38.435 INFO:journalctl@ceph.alertmanager.a.smithi087.stdout:Feb 13 14:16:38 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-alertmanager-a[76658]: ts=2024-02-13T14:16:38.141Z caller=main.go:240 level=info msg="Starting Alertmanager" version="(version=0.25.0, branch=HEAD, revision=258fab7cdd551f2cf251ed0348f0ad7289aee789)" 2024-02-13T14:16:38.435 INFO:journalctl@ceph.alertmanager.a.smithi087.stdout:Feb 13 14:16:38 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-alertmanager-a[76658]: ts=2024-02-13T14:16:38.141Z caller=main.go:241 level=info build_context="(go=go1.19.4, user=root@abe866dd5717, date=20221222-14:51:36)" 2024-02-13T14:16:38.435 INFO:journalctl@ceph.alertmanager.a.smithi087.stdout:Feb 13 14:16:38 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-alertmanager-a[76658]: ts=2024-02-13T14:16:38.142Z caller=cluster.go:185 level=info component=cluster msg="setting advertise address explicitly" addr=172.21.15.87 port=9094 2024-02-13T14:16:38.435 INFO:journalctl@ceph.alertmanager.a.smithi087.stdout:Feb 13 14:16:38 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-alertmanager-a[76658]: ts=2024-02-13T14:16:38.143Z caller=cluster.go:681 level=info component=cluster msg="Waiting for gossip to settle..." interval=2s 2024-02-13T14:16:38.435 INFO:journalctl@ceph.alertmanager.a.smithi087.stdout:Feb 13 14:16:38 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-alertmanager-a[76658]: ts=2024-02-13T14:16:38.166Z caller=coordinator.go:113 level=info component=configuration msg="Loading configuration file" file=/etc/alertmanager/alertmanager.yml 2024-02-13T14:16:38.435 INFO:journalctl@ceph.alertmanager.a.smithi087.stdout:Feb 13 14:16:38 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-alertmanager-a[76658]: ts=2024-02-13T14:16:38.166Z caller=coordinator.go:126 level=info component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/alertmanager.yml 2024-02-13T14:16:38.435 INFO:journalctl@ceph.alertmanager.a.smithi087.stdout:Feb 13 14:16:38 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-alertmanager-a[76658]: ts=2024-02-13T14:16:38.169Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9093 2024-02-13T14:16:38.435 INFO:journalctl@ceph.alertmanager.a.smithi087.stdout:Feb 13 14:16:38 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-alertmanager-a[76658]: ts=2024-02-13T14:16:38.169Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9093 2024-02-13T14:16:38.436 INFO:journalctl@ceph.alertmanager.a.smithi087.stdout:Feb 13 14:16:38 smithi087 bash[76631]: 689561ab5d2d12cfe733ccf3a3d77c114b936f8b532c2b8c5fde4bdc05b6f4fe 2024-02-13T14:16:38.508 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/mon.a/config 2024-02-13T14:16:38.687 INFO:journalctl@ceph.alertmanager.a.smithi087.stdout:Feb 13 14:16:38 smithi087 systemd[1]: Started Ceph alertmanager.a for 5c04b074-ca79-11ee-95b9-87774f69a715. 2024-02-13T14:16:38.687 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:16:38 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: [13/Feb/2024:14:16:38] ENGINE Bus STOPPING 2024-02-13T14:16:38.687 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:16:38 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: [13/Feb/2024:14:16:38] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-02-13T14:16:38.687 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:16:38 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: [13/Feb/2024:14:16:38] ENGINE Bus STOPPED 2024-02-13T14:16:38.687 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:16:38 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: [13/Feb/2024:14:16:38] ENGINE Bus STARTING 2024-02-13T14:16:38.687 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:16:38 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: [13/Feb/2024:14:16:38] ENGINE Serving on http://:::9283 2024-02-13T14:16:38.687 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:16:38 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: [13/Feb/2024:14:16:38] ENGINE Bus STARTED 2024-02-13T14:16:38.938 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:16:38 smithi087 ceph-mon[27372]: pgmap v25: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-13T14:16:38.939 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:16:38 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:16:38.939 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:16:38 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:16:38.939 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:16:38 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:16:38.939 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:16:38 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:16:38.939 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:16:38 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:16:38.939 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:16:38 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:16:38.939 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:16:38 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-13T14:16:38.939 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:16:38 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:16:38.939 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:16:38 smithi087 ceph-mon[39120]: pgmap v25: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-13T14:16:38.939 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:16:38 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:16:38.940 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:16:38 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:16:38.940 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:16:38 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:16:38.940 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:16:38 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:16:38.940 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:16:38 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:16:38.940 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:16:38 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:16:38.940 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:16:38 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-13T14:16:38.940 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:16:38 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:16:39.118 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:16:38 smithi148 ceph-mon[31198]: pgmap v25: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-13T14:16:39.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:16:38 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:16:39.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:16:38 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:16:39.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:16:38 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:16:39.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:16:38 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:16:39.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:16:38 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:16:39.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:16:38 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:16:39.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:16:38 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-13T14:16:39.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:16:38 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:16:39.960 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:16:39 smithi087 ceph-mon[39120]: Regenerating cephadm self-signed grafana TLS certificates 2024-02-13T14:16:39.960 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:16:39 smithi087 ceph-mon[39120]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-13T14:16:39.960 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:16:39 smithi087 ceph-mon[39120]: Deploying daemon grafana.a on smithi148 2024-02-13T14:16:39.960 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:16:39 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:16:39.960 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:16:39 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-13T14:16:39.960 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:16:39 smithi087 ceph-mon[27372]: Regenerating cephadm self-signed grafana TLS certificates 2024-02-13T14:16:39.960 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:16:39 smithi087 ceph-mon[27372]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-13T14:16:39.960 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:16:39 smithi087 ceph-mon[27372]: Deploying daemon grafana.a on smithi148 2024-02-13T14:16:39.960 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:16:39 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:16:39.961 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:16:39 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-13T14:16:40.118 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:16:39 smithi148 ceph-mon[31198]: Regenerating cephadm self-signed grafana TLS certificates 2024-02-13T14:16:40.118 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:16:39 smithi148 ceph-mon[31198]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-13T14:16:40.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:16:39 smithi148 ceph-mon[31198]: Deploying daemon grafana.a on smithi148 2024-02-13T14:16:40.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:16:39 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:16:40.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:16:39 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-13T14:16:40.241 INFO:journalctl@ceph.alertmanager.a.smithi087.stdout:Feb 13 14:16:40 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-alertmanager-a[76658]: ts=2024-02-13T14:16:40.143Z caller=cluster.go:706 level=info component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.000064517s 2024-02-13T14:16:40.442 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-02-13T14:16:40.442 INFO:teuthology.orchestra.run.smithi087.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-02-13T14:16:40.991 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:16:40 smithi087 ceph-mon[27372]: pgmap v26: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:16:40.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:16:40 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/1801848575' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-02-13T14:16:40.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:16:40 smithi087 ceph-mon[39120]: pgmap v26: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:16:40.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:16:40 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/1801848575' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-02-13T14:16:41.034 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-02-13T14:16:41.035 INFO:tasks.cephadm:Setup complete, yielding 2024-02-13T14:16:41.035 INFO:teuthology.run_tasks:Running task workunit... 2024-02-13T14:16:41.048 INFO:tasks.workunit:Pulling workunits from ref e1b79d5a4193a484176863ab96c0ca28c00e1662 2024-02-13T14:16:41.048 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-02-13T14:16:41.049 DEBUG:teuthology.orchestra.run.smithi087:> stat -- /home/ubuntu/cephtest/mnt.0 2024-02-13T14:16:41.065 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-13T14:16:41.066 INFO:teuthology.orchestra.run.smithi087.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-02-13T14:16:41.066 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -- /home/ubuntu/cephtest/mnt.0 2024-02-13T14:16:41.118 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:16:40 smithi148 ceph-mon[31198]: pgmap v26: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:16:41.118 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:16:40 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/1801848575' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-02-13T14:16:41.121 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2024-02-13T14:16:41.122 DEBUG:teuthology.orchestra.run.smithi087:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2024-02-13T14:16:41.178 INFO:tasks.workunit:timeout=1h 2024-02-13T14:16:41.178 INFO:tasks.workunit:cleanup=True 2024-02-13T14:16:41.179 DEBUG:teuthology.orchestra.run.smithi087:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout e1b79d5a4193a484176863ab96c0ca28c00e1662 2024-02-13T14:16:41.234 INFO:tasks.workunit.client.0.smithi087.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-02-13T14:16:43.118 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:16:42 smithi148 ceph-mon[31198]: pgmap v27: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-13T14:16:43.241 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:16:42 smithi087 ceph-mon[27372]: pgmap v27: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-13T14:16:43.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:16:42 smithi087 ceph-mon[39120]: pgmap v27: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-13T14:16:45.118 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:16:44 smithi148 ceph-mon[31198]: pgmap v28: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-13T14:16:45.119 INFO:journalctl@ceph.iscsi.iscsi.a.smithi148.stdout:Feb 13 14:16:44 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-iscsi-iscsi-a[59644]: debug there is no tcmu-runner data available 2024-02-13T14:16:45.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:16:44 smithi087 ceph-mon[27372]: pgmap v28: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-13T14:16:45.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:16:44 smithi087 ceph-mon[39120]: pgmap v28: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-13T14:16:46.118 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:16:45 smithi148 ceph-mon[31198]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:16:46.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:16:45 smithi087 ceph-mon[27372]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:16:46.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:16:45 smithi087 ceph-mon[39120]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:16:47.118 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:16:46 smithi148 ceph-mon[31198]: pgmap v29: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:16:47.222 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:16:46 smithi087 ceph-mon[27372]: pgmap v29: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:16:47.222 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:16:46 smithi087 ceph-mon[39120]: pgmap v29: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:16:47.492 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:16:47 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: ::ffff:172.21.15.148 - - [13/Feb/2024:14:16:47] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-13T14:16:48.492 INFO:journalctl@ceph.alertmanager.a.smithi087.stdout:Feb 13 14:16:48 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-alertmanager-a[76658]: ts=2024-02-13T14:16:48.145Z caller=cluster.go:698 level=info component=cluster msg="gossip settled; proceeding" elapsed=10.002561816s 2024-02-13T14:16:49.118 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:16:48 smithi148 ceph-mon[31198]: pgmap v30: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-13T14:16:49.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:16:48 smithi087 ceph-mon[39120]: pgmap v30: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-13T14:16:49.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:16:48 smithi087 ceph-mon[27372]: pgmap v30: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-13T14:16:51.118 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:16:50 smithi148 ceph-mon[31198]: pgmap v31: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:16:51.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:16:50 smithi087 ceph-mon[39120]: pgmap v31: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:16:51.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:16:50 smithi087 ceph-mon[27372]: pgmap v31: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:16:53.074 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:16:52 smithi148 ceph-mon[31198]: pgmap v32: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-13T14:16:53.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:16:52 smithi087 ceph-mon[27372]: pgmap v32: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-13T14:16:53.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:16:52 smithi087 ceph-mon[39120]: pgmap v32: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-13T14:16:54.612 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:54 smithi148 systemd[1]: Starting Ceph grafana.a for 5c04b074-ca79-11ee-95b9-87774f69a715... 2024-02-13T14:16:54.868 INFO:journalctl@ceph.iscsi.iscsi.a.smithi148.stdout:Feb 13 14:16:54 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-iscsi-iscsi-a[59644]: debug there is no tcmu-runner data available 2024-02-13T14:16:54.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:16:54 smithi148 ceph-mon[31198]: pgmap v33: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-13T14:16:54.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:16:54 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-13T14:16:54.869 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:54 smithi148 podman[63649]: 2024-02-13 14:16:54.51318322 +0000 UTC m=+0.017608522 image pull a0ea5e3c46d9434bd173d585b7c2638c7451eaf2543d1d22f9e78442615aaffb quay.io/ceph/ceph-grafana:9.4.12 2024-02-13T14:16:54.869 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:54 smithi148 podman[63649]: 2024-02-13 14:16:54.61912307 +0000 UTC m=+0.123548356 container create c87535293afd6bf0cc3f025d1f7af9f40a3411572d29b658fa7fbec31f287d2b (image=quay.io/ceph/ceph-grafana:9.4.12, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a, description=Ceph Grafana Container, io.openshift.tags=base rhel8, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, version=8.5, build-date=2022-04-29T11:48:11.055897, vendor=Red Hat, Inc., io.openshift.expose-services=, vcs-type=git, com.redhat.component=ubi8-container, architecture=x86_64, distribution-scope=public, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, summary=Grafana Container configured for Ceph mgr/dashboard integration, release=239.1651231664, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, io.buildah.version=1.32.0, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, io.k8s.display-name=Red Hat Universal Base Image 8, maintainer=Paul Cuzner , io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., name=ubi8) 2024-02-13T14:16:54.869 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:54 smithi148 podman[63649]: 2024-02-13 14:16:54.854568575 +0000 UTC m=+0.358993861 container init c87535293afd6bf0cc3f025d1f7af9f40a3411572d29b658fa7fbec31f287d2b (image=quay.io/ceph/ceph-grafana:9.4.12, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, name=ubi8, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, maintainer=Paul Cuzner , distribution-scope=public, release=239.1651231664, description=Ceph Grafana Container, vendor=Red Hat, Inc., architecture=x86_64, io.buildah.version=1.32.0, io.openshift.expose-services=, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., vcs-type=git, com.redhat.component=ubi8-container, summary=Grafana Container configured for Ceph mgr/dashboard integration, version=8.5, build-date=2022-04-29T11:48:11.055897, io.openshift.tags=base rhel8, io.k8s.display-name=Red Hat Universal Base Image 8) 2024-02-13T14:16:54.870 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:54 smithi148 podman[63649]: 2024-02-13 14:16:54.859681234 +0000 UTC m=+0.364106530 container start c87535293afd6bf0cc3f025d1f7af9f40a3411572d29b658fa7fbec31f287d2b (image=quay.io/ceph/ceph-grafana:9.4.12, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a, vcs-type=git, com.redhat.component=ubi8-container, distribution-scope=public, vendor=Red Hat, Inc., build-date=2022-04-29T11:48:11.055897, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, io.k8s.display-name=Red Hat Universal Base Image 8, maintainer=Paul Cuzner , summary=Grafana Container configured for Ceph mgr/dashboard integration, name=ubi8, release=239.1651231664, io.buildah.version=1.32.0, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., description=Ceph Grafana Container, io.openshift.expose-services=, io.openshift.tags=base rhel8, architecture=x86_64, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, version=8.5, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com) 2024-02-13T14:16:55.225 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:54 smithi148 bash[63649]: c87535293afd6bf0cc3f025d1f7af9f40a3411572d29b658fa7fbec31f287d2b 2024-02-13T14:16:55.225 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:54 smithi148 systemd[1]: Started Ceph grafana.a for 5c04b074-ca79-11ee-95b9-87774f69a715. 2024-02-13T14:16:55.225 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=settings t=2024-02-13T14:16:55.016442864Z level=info msg="Starting Grafana" version=9.4.12 commit=37ee25ba46 branch=HEAD compiled=2023-05-22T15:58:05Z 2024-02-13T14:16:55.225 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=settings t=2024-02-13T14:16:55.016742349Z level=warn msg="\"sentry\" frontend logging provider is deprecated and will be removed in the next major version. Use \"grafana\" provider instead." 2024-02-13T14:16:55.225 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=settings t=2024-02-13T14:16:55.016771573Z level=info msg="Config loaded from" file=/usr/share/grafana/conf/defaults.ini 2024-02-13T14:16:55.226 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=settings t=2024-02-13T14:16:55.016786307Z level=info msg="Config loaded from" file=/etc/grafana/grafana.ini 2024-02-13T14:16:55.226 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=settings t=2024-02-13T14:16:55.016798503Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_DATA=/var/lib/grafana" 2024-02-13T14:16:55.226 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=settings t=2024-02-13T14:16:55.016810043Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_LOGS=/var/log/grafana" 2024-02-13T14:16:55.226 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=settings t=2024-02-13T14:16:55.016821135Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_PLUGINS=/var/lib/grafana/plugins" 2024-02-13T14:16:55.226 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=settings t=2024-02-13T14:16:55.016833003Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_PROVISIONING=/etc/grafana/provisioning" 2024-02-13T14:16:55.226 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=settings t=2024-02-13T14:16:55.016846466Z level=info msg="Path Home" path=/usr/share/grafana 2024-02-13T14:16:55.226 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=settings t=2024-02-13T14:16:55.016857443Z level=info msg="Path Data" path=/var/lib/grafana 2024-02-13T14:16:55.226 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=settings t=2024-02-13T14:16:55.01686945Z level=info msg="Path Logs" path=/var/log/grafana 2024-02-13T14:16:55.226 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=settings t=2024-02-13T14:16:55.016880384Z level=info msg="Path Plugins" path=/var/lib/grafana/plugins 2024-02-13T14:16:55.226 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=settings t=2024-02-13T14:16:55.016891507Z level=info msg="Path Provisioning" path=/etc/grafana/provisioning 2024-02-13T14:16:55.226 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=settings t=2024-02-13T14:16:55.016904965Z level=info msg="App mode production" 2024-02-13T14:16:55.226 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=sqlstore t=2024-02-13T14:16:55.017005961Z level=info msg="Connecting to DB" dbtype=sqlite3 2024-02-13T14:16:55.227 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=sqlstore t=2024-02-13T14:16:55.017044063Z level=warn msg="SQLite database file has broader permissions than it should" path=/var/lib/grafana/grafana.db mode=-rw-r--r-- expected=-rw-r----- 2024-02-13T14:16:55.227 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.018270551Z level=info msg="Starting DB migrations" 2024-02-13T14:16:55.227 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.018410806Z level=info msg="Executing migration" id="create migration_log table" 2024-02-13T14:16:55.227 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.019164739Z level=info msg="Executing migration" id="create user table" 2024-02-13T14:16:55.227 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.019719999Z level=info msg="Executing migration" id="add unique index user.login" 2024-02-13T14:16:55.227 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.020220789Z level=info msg="Executing migration" id="add unique index user.email" 2024-02-13T14:16:55.227 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.020761157Z level=info msg="Executing migration" id="drop index UQE_user_login - v1" 2024-02-13T14:16:55.227 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.021420995Z level=info msg="Executing migration" id="drop index UQE_user_email - v1" 2024-02-13T14:16:55.227 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.021831262Z level=info msg="Executing migration" id="Rename table user to user_v1 - v1" 2024-02-13T14:16:55.227 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.022541119Z level=info msg="Executing migration" id="create user table v2" 2024-02-13T14:16:55.227 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.022941386Z level=info msg="Executing migration" id="create index UQE_user_login - v2" 2024-02-13T14:16:55.227 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.023346141Z level=info msg="Executing migration" id="create index UQE_user_email - v2" 2024-02-13T14:16:55.227 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.023784506Z level=info msg="Executing migration" id="copy data_source v1 to v2" 2024-02-13T14:16:55.228 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.024102881Z level=info msg="Executing migration" id="Drop old table user_v1" 2024-02-13T14:16:55.228 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.024466593Z level=info msg="Executing migration" id="Add column help_flags1 to user table" 2024-02-13T14:16:55.228 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.024863507Z level=info msg="Executing migration" id="Update user table charset" 2024-02-13T14:16:55.228 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.025178274Z level=info msg="Executing migration" id="Add last_seen_at column to user" 2024-02-13T14:16:55.228 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.025608216Z level=info msg="Executing migration" id="Add missing user data" 2024-02-13T14:16:55.228 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.025953179Z level=info msg="Executing migration" id="Add is_disabled column to user" 2024-02-13T14:16:55.228 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.026389227Z level=info msg="Executing migration" id="Add index user.login/user.email" 2024-02-13T14:16:55.228 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.026782195Z level=info msg="Executing migration" id="Add is_service_account column to user" 2024-02-13T14:16:55.228 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.027322702Z level=info msg="Executing migration" id="Update is_service_account column to nullable" 2024-02-13T14:16:55.229 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.028852873Z level=info msg="Executing migration" id="create temp user table v1-7" 2024-02-13T14:16:55.229 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.029309594Z level=info msg="Executing migration" id="create index IDX_temp_user_email - v1-7" 2024-02-13T14:16:55.229 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.029759103Z level=info msg="Executing migration" id="create index IDX_temp_user_org_id - v1-7" 2024-02-13T14:16:55.230 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.030643542Z level=info msg="Executing migration" id="create index IDX_temp_user_code - v1-7" 2024-02-13T14:16:55.230 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.031065258Z level=info msg="Executing migration" id="create index IDX_temp_user_status - v1-7" 2024-02-13T14:16:55.230 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.031469332Z level=info msg="Executing migration" id="Update temp_user table charset" 2024-02-13T14:16:55.230 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.031737119Z level=info msg="Executing migration" id="drop index IDX_temp_user_email - v1" 2024-02-13T14:16:55.230 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.032115109Z level=info msg="Executing migration" id="drop index IDX_temp_user_org_id - v1" 2024-02-13T14:16:55.230 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.032513244Z level=info msg="Executing migration" id="drop index IDX_temp_user_code - v1" 2024-02-13T14:16:55.230 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.032870386Z level=info msg="Executing migration" id="drop index IDX_temp_user_status - v1" 2024-02-13T14:16:55.230 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.033264479Z level=info msg="Executing migration" id="Rename table temp_user to temp_user_tmp_qwerty - v1" 2024-02-13T14:16:55.230 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.03388339Z level=info msg="Executing migration" id="create temp_user v2" 2024-02-13T14:16:55.230 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.034284893Z level=info msg="Executing migration" id="create index IDX_temp_user_email - v2" 2024-02-13T14:16:55.230 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.034692795Z level=info msg="Executing migration" id="create index IDX_temp_user_org_id - v2" 2024-02-13T14:16:55.230 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.035095327Z level=info msg="Executing migration" id="create index IDX_temp_user_code - v2" 2024-02-13T14:16:55.230 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.035517161Z level=info msg="Executing migration" id="create index IDX_temp_user_status - v2" 2024-02-13T14:16:55.231 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.035924024Z level=info msg="Executing migration" id="copy temp_user v1 to v2" 2024-02-13T14:16:55.231 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.036274171Z level=info msg="Executing migration" id="drop temp_user_tmp_qwerty" 2024-02-13T14:16:55.231 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.038701307Z level=info msg="Executing migration" id="Set created for temp users that will otherwise prematurely expire" 2024-02-13T14:16:55.231 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.041468333Z level=info msg="Executing migration" id="create star table" 2024-02-13T14:16:55.231 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.044271876Z level=info msg="Executing migration" id="add unique index star.user_id_dashboard_id" 2024-02-13T14:16:55.231 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.047152417Z level=info msg="Executing migration" id="create org table v1" 2024-02-13T14:16:55.231 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.050071175Z level=info msg="Executing migration" id="create index UQE_org_name - v1" 2024-02-13T14:16:55.231 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.052801874Z level=info msg="Executing migration" id="create org_user table v1" 2024-02-13T14:16:55.231 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.055568482Z level=info msg="Executing migration" id="create index IDX_org_user_org_id - v1" 2024-02-13T14:16:55.231 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.058264684Z level=info msg="Executing migration" id="create index UQE_org_user_org_id_user_id - v1" 2024-02-13T14:16:55.231 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.061254279Z level=info msg="Executing migration" id="create index IDX_org_user_user_id - v1" 2024-02-13T14:16:55.231 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.064204018Z level=info msg="Executing migration" id="Update org table charset" 2024-02-13T14:16:55.231 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.066678833Z level=info msg="Executing migration" id="Update org_user table charset" 2024-02-13T14:16:55.231 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.069370491Z level=info msg="Executing migration" id="Migrate all Read Only Viewers to Viewers" 2024-02-13T14:16:55.231 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.072321027Z level=info msg="Executing migration" id="create dashboard table" 2024-02-13T14:16:55.232 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.075129652Z level=info msg="Executing migration" id="add index dashboard.account_id" 2024-02-13T14:16:55.232 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.077996682Z level=info msg="Executing migration" id="add unique index dashboard_account_id_slug" 2024-02-13T14:16:55.232 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.080688644Z level=info msg="Executing migration" id="create dashboard_tag table" 2024-02-13T14:16:55.232 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.083251713Z level=info msg="Executing migration" id="add unique index dashboard_tag.dasboard_id_term" 2024-02-13T14:16:55.232 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.086047572Z level=info msg="Executing migration" id="drop index UQE_dashboard_tag_dashboard_id_term - v1" 2024-02-13T14:16:55.232 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.0886744Z level=info msg="Executing migration" id="Rename table dashboard to dashboard_v1 - v1" 2024-02-13T14:16:55.232 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.089590708Z level=info msg="Executing migration" id="create dashboard v2" 2024-02-13T14:16:55.232 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.092177479Z level=info msg="Executing migration" id="create index IDX_dashboard_org_id - v2" 2024-02-13T14:16:55.232 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.094795005Z level=info msg="Executing migration" id="create index UQE_dashboard_org_id_slug - v2" 2024-02-13T14:16:55.232 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.09746084Z level=info msg="Executing migration" id="copy dashboard v1 to v2" 2024-02-13T14:16:55.232 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.097813608Z level=info msg="Executing migration" id="drop table dashboard_v1" 2024-02-13T14:16:55.232 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.100450228Z level=info msg="Executing migration" id="alter dashboard.data to mediumtext v1" 2024-02-13T14:16:55.233 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.100788646Z level=info msg="Executing migration" id="Add column updated_by in dashboard - v2" 2024-02-13T14:16:55.233 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.103244937Z level=info msg="Executing migration" id="Add column created_by in dashboard - v2" 2024-02-13T14:16:55.233 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.103791981Z level=info msg="Executing migration" id="Add column gnetId in dashboard" 2024-02-13T14:16:55.233 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.105887691Z level=info msg="Executing migration" id="Add index for gnetId in dashboard" 2024-02-13T14:16:55.233 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.108651173Z level=info msg="Executing migration" id="Add column plugin_id in dashboard" 2024-02-13T14:16:55.233 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.111096658Z level=info msg="Executing migration" id="Add index for plugin_id in dashboard" 2024-02-13T14:16:55.233 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.113744099Z level=info msg="Executing migration" id="Add index for dashboard_id in dashboard_tag" 2024-02-13T14:16:55.233 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.116347327Z level=info msg="Executing migration" id="Update dashboard table charset" 2024-02-13T14:16:55.233 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.117173258Z level=info msg="Executing migration" id="Update dashboard_tag table charset" 2024-02-13T14:16:55.233 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.118133457Z level=info msg="Executing migration" id="Add column folder_id in dashboard" 2024-02-13T14:16:55.233 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.118891689Z level=info msg="Executing migration" id="Add column isFolder in dashboard" 2024-02-13T14:16:55.233 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.119900327Z level=info msg="Executing migration" id="Add column has_acl in dashboard" 2024-02-13T14:16:55.233 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.120641845Z level=info msg="Executing migration" id="Add column uid in dashboard" 2024-02-13T14:16:55.233 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.121524887Z level=info msg="Executing migration" id="Update uid column values in dashboard" 2024-02-13T14:16:55.234 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.122078688Z level=info msg="Executing migration" id="Add unique index dashboard_org_id_uid" 2024-02-13T14:16:55.234 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.122535563Z level=info msg="Executing migration" id="Remove unique index org_id_slug" 2024-02-13T14:16:55.234 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.122948583Z level=info msg="Executing migration" id="Update dashboard title length" 2024-02-13T14:16:55.234 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.123277411Z level=info msg="Executing migration" id="Add unique index for dashboard_org_id_title_folder_id" 2024-02-13T14:16:55.234 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.123768344Z level=info msg="Executing migration" id="create dashboard_provisioning" 2024-02-13T14:16:55.234 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.124191096Z level=info msg="Executing migration" id="Rename table dashboard_provisioning to dashboard_provisioning_tmp_qwerty - v1" 2024-02-13T14:16:55.234 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.125109336Z level=info msg="Executing migration" id="create dashboard_provisioning v2" 2024-02-13T14:16:55.234 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.126037982Z level=info msg="Executing migration" id="create index IDX_dashboard_provisioning_dashboard_id - v2" 2024-02-13T14:16:55.234 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.126508608Z level=info msg="Executing migration" id="create index IDX_dashboard_provisioning_dashboard_id_name - v2" 2024-02-13T14:16:55.234 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.127455998Z level=info msg="Executing migration" id="copy dashboard_provisioning v1 to v2" 2024-02-13T14:16:55.234 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.128188831Z level=info msg="Executing migration" id="drop dashboard_provisioning_tmp_qwerty" 2024-02-13T14:16:55.234 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.129064847Z level=info msg="Executing migration" id="Add check_sum column" 2024-02-13T14:16:55.234 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.129626582Z level=info msg="Executing migration" id="Add index for dashboard_title" 2024-02-13T14:16:55.234 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.130508059Z level=info msg="Executing migration" id="delete tags for deleted dashboards" 2024-02-13T14:16:55.234 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.131725766Z level=info msg="Executing migration" id="delete stars for deleted dashboards" 2024-02-13T14:16:55.235 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.13224196Z level=info msg="Executing migration" id="Add index for dashboard_is_folder" 2024-02-13T14:16:55.235 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.132711703Z level=info msg="Executing migration" id="Add isPublic for dashboard" 2024-02-13T14:16:55.235 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.133603349Z level=info msg="Executing migration" id="create data_source table" 2024-02-13T14:16:55.235 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.13459989Z level=info msg="Executing migration" id="add index data_source.account_id" 2024-02-13T14:16:55.235 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.135337026Z level=info msg="Executing migration" id="add unique index data_source.account_id_name" 2024-02-13T14:16:55.235 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.135802071Z level=info msg="Executing migration" id="drop index IDX_data_source_account_id - v1" 2024-02-13T14:16:55.235 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.136796954Z level=info msg="Executing migration" id="drop index UQE_data_source_account_id_name - v1" 2024-02-13T14:16:55.235 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.137559291Z level=info msg="Executing migration" id="Rename table data_source to data_source_v1 - v1" 2024-02-13T14:16:55.235 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.138657629Z level=info msg="Executing migration" id="create data_source table v2" 2024-02-13T14:16:55.235 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.13910353Z level=info msg="Executing migration" id="create index IDX_data_source_org_id - v2" 2024-02-13T14:16:55.235 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.140071904Z level=info msg="Executing migration" id="create index UQE_data_source_org_id_name - v2" 2024-02-13T14:16:55.235 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.140829525Z level=info msg="Executing migration" id="Drop old table data_source_v1 #2" 2024-02-13T14:16:55.235 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.14166207Z level=info msg="Executing migration" id="Add column with_credentials" 2024-02-13T14:16:55.235 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.142229421Z level=info msg="Executing migration" id="Add secure json data column" 2024-02-13T14:16:55.236 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.145088431Z level=info msg="Executing migration" id="Update data_source table charset" 2024-02-13T14:16:55.236 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.147845073Z level=info msg="Executing migration" id="Update initial version to 1" 2024-02-13T14:16:55.236 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.150744548Z level=info msg="Executing migration" id="Add read_only data column" 2024-02-13T14:16:55.236 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.151395844Z level=info msg="Executing migration" id="Migrate logging ds to loki ds" 2024-02-13T14:16:55.236 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.154259499Z level=info msg="Executing migration" id="Update json_data with nulls" 2024-02-13T14:16:55.236 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.156987156Z level=info msg="Executing migration" id="Add uid column" 2024-02-13T14:16:55.236 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.159848461Z level=info msg="Executing migration" id="Update uid value" 2024-02-13T14:16:55.236 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.160225272Z level=info msg="Executing migration" id="Add unique index datasource_org_id_uid" 2024-02-13T14:16:55.236 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.163293853Z level=info msg="Executing migration" id="add unique index datasource_org_id_is_default" 2024-02-13T14:16:55.236 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.165966262Z level=info msg="Executing migration" id="create api_key table" 2024-02-13T14:16:55.236 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.168826472Z level=info msg="Executing migration" id="add index api_key.account_id" 2024-02-13T14:16:55.236 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.169302922Z level=info msg="Executing migration" id="add index api_key.key" 2024-02-13T14:16:55.236 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.172030497Z level=info msg="Executing migration" id="add index api_key.account_id_name" 2024-02-13T14:16:55.236 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.174868504Z level=info msg="Executing migration" id="drop index IDX_api_key_account_id - v1" 2024-02-13T14:16:55.237 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.177692033Z level=info msg="Executing migration" id="drop index UQE_api_key_key - v1" 2024-02-13T14:16:55.237 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.18047748Z level=info msg="Executing migration" id="drop index UQE_api_key_account_id_name - v1" 2024-02-13T14:16:55.237 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.183345913Z level=info msg="Executing migration" id="Rename table api_key to api_key_v1 - v1" 2024-02-13T14:16:55.237 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.186478602Z level=info msg="Executing migration" id="create api_key table v2" 2024-02-13T14:16:55.237 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.189126179Z level=info msg="Executing migration" id="create index IDX_api_key_org_id - v2" 2024-02-13T14:16:55.237 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.191898062Z level=info msg="Executing migration" id="create index UQE_api_key_key - v2" 2024-02-13T14:16:55.237 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.194548706Z level=info msg="Executing migration" id="create index UQE_api_key_org_id_name - v2" 2024-02-13T14:16:55.237 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.19737802Z level=info msg="Executing migration" id="copy api_key v1 to v2" 2024-02-13T14:16:55.237 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.200177791Z level=info msg="Executing migration" id="Drop old table api_key_v1" 2024-02-13T14:16:55.237 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.202766979Z level=info msg="Executing migration" id="Update api_key table charset" 2024-02-13T14:16:55.237 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.205357293Z level=info msg="Executing migration" id="Add expires to api_key table" 2024-02-13T14:16:55.237 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.208131732Z level=info msg="Executing migration" id="Add service account foreign key" 2024-02-13T14:16:55.237 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.210792522Z level=info msg="Executing migration" id="set service account foreign key to nil if 0" 2024-02-13T14:16:55.237 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.213522435Z level=info msg="Executing migration" id="Add last_used_at to api_key table" 2024-02-13T14:16:55.238 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.21604546Z level=info msg="Executing migration" id="Add is_revoked column to api_key table" 2024-02-13T14:16:55.238 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.218833535Z level=info msg="Executing migration" id="create dashboard_snapshot table v4" 2024-02-13T14:16:55.238 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.221258934Z level=info msg="Executing migration" id="drop table dashboard_snapshot_v4 #1" 2024-02-13T14:16:55.238 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.223891339Z level=info msg="Executing migration" id="create dashboard_snapshot table v5 #2" 2024-02-13T14:16:55.238 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.22645067Z level=info msg="Executing migration" id="create index UQE_dashboard_snapshot_key - v5" 2024-02-13T14:16:55.241 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:16:54 smithi087 ceph-mon[27372]: pgmap v33: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-13T14:16:55.241 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:16:54 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-13T14:16:55.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:16:54 smithi087 ceph-mon[39120]: pgmap v33: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-13T14:16:55.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:16:54 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-13T14:16:55.476 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.229199714Z level=info msg="Executing migration" id="create index UQE_dashboard_snapshot_delete_key - v5" 2024-02-13T14:16:55.477 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.231848127Z level=info msg="Executing migration" id="create index IDX_dashboard_snapshot_user_id - v5" 2024-02-13T14:16:55.477 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.23267663Z level=info msg="Executing migration" id="alter dashboard_snapshot to mediumtext v2" 2024-02-13T14:16:55.477 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.233498437Z level=info msg="Executing migration" id="Update dashboard_snapshot table charset" 2024-02-13T14:16:55.477 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.234393745Z level=info msg="Executing migration" id="Add column external_delete_url to dashboard_snapshots table" 2024-02-13T14:16:55.477 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.235212154Z level=info msg="Executing migration" id="Add encrypted dashboard json column" 2024-02-13T14:16:55.477 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.236030103Z level=info msg="Executing migration" id="Change dashboard_encrypted column to MEDIUMBLOB" 2024-02-13T14:16:55.477 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.236996547Z level=info msg="Executing migration" id="create quota table v1" 2024-02-13T14:16:55.477 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.237755005Z level=info msg="Executing migration" id="create index UQE_quota_org_id_user_id_target - v1" 2024-02-13T14:16:55.477 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.238584531Z level=info msg="Executing migration" id="Update quota table charset" 2024-02-13T14:16:55.477 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.23953211Z level=info msg="Executing migration" id="create plugin_setting table" 2024-02-13T14:16:55.477 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.240240016Z level=info msg="Executing migration" id="create index UQE_plugin_setting_org_id_plugin_id - v1" 2024-02-13T14:16:55.478 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.241219696Z level=info msg="Executing migration" id="Add column plugin_version to plugin_settings" 2024-02-13T14:16:55.478 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.241944008Z level=info msg="Executing migration" id="Update plugin_setting table charset" 2024-02-13T14:16:55.478 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.24276229Z level=info msg="Executing migration" id="create session table" 2024-02-13T14:16:55.478 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.243723472Z level=info msg="Executing migration" id="Drop old table playlist table" 2024-02-13T14:16:55.478 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.244473566Z level=info msg="Executing migration" id="Drop old table playlist_item table" 2024-02-13T14:16:55.478 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.245443424Z level=info msg="Executing migration" id="create playlist table v2" 2024-02-13T14:16:55.478 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.246133892Z level=info msg="Executing migration" id="create playlist item table v2" 2024-02-13T14:16:55.478 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.247135343Z level=info msg="Executing migration" id="Update playlist table charset" 2024-02-13T14:16:55.478 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.247958401Z level=info msg="Executing migration" id="Update playlist_item table charset" 2024-02-13T14:16:55.478 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.248760458Z level=info msg="Executing migration" id="drop preferences table v2" 2024-02-13T14:16:55.478 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.249692348Z level=info msg="Executing migration" id="drop preferences table v3" 2024-02-13T14:16:55.478 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.250199031Z level=info msg="Executing migration" id="create preferences table v3" 2024-02-13T14:16:55.478 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.250603926Z level=info msg="Executing migration" id="Update preferences table charset" 2024-02-13T14:16:55.479 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.251448334Z level=info msg="Executing migration" id="Add column team_id in preferences" 2024-02-13T14:16:55.479 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.252075293Z level=info msg="Executing migration" id="Update team_id column values in preferences" 2024-02-13T14:16:55.479 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.253115116Z level=info msg="Executing migration" id="Add column week_start in preferences" 2024-02-13T14:16:55.479 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.253721663Z level=info msg="Executing migration" id="Add column preferences.json_data" 2024-02-13T14:16:55.479 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.256368927Z level=info msg="Executing migration" id="alter preferences.json_data to mediumtext v1" 2024-02-13T14:16:55.479 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.258822362Z level=info msg="Executing migration" id="Add preferences index org_id" 2024-02-13T14:16:55.479 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.259335718Z level=info msg="Executing migration" id="Add preferences index user_id" 2024-02-13T14:16:55.479 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.262100217Z level=info msg="Executing migration" id="create alert table v1" 2024-02-13T14:16:55.479 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.264815443Z level=info msg="Executing migration" id="add index alert org_id & id " 2024-02-13T14:16:55.479 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.267262354Z level=info msg="Executing migration" id="add index alert state" 2024-02-13T14:16:55.479 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.267730383Z level=info msg="Executing migration" id="add index alert dashboard_id" 2024-02-13T14:16:55.479 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.26817606Z level=info msg="Executing migration" id="Create alert_rule_tag table v1" 2024-02-13T14:16:55.480 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.26857342Z level=info msg="Executing migration" id="Add unique index alert_rule_tag.alert_id_tag_id" 2024-02-13T14:16:55.481 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.269012962Z level=info msg="Executing migration" id="drop index UQE_alert_rule_tag_alert_id_tag_id - v1" 2024-02-13T14:16:55.481 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.269445362Z level=info msg="Executing migration" id="Rename table alert_rule_tag to alert_rule_tag_v1 - v1" 2024-02-13T14:16:55.481 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.270855719Z level=info msg="Executing migration" id="Create alert_rule_tag table v2" 2024-02-13T14:16:55.481 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.271276187Z level=info msg="Executing migration" id="create index UQE_alert_rule_tag_alert_id_tag_id - Add unique index alert_rule_tag.alert_id_tag_id V2" 2024-02-13T14:16:55.481 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.27172213Z level=info msg="Executing migration" id="copy alert_rule_tag v1 to v2" 2024-02-13T14:16:55.481 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.272044867Z level=info msg="Executing migration" id="drop table alert_rule_tag_v1" 2024-02-13T14:16:55.482 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.272429633Z level=info msg="Executing migration" id="create alert_notification table v1" 2024-02-13T14:16:55.482 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.272812267Z level=info msg="Executing migration" id="Add column is_default" 2024-02-13T14:16:55.482 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.273529308Z level=info msg="Executing migration" id="Add column frequency" 2024-02-13T14:16:55.482 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.274175548Z level=info msg="Executing migration" id="Add column send_reminder" 2024-02-13T14:16:55.482 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.274831552Z level=info msg="Executing migration" id="Add column disable_resolve_message" 2024-02-13T14:16:55.482 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.275521687Z level=info msg="Executing migration" id="add index alert_notification org_id & name" 2024-02-13T14:16:55.482 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.275989203Z level=info msg="Executing migration" id="Update alert table charset" 2024-02-13T14:16:55.482 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.27631441Z level=info msg="Executing migration" id="Update alert_notification table charset" 2024-02-13T14:16:55.482 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.276605229Z level=info msg="Executing migration" id="create notification_journal table v1" 2024-02-13T14:16:55.482 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.277048873Z level=info msg="Executing migration" id="add index notification_journal org_id & alert_id & notifier_id" 2024-02-13T14:16:55.482 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.277533391Z level=info msg="Executing migration" id="drop alert_notification_journal" 2024-02-13T14:16:55.482 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.277906322Z level=info msg="Executing migration" id="create alert_notification_state table v1" 2024-02-13T14:16:55.482 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.27834337Z level=info msg="Executing migration" id="add index alert_notification_state org_id & alert_id & notifier_id" 2024-02-13T14:16:55.482 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.27882159Z level=info msg="Executing migration" id="Add for to alert table" 2024-02-13T14:16:55.483 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.279462475Z level=info msg="Executing migration" id="Add column uid in alert_notification" 2024-02-13T14:16:55.483 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.280098357Z level=info msg="Executing migration" id="Update uid column values in alert_notification" 2024-02-13T14:16:55.483 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.280478196Z level=info msg="Executing migration" id="Add unique index alert_notification_org_id_uid" 2024-02-13T14:16:55.483 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.280946486Z level=info msg="Executing migration" id="Remove unique index org_id_name" 2024-02-13T14:16:55.483 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.281426209Z level=info msg="Executing migration" id="Add column secure_settings in alert_notification" 2024-02-13T14:16:55.483 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.282111455Z level=info msg="Executing migration" id="alter alert.settings to mediumtext" 2024-02-13T14:16:55.483 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.282428758Z level=info msg="Executing migration" id="Add non-unique index alert_notification_state_alert_id" 2024-02-13T14:16:55.483 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.282838369Z level=info msg="Executing migration" id="Add non-unique index alert_rule_tag_alert_id" 2024-02-13T14:16:55.483 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.28327028Z level=info msg="Executing migration" id="Drop old annotation table v4" 2024-02-13T14:16:55.483 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.283587923Z level=info msg="Executing migration" id="create annotation table v5" 2024-02-13T14:16:55.483 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.284027015Z level=info msg="Executing migration" id="add index annotation 0 v3" 2024-02-13T14:16:55.484 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.28673067Z level=info msg="Executing migration" id="add index annotation 1 v3" 2024-02-13T14:16:55.484 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.287187222Z level=info msg="Executing migration" id="add index annotation 2 v3" 2024-02-13T14:16:55.484 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.287651325Z level=info msg="Executing migration" id="add index annotation 3 v3" 2024-02-13T14:16:55.484 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.28814331Z level=info msg="Executing migration" id="add index annotation 4 v3" 2024-02-13T14:16:55.484 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.290779224Z level=info msg="Executing migration" id="Update annotation table charset" 2024-02-13T14:16:55.484 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.293419284Z level=info msg="Executing migration" id="Add column region_id to annotation table" 2024-02-13T14:16:55.484 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.294177317Z level=info msg="Executing migration" id="Drop category_id index" 2024-02-13T14:16:55.484 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.29694652Z level=info msg="Executing migration" id="Add column tags to annotation table" 2024-02-13T14:16:55.484 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.299568158Z level=info msg="Executing migration" id="Create annotation_tag table v2" 2024-02-13T14:16:55.484 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.300288138Z level=info msg="Executing migration" id="Add unique index annotation_tag.annotation_id_tag_id" 2024-02-13T14:16:55.484 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.301041923Z level=info msg="Executing migration" id="drop index UQE_annotation_tag_annotation_id_tag_id - v2" 2024-02-13T14:16:55.484 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.301852156Z level=info msg="Executing migration" id="Rename table annotation_tag to annotation_tag_v2 - v2" 2024-02-13T14:16:55.484 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.303433917Z level=info msg="Executing migration" id="Create annotation_tag table v3" 2024-02-13T14:16:55.485 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.304354214Z level=info msg="Executing migration" id="create index UQE_annotation_tag_annotation_id_tag_id - Add unique index annotation_tag.annotation_id_tag_id V3" 2024-02-13T14:16:55.485 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.305105212Z level=info msg="Executing migration" id="copy annotation_tag v2 to v3" 2024-02-13T14:16:55.485 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.305880643Z level=info msg="Executing migration" id="drop table annotation_tag_v2" 2024-02-13T14:16:55.485 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.306751635Z level=info msg="Executing migration" id="Update alert annotations and set TEXT to empty" 2024-02-13T14:16:55.485 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.307591325Z level=info msg="Executing migration" id="Add created time to annotation table" 2024-02-13T14:16:55.485 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.308393317Z level=info msg="Executing migration" id="Add updated time to annotation table" 2024-02-13T14:16:55.485 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.309240384Z level=info msg="Executing migration" id="Add index for created in annotation table" 2024-02-13T14:16:55.485 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.310066541Z level=info msg="Executing migration" id="Add index for updated in annotation table" 2024-02-13T14:16:55.485 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.31104495Z level=info msg="Executing migration" id="Convert existing annotations from seconds to milliseconds" 2024-02-13T14:16:55.485 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.311670744Z level=info msg="Executing migration" id="Add epoch_end column" 2024-02-13T14:16:55.485 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.312436895Z level=info msg="Executing migration" id="Add index for epoch_end" 2024-02-13T14:16:55.485 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.313258087Z level=info msg="Executing migration" id="Make epoch_end the same as epoch" 2024-02-13T14:16:55.485 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.31415883Z level=info msg="Executing migration" id="Move region to single row" 2024-02-13T14:16:55.486 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.314997861Z level=info msg="Executing migration" id="Remove index org_id_epoch from annotation table" 2024-02-13T14:16:55.486 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.315690621Z level=info msg="Executing migration" id="Remove index org_id_dashboard_id_panel_id_epoch from annotation table" 2024-02-13T14:16:55.486 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.31642896Z level=info msg="Executing migration" id="Add index for org_id_dashboard_id_epoch_end_epoch on annotation table" 2024-02-13T14:16:55.486 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.316946027Z level=info msg="Executing migration" id="Add index for org_id_epoch_end_epoch on annotation table" 2024-02-13T14:16:55.486 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.317447959Z level=info msg="Executing migration" id="Remove index org_id_epoch_epoch_end from annotation table" 2024-02-13T14:16:55.486 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.318251286Z level=info msg="Executing migration" id="Add index for alert_id on annotation table" 2024-02-13T14:16:55.486 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.319178747Z level=info msg="Executing migration" id="Increase tags column to length 4096" 2024-02-13T14:16:55.486 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.319851069Z level=info msg="Executing migration" id="create test_data table" 2024-02-13T14:16:55.486 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.32070709Z level=info msg="Executing migration" id="create dashboard_version table v1" 2024-02-13T14:16:55.486 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.321238415Z level=info msg="Executing migration" id="add index dashboard_version.dashboard_id" 2024-02-13T14:16:55.487 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.322245732Z level=info msg="Executing migration" id="add unique index dashboard_version.dashboard_id and dashboard_version.version" 2024-02-13T14:16:55.487 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.322960266Z level=info msg="Executing migration" id="Set dashboard version to 1 where 0" 2024-02-13T14:16:55.487 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.323901401Z level=info msg="Executing migration" id="save existing dashboard data in dashboard_version table v1" 2024-02-13T14:16:55.487 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.324597739Z level=info msg="Executing migration" id="alter dashboard_version.data to mediumtext v1" 2024-02-13T14:16:55.487 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.325519849Z level=info msg="Executing migration" id="create team table" 2024-02-13T14:16:55.487 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.326211103Z level=info msg="Executing migration" id="add index team.org_id" 2024-02-13T14:16:55.487 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.326688663Z level=info msg="Executing migration" id="add unique index team_org_id_name" 2024-02-13T14:16:55.487 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.327512309Z level=info msg="Executing migration" id="create team member table" 2024-02-13T14:16:55.487 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.328347365Z level=info msg="Executing migration" id="add index team_member.org_id" 2024-02-13T14:16:55.487 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.33131826Z level=info msg="Executing migration" id="add unique index team_member_org_id_team_id_user_id" 2024-02-13T14:16:55.487 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.333782355Z level=info msg="Executing migration" id="add index team_member.team_id" 2024-02-13T14:16:55.487 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.336552482Z level=info msg="Executing migration" id="Add column email to team table" 2024-02-13T14:16:55.487 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.339307884Z level=info msg="Executing migration" id="Add column external to team_member table" 2024-02-13T14:16:55.487 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.342181885Z level=info msg="Executing migration" id="Add column permission to team_member table" 2024-02-13T14:16:55.488 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.344861649Z level=info msg="Executing migration" id="create dashboard acl table" 2024-02-13T14:16:55.488 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.347647187Z level=info msg="Executing migration" id="add index dashboard_acl_dashboard_id" 2024-02-13T14:16:55.488 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.350376186Z level=info msg="Executing migration" id="add unique index dashboard_acl_dashboard_id_user_id" 2024-02-13T14:16:55.488 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.353004015Z level=info msg="Executing migration" id="add unique index dashboard_acl_dashboard_id_team_id" 2024-02-13T14:16:55.488 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.353525429Z level=info msg="Executing migration" id="add index dashboard_acl_user_id" 2024-02-13T14:16:55.488 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.356282498Z level=info msg="Executing migration" id="add index dashboard_acl_team_id" 2024-02-13T14:16:55.488 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.358997244Z level=info msg="Executing migration" id="add index dashboard_acl_org_id_role" 2024-02-13T14:16:55.488 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.36163022Z level=info msg="Executing migration" id="add index dashboard_permission" 2024-02-13T14:16:55.488 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.364424318Z level=info msg="Executing migration" id="save default acl rules in dashboard_acl table" 2024-02-13T14:16:55.488 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.367138634Z level=info msg="Executing migration" id="delete acl rules for deleted dashboards and folders" 2024-02-13T14:16:55.488 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.367663536Z level=info msg="Executing migration" id="create tag table" 2024-02-13T14:16:55.488 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.370287024Z level=info msg="Executing migration" id="add index tag.key_value" 2024-02-13T14:16:55.488 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.373110966Z level=info msg="Executing migration" id="create login attempt table" 2024-02-13T14:16:55.489 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.375772733Z level=info msg="Executing migration" id="add index login_attempt.username" 2024-02-13T14:16:55.489 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.376256141Z level=info msg="Executing migration" id="drop index IDX_login_attempt_username - v1" 2024-02-13T14:16:55.489 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.378825654Z level=info msg="Executing migration" id="Rename table login_attempt to login_attempt_tmp_qwerty - v1" 2024-02-13T14:16:55.489 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.380749649Z level=info msg="Executing migration" id="create login_attempt v2" 2024-02-13T14:16:55.489 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.381687087Z level=info msg="Executing migration" id="create index IDX_login_attempt_username - v2" 2024-02-13T14:16:55.489 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.384309401Z level=info msg="Executing migration" id="copy login_attempt v1 to v2" 2024-02-13T14:16:55.489 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.387009774Z level=info msg="Executing migration" id="drop login_attempt_tmp_qwerty" 2024-02-13T14:16:55.489 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.389521476Z level=info msg="Executing migration" id="create user auth table" 2024-02-13T14:16:55.489 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.392216867Z level=info msg="Executing migration" id="create index IDX_user_auth_auth_module_auth_id - v1" 2024-02-13T14:16:55.489 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.394865251Z level=info msg="Executing migration" id="alter user_auth.auth_id to length 190" 2024-02-13T14:16:55.489 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.395402699Z level=info msg="Executing migration" id="Add OAuth access token to user_auth" 2024-02-13T14:16:55.489 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.398009326Z level=info msg="Executing migration" id="Add OAuth refresh token to user_auth" 2024-02-13T14:16:55.489 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.400523438Z level=info msg="Executing migration" id="Add OAuth token type to user_auth" 2024-02-13T14:16:55.489 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.403323677Z level=info msg="Executing migration" id="Add OAuth expiry to user_auth" 2024-02-13T14:16:55.489 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.404135995Z level=info msg="Executing migration" id="Add index to user_id column in user_auth" 2024-02-13T14:16:55.490 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.406813049Z level=info msg="Executing migration" id="Add OAuth ID token to user_auth" 2024-02-13T14:16:55.490 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.409274614Z level=info msg="Executing migration" id="create server_lock table" 2024-02-13T14:16:55.490 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.411971781Z level=info msg="Executing migration" id="add index server_lock.operation_uid" 2024-02-13T14:16:55.490 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.412489087Z level=info msg="Executing migration" id="create user auth token table" 2024-02-13T14:16:55.490 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.41323331Z level=info msg="Executing migration" id="add unique index user_auth_token.auth_token" 2024-02-13T14:16:55.490 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.414092442Z level=info msg="Executing migration" id="add unique index user_auth_token.prev_auth_token" 2024-02-13T14:16:55.490 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.415003662Z level=info msg="Executing migration" id="add index user_auth_token.user_id" 2024-02-13T14:16:55.490 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.415599338Z level=info msg="Executing migration" id="Add revoked_at to the user auth token" 2024-02-13T14:16:55.490 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.416451278Z level=info msg="Executing migration" id="create cache_data table" 2024-02-13T14:16:55.490 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.417207613Z level=info msg="Executing migration" id="add unique index cache_data.cache_key" 2024-02-13T14:16:55.490 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.418174393Z level=info msg="Executing migration" id="create short_url table v1" 2024-02-13T14:16:55.490 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.41868253Z level=info msg="Executing migration" id="add index short_url.org_id-uid" 2024-02-13T14:16:55.490 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.419437269Z level=info msg="Executing migration" id="alter table short_url alter column created_by type to bigint" 2024-02-13T14:16:55.490 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.420161999Z level=info msg="Executing migration" id="delete alert_definition table" 2024-02-13T14:16:55.491 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.420551706Z level=info msg="Executing migration" id="recreate alert_definition table" 2024-02-13T14:16:55.491 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.42102865Z level=info msg="Executing migration" id="add index in alert_definition on org_id and title columns" 2024-02-13T14:16:55.491 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.421808178Z level=info msg="Executing migration" id="add index in alert_definition on org_id and uid columns" 2024-02-13T14:16:55.491 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.422570185Z level=info msg="Executing migration" id="alter alert_definition table data column to mediumtext in mysql" 2024-02-13T14:16:55.491 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.422901188Z level=info msg="Executing migration" id="drop index in alert_definition on org_id and title columns" 2024-02-13T14:16:55.491 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.423384363Z level=info msg="Executing migration" id="drop index in alert_definition on org_id and uid columns" 2024-02-13T14:16:55.491 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.424228585Z level=info msg="Executing migration" id="add unique index in alert_definition on org_id and title columns" 2024-02-13T14:16:55.491 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.425083854Z level=info msg="Executing migration" id="add unique index in alert_definition on org_id and uid columns" 2024-02-13T14:16:55.491 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.425990165Z level=info msg="Executing migration" id="Add column paused in alert_definition" 2024-02-13T14:16:55.491 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.426865034Z level=info msg="Executing migration" id="drop alert_definition table" 2024-02-13T14:16:55.491 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.427843201Z level=info msg="Executing migration" id="delete alert_definition_version table" 2024-02-13T14:16:55.491 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.428591627Z level=info msg="Executing migration" id="recreate alert_definition_version table" 2024-02-13T14:16:55.491 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.429561052Z level=info msg="Executing migration" id="add index in alert_definition_version table on alert_definition_id and version columns" 2024-02-13T14:16:55.491 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.430115435Z level=info msg="Executing migration" id="add index in alert_definition_version table on alert_definition_uid and version columns" 2024-02-13T14:16:55.492 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.431105984Z level=info msg="Executing migration" id="alter alert_definition_version table data column to mediumtext in mysql" 2024-02-13T14:16:55.492 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.431845904Z level=info msg="Executing migration" id="drop alert_definition_version table" 2024-02-13T14:16:55.492 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.434834513Z level=info msg="Executing migration" id="create alert_instance table" 2024-02-13T14:16:55.492 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.435402934Z level=info msg="Executing migration" id="add index in alert_instance table on def_org_id, def_uid and current_state columns" 2024-02-13T14:16:55.492 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.438412976Z level=info msg="Executing migration" id="add index in alert_instance table on def_org_id, current_state columns" 2024-02-13T14:16:55.492 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.439084362Z level=info msg="Executing migration" id="add column current_state_end to alert_instance" 2024-02-13T14:16:55.492 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.441318958Z level=info msg="Executing migration" id="remove index def_org_id, def_uid, current_state on alert_instance" 2024-02-13T14:16:55.492 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.444122827Z level=info msg="Executing migration" id="remove index def_org_id, current_state on alert_instance" 2024-02-13T14:16:55.492 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.444597644Z level=info msg="Executing migration" id="rename def_org_id to rule_org_id in alert_instance" 2024-02-13T14:16:55.492 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.447976551Z level=info msg="Executing migration" id="rename def_uid to rule_uid in alert_instance" 2024-02-13T14:16:55.492 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.451426666Z level=info msg="Executing migration" id="add index rule_org_id, rule_uid, current_state on alert_instance" 2024-02-13T14:16:55.492 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.45424628Z level=info msg="Executing migration" id="add index rule_org_id, current_state on alert_instance" 2024-02-13T14:16:55.492 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.454740211Z level=info msg="Executing migration" id="add current_reason column related to current_state" 2024-02-13T14:16:55.492 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.457646521Z level=info msg="Executing migration" id="create alert_rule table" 2024-02-13T14:16:55.493 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.460377321Z level=info msg="Executing migration" id="add index in alert_rule on org_id and title columns" 2024-02-13T14:16:55.493 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.463075536Z level=info msg="Executing migration" id="add index in alert_rule on org_id and uid columns" 2024-02-13T14:16:55.493 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.463543431Z level=info msg="Executing migration" id="add index in alert_rule on org_id, namespace_uid, group_uid columns" 2024-02-13T14:16:55.493 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.466368631Z level=info msg="Executing migration" id="alter alert_rule table data column to mediumtext in mysql" 2024-02-13T14:16:55.493 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.469142825Z level=info msg="Executing migration" id="add column for to alert_rule" 2024-02-13T14:16:55.493 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.471869255Z level=info msg="Executing migration" id="add column annotations to alert_rule" 2024-02-13T14:16:55.493 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.472733353Z level=info msg="Executing migration" id="add column labels to alert_rule" 2024-02-13T14:16:55.493 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.475579742Z level=info msg="Executing migration" id="remove unique index from alert_rule on org_id, title columns" 2024-02-13T14:16:55.493 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.478260548Z level=info msg="Executing migration" id="add index in alert_rule on org_id, namespase_uid and title columns" 2024-02-13T14:16:55.730 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.481032475Z level=info msg="Executing migration" id="add dashboard_uid column to alert_rule" 2024-02-13T14:16:55.730 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.481840231Z level=info msg="Executing migration" id="add panel_id column to alert_rule" 2024-02-13T14:16:55.730 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.48451964Z level=info msg="Executing migration" id="add index in alert_rule on org_id, dashboard_uid and panel_id columns" 2024-02-13T14:16:55.730 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.487198519Z level=info msg="Executing migration" id="add rule_group_idx column to alert_rule" 2024-02-13T14:16:55.730 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.49000187Z level=info msg="Executing migration" id="add is_paused column to alert_rule table" 2024-02-13T14:16:55.730 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.490815538Z level=info msg="Executing migration" id="fix is_paused column for alert_rule table" 2024-02-13T14:16:55.731 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.493692845Z level=info msg="Executing migration" id="create alert_rule_version table" 2024-02-13T14:16:55.731 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.494369921Z level=info msg="Executing migration" id="add index in alert_rule_version table on rule_org_id, rule_uid and version columns" 2024-02-13T14:16:55.731 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.496493532Z level=info msg="Executing migration" id="add index in alert_rule_version table on rule_org_id, rule_namespace_uid and rule_group columns" 2024-02-13T14:16:55.731 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.499010435Z level=info msg="Executing migration" id="alter alert_rule_version table data column to mediumtext in mysql" 2024-02-13T14:16:55.731 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.499318155Z level=info msg="Executing migration" id="add column for to alert_rule_version" 2024-02-13T14:16:55.732 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.501865679Z level=info msg="Executing migration" id="add column annotations to alert_rule_version" 2024-02-13T14:16:55.732 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.504721317Z level=info msg="Executing migration" id="add column labels to alert_rule_version" 2024-02-13T14:16:55.732 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.507123831Z level=info msg="Executing migration" id="add rule_group_idx column to alert_rule_version" 2024-02-13T14:16:55.732 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.507937277Z level=info msg="Executing migration" id="add is_paused column to alert_rule_versions table" 2024-02-13T14:16:55.732 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.508785576Z level=info msg="Executing migration" id="fix is_paused column for alert_rule_version table" 2024-02-13T14:16:55.732 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.509673993Z level=info msg="Executing migration" id=create_alert_configuration_table 2024-02-13T14:16:55.732 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.510214121Z level=info msg="Executing migration" id="Add column default in alert_configuration" 2024-02-13T14:16:55.733 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.511046288Z level=info msg="Executing migration" id="alert alert_configuration alertmanager_configuration column from TEXT to MEDIUMTEXT if mysql" 2024-02-13T14:16:55.733 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.511343005Z level=info msg="Executing migration" id="add column org_id in alert_configuration" 2024-02-13T14:16:55.733 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.512216518Z level=info msg="Executing migration" id="add index in alert_configuration table on org_id column" 2024-02-13T14:16:55.733 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.513003061Z level=info msg="Executing migration" id="add configuration_hash column to alert_configuration" 2024-02-13T14:16:55.733 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.513856208Z level=info msg="Executing migration" id=create_ngalert_configuration_table 2024-02-13T14:16:55.733 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.51426506Z level=info msg="Executing migration" id="add index in ngalert_configuration on org_id column" 2024-02-13T14:16:55.734 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.515243461Z level=info msg="Executing migration" id="add column send_alerts_to in ngalert_configuration" 2024-02-13T14:16:55.734 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.516223668Z level=info msg="Executing migration" id="create provenance_type table" 2024-02-13T14:16:55.734 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.51703435Z level=info msg="Executing migration" id="add index to uniquify (record_key, record_type, org_id) columns" 2024-02-13T14:16:55.734 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.517478734Z level=info msg="Executing migration" id="create alert_image table" 2024-02-13T14:16:55.734 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.518351639Z level=info msg="Executing migration" id="add unique index on token to alert_image table" 2024-02-13T14:16:55.734 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.519177835Z level=info msg="Executing migration" id="support longer URLs in alert_image table" 2024-02-13T14:16:55.734 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.520227425Z level=info msg="Executing migration" id=create_alert_configuration_history_table 2024-02-13T14:16:55.734 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.520658789Z level=info msg="Executing migration" id="drop non-unique orgID index on alert_configuration" 2024-02-13T14:16:55.734 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.521495318Z level=info msg="Executing migration" id="drop unique orgID index on alert_configuration if exists" 2024-02-13T14:16:55.734 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.521591889Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop unique orgID index on alert_configuration if exists" 2024-02-13T14:16:55.734 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.521879503Z level=info msg="Executing migration" id="extract alertmanager configuration history to separate table" 2024-02-13T14:16:55.734 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.52239713Z level=info msg="Executing migration" id="add unique index on orgID to alert_configuration" 2024-02-13T14:16:55.735 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.523280111Z level=info msg="Executing migration" id="clear migration entry \"remove unified alerting data\"" 2024-02-13T14:16:55.735 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.523402538Z level=info msg="Executing migration" id="move dashboard alerts to unified alerting" 2024-02-13T14:16:55.735 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.523517446Z level=info msg="alerts found to migrate" alerts=0 2024-02-13T14:16:55.735 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.524117169Z level=info msg="Executing migration" id="create library_element table v1" 2024-02-13T14:16:55.735 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.525157011Z level=info msg="Executing migration" id="add index library_element org_id-folder_id-name-kind" 2024-02-13T14:16:55.735 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.52587971Z level=info msg="Executing migration" id="create library_element_connection table v1" 2024-02-13T14:16:55.735 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.526743384Z level=info msg="Executing migration" id="add index library_element_connection element_id-kind-connection_id" 2024-02-13T14:16:55.736 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.527213763Z level=info msg="Executing migration" id="add unique index library_element org_id_uid" 2024-02-13T14:16:55.736 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.52812083Z level=info msg="Executing migration" id="increase max description length to 2048" 2024-02-13T14:16:55.736 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.528926052Z level=info msg="Executing migration" id="clone move dashboard alerts to unified alerting" 2024-02-13T14:16:55.736 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.529774077Z level=info msg="Executing migration" id="create data_keys table" 2024-02-13T14:16:55.736 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.530223531Z level=info msg="Executing migration" id="create secrets table" 2024-02-13T14:16:55.736 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.531073978Z level=info msg="Executing migration" id="rename data_keys name column to id" 2024-02-13T14:16:55.736 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.535322861Z level=info msg="Executing migration" id="add name column into data_keys" 2024-02-13T14:16:55.736 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.536235272Z level=info msg="Executing migration" id="copy data_keys id column values into name" 2024-02-13T14:16:55.736 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.536550059Z level=info msg="Executing migration" id="rename data_keys name column to label" 2024-02-13T14:16:55.736 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.540814613Z level=info msg="Executing migration" id="rename data_keys id column back to name" 2024-02-13T14:16:55.736 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.545091972Z level=info msg="Executing migration" id="create kv_store table v1" 2024-02-13T14:16:55.737 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.548089253Z level=info msg="Executing migration" id="add index kv_store.org_id-namespace-key" 2024-02-13T14:16:55.737 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.548615953Z level=info msg="Executing migration" id="update dashboard_uid and panel_id from existing annotations" 2024-02-13T14:16:55.737 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.551433844Z level=info msg="Executing migration" id="create permission table" 2024-02-13T14:16:55.737 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.551892165Z level=info msg="Executing migration" id="add unique index permission.role_id" 2024-02-13T14:16:55.737 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.554229633Z level=info msg="Executing migration" id="add unique index role_id_action_scope" 2024-02-13T14:16:55.737 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.557196119Z level=info msg="Executing migration" id="create role table" 2024-02-13T14:16:55.737 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.557662635Z level=info msg="Executing migration" id="add column display_name" 2024-02-13T14:16:55.737 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.560288422Z level=info msg="Executing migration" id="add column group_name" 2024-02-13T14:16:55.737 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.561270429Z level=info msg="Executing migration" id="add index role.org_id" 2024-02-13T14:16:55.737 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.563190397Z level=info msg="Executing migration" id="add unique index role_org_id_name" 2024-02-13T14:16:55.737 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.565856442Z level=info msg="Executing migration" id="add index role_org_id_uid" 2024-02-13T14:16:55.738 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.566349549Z level=info msg="Executing migration" id="create team role table" 2024-02-13T14:16:55.738 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.569253701Z level=info msg="Executing migration" id="add index team_role.org_id" 2024-02-13T14:16:55.738 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.572038566Z level=info msg="Executing migration" id="add unique index team_role_org_id_team_id_role_id" 2024-02-13T14:16:55.738 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.574877004Z level=info msg="Executing migration" id="add index team_role.team_id" 2024-02-13T14:16:55.738 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.575410151Z level=info msg="Executing migration" id="create user role table" 2024-02-13T14:16:55.738 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.578118912Z level=info msg="Executing migration" id="add index user_role.org_id" 2024-02-13T14:16:55.738 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.581048576Z level=info msg="Executing migration" id="add unique index user_role_org_id_user_id_role_id" 2024-02-13T14:16:55.738 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.583769915Z level=info msg="Executing migration" id="add index user_role.user_id" 2024-02-13T14:16:55.738 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.584419758Z level=info msg="Executing migration" id="create builtin role table" 2024-02-13T14:16:55.738 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.58721305Z level=info msg="Executing migration" id="add index builtin_role.role_id" 2024-02-13T14:16:55.738 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.589952119Z level=info msg="Executing migration" id="add index builtin_role.name" 2024-02-13T14:16:55.738 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.592627196Z level=info msg="Executing migration" id="Add column org_id to builtin_role table" 2024-02-13T14:16:55.738 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.593787206Z level=info msg="Executing migration" id="add index builtin_role.org_id" 2024-02-13T14:16:55.738 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.596597257Z level=info msg="Executing migration" id="add unique index builtin_role_org_id_role_id_role" 2024-02-13T14:16:55.738 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.5992775Z level=info msg="Executing migration" id="Remove unique index role_org_id_uid" 2024-02-13T14:16:55.739 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.602057148Z level=info msg="Executing migration" id="add unique index role.uid" 2024-02-13T14:16:55.739 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.602687373Z level=info msg="Executing migration" id="create seed assignment table" 2024-02-13T14:16:55.739 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.605362015Z level=info msg="Executing migration" id="add unique index builtin_role_role_name" 2024-02-13T14:16:55.739 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.608120788Z level=info msg="Executing migration" id="add column hidden to role table" 2024-02-13T14:16:55.739 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.610678011Z level=info msg="Executing migration" id="create query_history table v1" 2024-02-13T14:16:55.739 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.611112028Z level=info msg="Executing migration" id="add index query_history.org_id-created_by-datasource_uid" 2024-02-13T14:16:55.739 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.613864008Z level=info msg="Executing migration" id="alter table query_history alter column created_by type to bigint" 2024-02-13T14:16:55.739 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.61467734Z level=info msg="Executing migration" id="teams permissions migration" 2024-02-13T14:16:55.739 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.615513157Z level=info msg="Executing migration" id="dashboard permissions" 2024-02-13T14:16:55.739 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.616002259Z level=info msg="Executing migration" id="dashboard permissions uid scopes" 2024-02-13T14:16:55.739 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.616995075Z level=info msg="Executing migration" id="drop managed folder create actions" 2024-02-13T14:16:55.739 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.617690621Z level=info msg="Executing migration" id="alerting notification permissions" 2024-02-13T14:16:55.739 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.618599556Z level=info msg="Executing migration" id="create query_history_star table v1" 2024-02-13T14:16:55.740 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.619129295Z level=info msg="Executing migration" id="add index query_history.user_id-query_uid" 2024-02-13T14:16:55.741 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.619632116Z level=info msg="Executing migration" id="add column org_id in query_history_star" 2024-02-13T14:16:55.741 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.620628282Z level=info msg="Executing migration" id="alter table query_history_star_mig column user_id type to bigint" 2024-02-13T14:16:55.741 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.621496288Z level=info msg="Executing migration" id="create correlation table v1" 2024-02-13T14:16:55.741 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.622352526Z level=info msg="Executing migration" id="add index correlations.uid" 2024-02-13T14:16:55.741 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.622833007Z level=info msg="Executing migration" id="add index correlations.source_uid" 2024-02-13T14:16:55.741 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.623680177Z level=info msg="Executing migration" id="add correlation config column" 2024-02-13T14:16:55.741 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.6246592Z level=info msg="Executing migration" id="create entity_events table" 2024-02-13T14:16:55.742 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.62568048Z level=info msg="Executing migration" id="create dashboard public config v1" 2024-02-13T14:16:55.742 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.626262688Z level=info msg="Executing migration" id="drop index UQE_dashboard_public_config_uid - v1" 2024-02-13T14:16:55.742 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.626351459Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop index UQE_dashboard_public_config_uid - v1" 2024-02-13T14:16:55.742 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.627135794Z level=info msg="Executing migration" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-02-13T14:16:55.742 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.627252246Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-02-13T14:16:55.745 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.627907249Z level=info msg="Executing migration" id="Drop old dashboard public config table" 2024-02-13T14:16:55.745 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.628796783Z level=info msg="Executing migration" id="recreate dashboard public config v1" 2024-02-13T14:16:55.745 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.629245508Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_uid - v1" 2024-02-13T14:16:55.745 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.629704521Z level=info msg="Executing migration" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-02-13T14:16:55.745 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.630169788Z level=info msg="Executing migration" id="drop index UQE_dashboard_public_config_uid - v2" 2024-02-13T14:16:55.746 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.630609797Z level=info msg="Executing migration" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v2" 2024-02-13T14:16:55.746 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.631039397Z level=info msg="Executing migration" id="Drop public config table" 2024-02-13T14:16:55.746 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.631534157Z level=info msg="Executing migration" id="Recreate dashboard public config v2" 2024-02-13T14:16:55.746 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.631949469Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_uid - v2" 2024-02-13T14:16:55.746 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.632438521Z level=info msg="Executing migration" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v2" 2024-02-13T14:16:55.746 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.632949743Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_access_token - v2" 2024-02-13T14:16:55.746 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.633450323Z level=info msg="Executing migration" id="Rename table dashboard_public_config to dashboard_public - v2" 2024-02-13T14:16:55.747 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.636529769Z level=info msg="Executing migration" id="add annotations_enabled column" 2024-02-13T14:16:55.747 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.637604472Z level=info msg="Executing migration" id="add time_selection_enabled column" 2024-02-13T14:16:55.747 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.638664357Z level=info msg="Executing migration" id="delete orphaned public dashboards" 2024-02-13T14:16:55.747 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.639036347Z level=info msg="Executing migration" id="add share column" 2024-02-13T14:16:55.747 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.640003277Z level=info msg="Executing migration" id="create default alerting folders" 2024-02-13T14:16:55.747 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.640480426Z level=info msg="Executing migration" id="create file table" 2024-02-13T14:16:55.747 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.64095896Z level=info msg="Executing migration" id="file table idx: path natural pk" 2024-02-13T14:16:55.748 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.641682741Z level=info msg="Executing migration" id="file table idx: parent_folder_path_hash fast folder retrieval" 2024-02-13T14:16:55.748 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.64223271Z level=info msg="Executing migration" id="create file_meta table" 2024-02-13T14:16:55.748 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.642706015Z level=info msg="Executing migration" id="file table idx: path key" 2024-02-13T14:16:55.748 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.643686913Z level=info msg="Executing migration" id="set path collation in file table" 2024-02-13T14:16:55.748 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.644294909Z level=info msg="Executing migration" id="managed permissions migration" 2024-02-13T14:16:55.751 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.645161135Z level=info msg="Executing migration" id="managed folder permissions alert actions migration" 2024-02-13T14:16:55.751 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.645500878Z level=info msg="Executing migration" id="RBAC action name migrator" 2024-02-13T14:16:55.751 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.646403758Z level=info msg="Executing migration" id="Add UID column to playlist" 2024-02-13T14:16:55.752 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.647638759Z level=info msg="Executing migration" id="Update uid column values in playlist" 2024-02-13T14:16:55.752 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.647963749Z level=info msg="Executing migration" id="Add index for uid in playlist" 2024-02-13T14:16:55.752 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.648786949Z level=info msg="Executing migration" id="update group index for alert rules" 2024-02-13T14:16:55.752 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.649203663Z level=info msg="Executing migration" id="managed folder permissions alert actions repeated migration" 2024-02-13T14:16:55.752 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.650011722Z level=info msg="Executing migration" id="admin only folder/dashboard permission" 2024-02-13T14:16:55.752 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.650902067Z level=info msg="Executing migration" id="add action column to seed_assignment" 2024-02-13T14:16:55.752 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.651945648Z level=info msg="Executing migration" id="add scope column to seed_assignment" 2024-02-13T14:16:55.752 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.652987343Z level=info msg="Executing migration" id="remove unique index builtin_role_role_name before nullable update" 2024-02-13T14:16:55.752 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.653484602Z level=info msg="Executing migration" id="update seed_assignment role_name column to nullable" 2024-02-13T14:16:55.752 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.663326415Z level=info msg="Executing migration" id="add unique index builtin_role_name back" 2024-02-13T14:16:55.752 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.664130389Z level=info msg="Executing migration" id="add unique index builtin_role_action_scope" 2024-02-13T14:16:55.753 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.665006987Z level=info msg="Executing migration" id="add primary key to seed_assigment" 2024-02-13T14:16:55.753 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.668589585Z level=info msg="Executing migration" id="managed folder permissions alert actions repeated fixed migration" 2024-02-13T14:16:55.753 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.669423732Z level=info msg="Executing migration" id="migrate external alertmanagers to datsourcse" 2024-02-13T14:16:55.753 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.67036683Z level=info msg="Executing migration" id="create folder table" 2024-02-13T14:16:55.753 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.671031289Z level=info msg="Executing migration" id="Add index for parent_uid" 2024-02-13T14:16:55.753 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.671876555Z level=info msg="Executing migration" id="Add unique index for folder.uid and folder.org_id" 2024-02-13T14:16:55.753 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.672789319Z level=info msg="Executing migration" id="Update folder title length" 2024-02-13T14:16:55.753 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.673495264Z level=info msg="Executing migration" id="Add unique index for folder.title and folder.parent_uid" 2024-02-13T14:16:55.753 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=migrator t=2024-02-13T14:16:55.674333469Z level=info msg="migrations completed" performed=485 skipped=0 duration=655.97227ms 2024-02-13T14:16:55.753 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=sqlstore t=2024-02-13T14:16:55.674757577Z level=info msg="Created default organization" 2024-02-13T14:16:55.754 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=plugin.loader t=2024-02-13T14:16:55.728573688Z level=info msg="Plugin registered" pluginID=input 2024-02-13T14:16:56.033 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=plugin.loader t=2024-02-13T14:16:55.758471494Z level=info msg="Plugin registered" pluginID=vonage-status-panel 2024-02-13T14:16:56.033 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=plugin.loader t=2024-02-13T14:16:55.758492517Z level=info msg="Plugin registered" pluginID=grafana-piechart-panel 2024-02-13T14:16:56.034 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=secrets t=2024-02-13T14:16:55.758524748Z level=info msg="Envelope encryption state" enabled=true currentprovider=secretKey.v1 2024-02-13T14:16:56.034 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=query_data t=2024-02-13T14:16:55.760625918Z level=info msg="Query Service initialization" 2024-02-13T14:16:56.034 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=live.push_http t=2024-02-13T14:16:55.764664291Z level=info msg="Live Push Gateway initialization" 2024-02-13T14:16:56.334 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:16:56 smithi148 ceph-mon[31198]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:16:56.334 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:16:56 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:16:56.334 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:16:56 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:16:56.334 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:16:56 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:16:56.334 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:16:56 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:16:56.334 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:16:56 smithi148 ceph-mon[31198]: Deploying daemon node-exporter.a on smithi087 2024-02-13T14:16:56.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:16:56 smithi087 ceph-mon[27372]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:16:56.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:16:56 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:16:56.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:16:56 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:16:56.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:16:56 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:16:56.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:16:56 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:16:56.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:16:56 smithi087 ceph-mon[27372]: Deploying daemon node-exporter.a on smithi087 2024-02-13T14:16:56.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:16:56 smithi087 ceph-mon[39120]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:16:56.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:16:56 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:16:56.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:16:56 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:16:56.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:16:56 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:16:56.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:16:56 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:16:56.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:16:56 smithi087 ceph-mon[39120]: Deploying daemon node-exporter.a on smithi087 2024-02-13T14:16:56.493 INFO:journalctl@ceph.node-exporter.a.smithi087.stdout:Feb 13 14:16:56 smithi087 bash[77194]: Trying to pull quay.io/prometheus/node-exporter:v1.5.0... 2024-02-13T14:16:56.619 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:56 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=infra.usagestats.collector t=2024-02-13T14:16:56.41775442Z level=info msg="registering usage stat providers" usageStatsProvidersLen=2 2024-02-13T14:16:56.619 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:56 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=provisioning.datasources t=2024-02-13T14:16:56.418791674Z level=info msg="inserting datasource from configuration " name=Dashboard1 uid=P43CA22E17D0F9596 2024-02-13T14:16:56.619 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:56 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=provisioning.datasources t=2024-02-13T14:16:56.430547681Z level=info msg="inserting datasource from configuration " name=Loki uid=P8E80F9AEF21F6940 2024-02-13T14:16:56.619 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:56 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=provisioning.alerting t=2024-02-13T14:16:56.442234206Z level=info msg="starting to provision alerting" 2024-02-13T14:16:56.619 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:56 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=provisioning.alerting t=2024-02-13T14:16:56.442263533Z level=info msg="finished to provision alerting" 2024-02-13T14:16:56.619 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:56 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=grafanaStorageLogger t=2024-02-13T14:16:56.442423229Z level=info msg="storage starting" 2024-02-13T14:16:56.619 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:56 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=ngalert.state.manager t=2024-02-13T14:16:56.442471431Z level=info msg="Warming state cache for startup" 2024-02-13T14:16:56.619 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:56 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=ngalert.state.manager t=2024-02-13T14:16:56.442586623Z level=info msg="State cache has been initialized" states=0 duration=115.178µs 2024-02-13T14:16:56.619 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:56 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=ticker t=2024-02-13T14:16:56.442654443Z level=info msg=starting first_tick=2024-02-13T14:17:00Z 2024-02-13T14:16:56.619 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:56 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=ngalert.multiorg.alertmanager t=2024-02-13T14:16:56.442849245Z level=info msg="starting MultiOrg Alertmanager" 2024-02-13T14:16:56.620 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:56 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=http.server t=2024-02-13T14:16:56.44435952Z level=info msg="HTTP Server Listen" address=[::]:3000 protocol=https subUrl= socket= 2024-02-13T14:16:56.620 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:16:56 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a[63664]: logger=sqlstore.transactions t=2024-02-13T14:16:56.458950843Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-02-13T14:16:56.991 INFO:journalctl@ceph.node-exporter.a.smithi087.stdout:Feb 13 14:16:56 smithi087 bash[77194]: Getting image source signatures 2024-02-13T14:16:56.992 INFO:journalctl@ceph.node-exporter.a.smithi087.stdout:Feb 13 14:16:56 smithi087 bash[77194]: Copying blob sha256:22b70bddd3acadc892fca4c2af4260629bfda5dfd11ebc106a93ce24e752b5ed 2024-02-13T14:16:56.992 INFO:journalctl@ceph.node-exporter.a.smithi087.stdout:Feb 13 14:16:56 smithi087 bash[77194]: Copying blob sha256:c0e87333d3805a7b7e1c25783dd25f27da0fe70a8e262eee7ed190c8dc843fa7 2024-02-13T14:16:56.992 INFO:journalctl@ceph.node-exporter.a.smithi087.stdout:Feb 13 14:16:56 smithi087 bash[77194]: Copying blob sha256:5c12815fee558b157dee7f7509dedbaba0a8379098858a65ec869e1f1526ea0c 2024-02-13T14:16:57.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:16:57 smithi148 ceph-mon[31198]: pgmap v34: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:16:57.491 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:16:57 smithi087 ceph-mon[27372]: pgmap v34: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:16:57.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:16:57 smithi087 ceph-mon[39120]: pgmap v34: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:16:57.492 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:16:57 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: ::ffff:172.21.15.148 - - [13/Feb/2024:14:16:57] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-13T14:16:58.491 INFO:journalctl@ceph.node-exporter.a.smithi087.stdout:Feb 13 14:16:58 smithi087 bash[77194]: Copying config sha256:0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 2024-02-13T14:16:58.492 INFO:journalctl@ceph.node-exporter.a.smithi087.stdout:Feb 13 14:16:58 smithi087 bash[77194]: Writing manifest to image destination 2024-02-13T14:16:58.851 INFO:journalctl@ceph.node-exporter.a.smithi087.stdout:Feb 13 14:16:58 smithi087 podman[77194]: 2024-02-13 14:16:58.491849942 +0000 UTC m=+2.340584645 image pull 0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 quay.io/prometheus/node-exporter:v1.5.0 2024-02-13T14:16:58.851 INFO:journalctl@ceph.node-exporter.a.smithi087.stdout:Feb 13 14:16:58 smithi087 podman[77194]: 2024-02-13 14:16:58.606954663 +0000 UTC m=+2.455689362 container create af62e3ac6b0a15606ddf42ea6f3d4efe42d8a32c8dcf2425a6279a31597f7d08 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-node-exporter-a, maintainer=The Prometheus Authors ) 2024-02-13T14:16:59.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:16:59 smithi087 ceph-mon[27372]: pgmap v35: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-13T14:16:59.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:16:59 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:16:59.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:16:59 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:16:59.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:16:59 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:16:59.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:16:59 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:16:59.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:16:59 smithi087 ceph-mon[39120]: pgmap v35: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-13T14:16:59.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:16:59 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:16:59.243 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:16:59 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:16:59.243 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:16:59 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:16:59.243 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:16:59 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:16:59.243 INFO:journalctl@ceph.node-exporter.a.smithi087.stdout:Feb 13 14:16:58 smithi087 podman[77194]: 2024-02-13 14:16:58.850395487 +0000 UTC m=+2.699130184 container init af62e3ac6b0a15606ddf42ea6f3d4efe42d8a32c8dcf2425a6279a31597f7d08 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-node-exporter-a, maintainer=The Prometheus Authors ) 2024-02-13T14:16:59.243 INFO:journalctl@ceph.node-exporter.a.smithi087.stdout:Feb 13 14:16:58 smithi087 podman[77194]: 2024-02-13 14:16:58.856276634 +0000 UTC m=+2.705011330 container start af62e3ac6b0a15606ddf42ea6f3d4efe42d8a32c8dcf2425a6279a31597f7d08 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-node-exporter-a, maintainer=The Prometheus Authors ) 2024-02-13T14:16:59.243 INFO:journalctl@ceph.node-exporter.a.smithi087.stdout:Feb 13 14:16:58 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-node-exporter-a[77245]: ts=2024-02-13T14:16:58.859Z caller=node_exporter.go:180 level=info msg="Starting node_exporter" version="(version=1.5.0, branch=HEAD, revision=1b48970ffcf5630534fb00bb0687d73c66d1c959)" 2024-02-13T14:16:59.244 INFO:journalctl@ceph.node-exporter.a.smithi087.stdout:Feb 13 14:16:58 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-node-exporter-a[77245]: ts=2024-02-13T14:16:58.859Z caller=node_exporter.go:181 level=info msg="Build context" build_context="(go=go1.19.3, user=root@6e7732a7b81b, date=20221129-18:59:09)" 2024-02-13T14:16:59.244 INFO:journalctl@ceph.node-exporter.a.smithi087.stdout:Feb 13 14:16:58 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-node-exporter-a[77245]: ts=2024-02-13T14:16:58.860Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2024-02-13T14:16:59.244 INFO:journalctl@ceph.node-exporter.a.smithi087.stdout:Feb 13 14:16:58 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-node-exporter-a[77245]: ts=2024-02-13T14:16:58.860Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-02-13T14:16:59.244 INFO:journalctl@ceph.node-exporter.a.smithi087.stdout:Feb 13 14:16:58 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-node-exporter-a[77245]: ts=2024-02-13T14:16:58.860Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2024-02-13T14:16:59.244 INFO:journalctl@ceph.node-exporter.a.smithi087.stdout:Feb 13 14:16:58 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-node-exporter-a[77245]: ts=2024-02-13T14:16:58.860Z caller=diskstats_linux.go:264 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2024-02-13T14:16:59.244 INFO:journalctl@ceph.node-exporter.a.smithi087.stdout:Feb 13 14:16:58 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-node-exporter-a[77245]: ts=2024-02-13T14:16:58.861Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2024-02-13T14:16:59.244 INFO:journalctl@ceph.node-exporter.a.smithi087.stdout:Feb 13 14:16:58 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-node-exporter-a[77245]: ts=2024-02-13T14:16:58.861Z caller=node_exporter.go:117 level=info collector=arp 2024-02-13T14:16:59.244 INFO:journalctl@ceph.node-exporter.a.smithi087.stdout:Feb 13 14:16:58 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-node-exporter-a[77245]: ts=2024-02-13T14:16:58.861Z caller=node_exporter.go:117 level=info collector=bcache 2024-02-13T14:16:59.244 INFO:journalctl@ceph.node-exporter.a.smithi087.stdout:Feb 13 14:16:58 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-node-exporter-a[77245]: ts=2024-02-13T14:16:58.861Z caller=node_exporter.go:117 level=info collector=bonding 2024-02-13T14:16:59.244 INFO:journalctl@ceph.node-exporter.a.smithi087.stdout:Feb 13 14:16:58 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-node-exporter-a[77245]: ts=2024-02-13T14:16:58.861Z caller=node_exporter.go:117 level=info collector=btrfs 2024-02-13T14:16:59.244 INFO:journalctl@ceph.node-exporter.a.smithi087.stdout:Feb 13 14:16:58 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-node-exporter-a[77245]: ts=2024-02-13T14:16:58.861Z caller=node_exporter.go:117 level=info collector=conntrack 2024-02-13T14:16:59.244 INFO:journalctl@ceph.node-exporter.a.smithi087.stdout:Feb 13 14:16:58 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-node-exporter-a[77245]: ts=2024-02-13T14:16:58.861Z caller=node_exporter.go:117 level=info collector=cpu 2024-02-13T14:16:59.244 INFO:journalctl@ceph.node-exporter.a.smithi087.stdout:Feb 13 14:16:58 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-node-exporter-a[77245]: ts=2024-02-13T14:16:58.861Z caller=node_exporter.go:117 level=info collector=cpufreq 2024-02-13T14:16:59.245 INFO:journalctl@ceph.node-exporter.a.smithi087.stdout:Feb 13 14:16:58 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-node-exporter-a[77245]: ts=2024-02-13T14:16:58.861Z caller=node_exporter.go:117 level=info collector=diskstats 2024-02-13T14:16:59.245 INFO:journalctl@ceph.node-exporter.a.smithi087.stdout:Feb 13 14:16:58 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-node-exporter-a[77245]: ts=2024-02-13T14:16:58.861Z caller=node_exporter.go:117 level=info collector=dmi 2024-02-13T14:16:59.245 INFO:journalctl@ceph.node-exporter.a.smithi087.stdout:Feb 13 14:16:58 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-node-exporter-a[77245]: ts=2024-02-13T14:16:58.861Z caller=node_exporter.go:117 level=info collector=edac 2024-02-13T14:16:59.245 INFO:journalctl@ceph.node-exporter.a.smithi087.stdout:Feb 13 14:16:58 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-node-exporter-a[77245]: ts=2024-02-13T14:16:58.861Z caller=node_exporter.go:117 level=info collector=entropy 2024-02-13T14:16:59.245 INFO:journalctl@ceph.node-exporter.a.smithi087.stdout:Feb 13 14:16:58 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-node-exporter-a[77245]: ts=2024-02-13T14:16:58.861Z caller=node_exporter.go:117 level=info collector=fibrechannel 2024-02-13T14:16:59.245 INFO:journalctl@ceph.node-exporter.a.smithi087.stdout:Feb 13 14:16:58 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-node-exporter-a[77245]: ts=2024-02-13T14:16:58.861Z caller=node_exporter.go:117 level=info collector=filefd 2024-02-13T14:16:59.245 INFO:journalctl@ceph.node-exporter.a.smithi087.stdout:Feb 13 14:16:58 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-node-exporter-a[77245]: ts=2024-02-13T14:16:58.861Z caller=node_exporter.go:117 level=info collector=filesystem 2024-02-13T14:16:59.245 INFO:journalctl@ceph.node-exporter.a.smithi087.stdout:Feb 13 14:16:58 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-node-exporter-a[77245]: ts=2024-02-13T14:16:58.861Z caller=node_exporter.go:117 level=info collector=hwmon 2024-02-13T14:16:59.245 INFO:journalctl@ceph.node-exporter.a.smithi087.stdout:Feb 13 14:16:58 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-node-exporter-a[77245]: ts=2024-02-13T14:16:58.861Z caller=node_exporter.go:117 level=info collector=infiniband 2024-02-13T14:16:59.245 INFO:journalctl@ceph.node-exporter.a.smithi087.stdout:Feb 13 14:16:58 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-node-exporter-a[77245]: ts=2024-02-13T14:16:58.861Z caller=node_exporter.go:117 level=info collector=ipvs 2024-02-13T14:16:59.245 INFO:journalctl@ceph.node-exporter.a.smithi087.stdout:Feb 13 14:16:58 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-node-exporter-a[77245]: ts=2024-02-13T14:16:58.861Z caller=node_exporter.go:117 level=info collector=loadavg 2024-02-13T14:16:59.245 INFO:journalctl@ceph.node-exporter.a.smithi087.stdout:Feb 13 14:16:58 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-node-exporter-a[77245]: ts=2024-02-13T14:16:58.861Z caller=node_exporter.go:117 level=info collector=mdadm 2024-02-13T14:16:59.245 INFO:journalctl@ceph.node-exporter.a.smithi087.stdout:Feb 13 14:16:58 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-node-exporter-a[77245]: ts=2024-02-13T14:16:58.861Z caller=node_exporter.go:117 level=info collector=meminfo 2024-02-13T14:16:59.245 INFO:journalctl@ceph.node-exporter.a.smithi087.stdout:Feb 13 14:16:58 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-node-exporter-a[77245]: ts=2024-02-13T14:16:58.861Z caller=node_exporter.go:117 level=info collector=netclass 2024-02-13T14:16:59.246 INFO:journalctl@ceph.node-exporter.a.smithi087.stdout:Feb 13 14:16:58 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-node-exporter-a[77245]: ts=2024-02-13T14:16:58.861Z caller=node_exporter.go:117 level=info collector=netdev 2024-02-13T14:16:59.246 INFO:journalctl@ceph.node-exporter.a.smithi087.stdout:Feb 13 14:16:58 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-node-exporter-a[77245]: ts=2024-02-13T14:16:58.861Z caller=node_exporter.go:117 level=info collector=netstat 2024-02-13T14:16:59.246 INFO:journalctl@ceph.node-exporter.a.smithi087.stdout:Feb 13 14:16:58 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-node-exporter-a[77245]: ts=2024-02-13T14:16:58.861Z caller=node_exporter.go:117 level=info collector=nfs 2024-02-13T14:16:59.246 INFO:journalctl@ceph.node-exporter.a.smithi087.stdout:Feb 13 14:16:58 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-node-exporter-a[77245]: ts=2024-02-13T14:16:58.861Z caller=node_exporter.go:117 level=info collector=nfsd 2024-02-13T14:16:59.246 INFO:journalctl@ceph.node-exporter.a.smithi087.stdout:Feb 13 14:16:58 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-node-exporter-a[77245]: ts=2024-02-13T14:16:58.861Z caller=node_exporter.go:117 level=info collector=nvme 2024-02-13T14:16:59.246 INFO:journalctl@ceph.node-exporter.a.smithi087.stdout:Feb 13 14:16:58 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-node-exporter-a[77245]: ts=2024-02-13T14:16:58.861Z caller=node_exporter.go:117 level=info collector=os 2024-02-13T14:16:59.246 INFO:journalctl@ceph.node-exporter.a.smithi087.stdout:Feb 13 14:16:58 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-node-exporter-a[77245]: ts=2024-02-13T14:16:58.861Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2024-02-13T14:16:59.246 INFO:journalctl@ceph.node-exporter.a.smithi087.stdout:Feb 13 14:16:58 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-node-exporter-a[77245]: ts=2024-02-13T14:16:58.861Z caller=node_exporter.go:117 level=info collector=pressure 2024-02-13T14:16:59.246 INFO:journalctl@ceph.node-exporter.a.smithi087.stdout:Feb 13 14:16:58 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-node-exporter-a[77245]: ts=2024-02-13T14:16:58.861Z caller=node_exporter.go:117 level=info collector=rapl 2024-02-13T14:16:59.246 INFO:journalctl@ceph.node-exporter.a.smithi087.stdout:Feb 13 14:16:58 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-node-exporter-a[77245]: ts=2024-02-13T14:16:58.861Z caller=node_exporter.go:117 level=info collector=schedstat 2024-02-13T14:16:59.246 INFO:journalctl@ceph.node-exporter.a.smithi087.stdout:Feb 13 14:16:58 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-node-exporter-a[77245]: ts=2024-02-13T14:16:58.861Z caller=node_exporter.go:117 level=info collector=selinux 2024-02-13T14:16:59.246 INFO:journalctl@ceph.node-exporter.a.smithi087.stdout:Feb 13 14:16:58 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-node-exporter-a[77245]: ts=2024-02-13T14:16:58.861Z caller=node_exporter.go:117 level=info collector=sockstat 2024-02-13T14:16:59.246 INFO:journalctl@ceph.node-exporter.a.smithi087.stdout:Feb 13 14:16:58 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-node-exporter-a[77245]: ts=2024-02-13T14:16:58.861Z caller=node_exporter.go:117 level=info collector=softnet 2024-02-13T14:16:59.246 INFO:journalctl@ceph.node-exporter.a.smithi087.stdout:Feb 13 14:16:58 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-node-exporter-a[77245]: ts=2024-02-13T14:16:58.861Z caller=node_exporter.go:117 level=info collector=stat 2024-02-13T14:16:59.246 INFO:journalctl@ceph.node-exporter.a.smithi087.stdout:Feb 13 14:16:58 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-node-exporter-a[77245]: ts=2024-02-13T14:16:58.861Z caller=node_exporter.go:117 level=info collector=tapestats 2024-02-13T14:16:59.247 INFO:journalctl@ceph.node-exporter.a.smithi087.stdout:Feb 13 14:16:58 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-node-exporter-a[77245]: ts=2024-02-13T14:16:58.861Z caller=node_exporter.go:117 level=info collector=textfile 2024-02-13T14:16:59.247 INFO:journalctl@ceph.node-exporter.a.smithi087.stdout:Feb 13 14:16:58 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-node-exporter-a[77245]: ts=2024-02-13T14:16:58.861Z caller=node_exporter.go:117 level=info collector=thermal_zone 2024-02-13T14:16:59.247 INFO:journalctl@ceph.node-exporter.a.smithi087.stdout:Feb 13 14:16:58 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-node-exporter-a[77245]: ts=2024-02-13T14:16:58.861Z caller=node_exporter.go:117 level=info collector=time 2024-02-13T14:16:59.247 INFO:journalctl@ceph.node-exporter.a.smithi087.stdout:Feb 13 14:16:58 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-node-exporter-a[77245]: ts=2024-02-13T14:16:58.861Z caller=node_exporter.go:117 level=info collector=udp_queues 2024-02-13T14:16:59.247 INFO:journalctl@ceph.node-exporter.a.smithi087.stdout:Feb 13 14:16:58 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-node-exporter-a[77245]: ts=2024-02-13T14:16:58.861Z caller=node_exporter.go:117 level=info collector=uname 2024-02-13T14:16:59.247 INFO:journalctl@ceph.node-exporter.a.smithi087.stdout:Feb 13 14:16:58 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-node-exporter-a[77245]: ts=2024-02-13T14:16:58.861Z caller=node_exporter.go:117 level=info collector=vmstat 2024-02-13T14:16:59.247 INFO:journalctl@ceph.node-exporter.a.smithi087.stdout:Feb 13 14:16:58 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-node-exporter-a[77245]: ts=2024-02-13T14:16:58.861Z caller=node_exporter.go:117 level=info collector=xfs 2024-02-13T14:16:59.247 INFO:journalctl@ceph.node-exporter.a.smithi087.stdout:Feb 13 14:16:58 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-node-exporter-a[77245]: ts=2024-02-13T14:16:58.861Z caller=node_exporter.go:117 level=info collector=zfs 2024-02-13T14:16:59.247 INFO:journalctl@ceph.node-exporter.a.smithi087.stdout:Feb 13 14:16:58 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-node-exporter-a[77245]: ts=2024-02-13T14:16:58.862Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9100 2024-02-13T14:16:59.247 INFO:journalctl@ceph.node-exporter.a.smithi087.stdout:Feb 13 14:16:58 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-node-exporter-a[77245]: ts=2024-02-13T14:16:58.862Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9100 2024-02-13T14:16:59.247 INFO:journalctl@ceph.node-exporter.a.smithi087.stdout:Feb 13 14:16:58 smithi087 bash[77194]: af62e3ac6b0a15606ddf42ea6f3d4efe42d8a32c8dcf2425a6279a31597f7d08 2024-02-13T14:16:59.247 INFO:journalctl@ceph.node-exporter.a.smithi087.stdout:Feb 13 14:16:58 smithi087 systemd[1]: Started Ceph node-exporter.a for 5c04b074-ca79-11ee-95b9-87774f69a715. 2024-02-13T14:16:59.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:16:59 smithi148 ceph-mon[31198]: pgmap v35: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-13T14:16:59.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:16:59 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:16:59.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:16:59 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:16:59.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:16:59 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:16:59.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:16:59 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:17:00.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:17:00 smithi148 ceph-mon[31198]: Deploying daemon node-exporter.b on smithi148 2024-02-13T14:17:00.369 INFO:journalctl@ceph.node-exporter.b.smithi148.stdout:Feb 13 14:17:00 smithi148 bash[63946]: Trying to pull quay.io/prometheus/node-exporter:v1.5.0... 2024-02-13T14:17:00.491 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:17:00 smithi087 ceph-mon[27372]: Deploying daemon node-exporter.b on smithi148 2024-02-13T14:17:00.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:17:00 smithi087 ceph-mon[39120]: Deploying daemon node-exporter.b on smithi148 2024-02-13T14:17:01.118 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:17:01 smithi148 ceph-mon[31198]: pgmap v36: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:17:01.119 INFO:journalctl@ceph.node-exporter.b.smithi148.stdout:Feb 13 14:17:00 smithi148 bash[63946]: Getting image source signatures 2024-02-13T14:17:01.119 INFO:journalctl@ceph.node-exporter.b.smithi148.stdout:Feb 13 14:17:00 smithi148 bash[63946]: Copying blob sha256:c0e87333d3805a7b7e1c25783dd25f27da0fe70a8e262eee7ed190c8dc843fa7 2024-02-13T14:17:01.119 INFO:journalctl@ceph.node-exporter.b.smithi148.stdout:Feb 13 14:17:00 smithi148 bash[63946]: Copying blob sha256:22b70bddd3acadc892fca4c2af4260629bfda5dfd11ebc106a93ce24e752b5ed 2024-02-13T14:17:01.119 INFO:journalctl@ceph.node-exporter.b.smithi148.stdout:Feb 13 14:17:00 smithi148 bash[63946]: Copying blob sha256:5c12815fee558b157dee7f7509dedbaba0a8379098858a65ec869e1f1526ea0c 2024-02-13T14:17:01.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:17:01 smithi087 ceph-mon[27372]: pgmap v36: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:17:01.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:17:01 smithi087 ceph-mon[39120]: pgmap v36: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:17:02.871 INFO:journalctl@ceph.node-exporter.b.smithi148.stdout:Feb 13 14:17:02 smithi148 bash[63946]: Copying config sha256:0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 2024-02-13T14:17:02.871 INFO:journalctl@ceph.node-exporter.b.smithi148.stdout:Feb 13 14:17:02 smithi148 bash[63946]: Writing manifest to image destination 2024-02-13T14:17:03.268 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:17:03 smithi148 ceph-mon[31198]: pgmap v37: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-13T14:17:03.269 INFO:journalctl@ceph.node-exporter.b.smithi148.stdout:Feb 13 14:17:02 smithi148 podman[63946]: 2024-02-13 14:17:02.866886583 +0000 UTC m=+2.738435290 image pull 0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 quay.io/prometheus/node-exporter:v1.5.0 2024-02-13T14:17:03.269 INFO:journalctl@ceph.node-exporter.b.smithi148.stdout:Feb 13 14:17:02 smithi148 podman[63946]: 2024-02-13 14:17:02.982622738 +0000 UTC m=+2.854171443 container create d7d208c49b58e9e0c15345a8dd874a1b8e1f3222f758b3be3e848a81fd094bc1 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-node-exporter-b, maintainer=The Prometheus Authors ) 2024-02-13T14:17:03.269 INFO:journalctl@ceph.node-exporter.b.smithi148.stdout:Feb 13 14:17:03 smithi148 podman[63946]: 2024-02-13 14:17:03.192847195 +0000 UTC m=+3.064395901 container init d7d208c49b58e9e0c15345a8dd874a1b8e1f3222f758b3be3e848a81fd094bc1 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-node-exporter-b, maintainer=The Prometheus Authors ) 2024-02-13T14:17:03.269 INFO:journalctl@ceph.node-exporter.b.smithi148.stdout:Feb 13 14:17:03 smithi148 podman[63946]: 2024-02-13 14:17:03.198351034 +0000 UTC m=+3.069899741 container start d7d208c49b58e9e0c15345a8dd874a1b8e1f3222f758b3be3e848a81fd094bc1 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-node-exporter-b, maintainer=The Prometheus Authors ) 2024-02-13T14:17:03.269 INFO:journalctl@ceph.node-exporter.b.smithi148.stdout:Feb 13 14:17:03 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-node-exporter-b[63998]: ts=2024-02-13T14:17:03.201Z caller=node_exporter.go:180 level=info msg="Starting node_exporter" version="(version=1.5.0, branch=HEAD, revision=1b48970ffcf5630534fb00bb0687d73c66d1c959)" 2024-02-13T14:17:03.269 INFO:journalctl@ceph.node-exporter.b.smithi148.stdout:Feb 13 14:17:03 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-node-exporter-b[63998]: ts=2024-02-13T14:17:03.202Z caller=node_exporter.go:181 level=info msg="Build context" build_context="(go=go1.19.3, user=root@6e7732a7b81b, date=20221129-18:59:09)" 2024-02-13T14:17:03.270 INFO:journalctl@ceph.node-exporter.b.smithi148.stdout:Feb 13 14:17:03 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-node-exporter-b[63998]: ts=2024-02-13T14:17:03.202Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2024-02-13T14:17:03.270 INFO:journalctl@ceph.node-exporter.b.smithi148.stdout:Feb 13 14:17:03 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-node-exporter-b[63998]: ts=2024-02-13T14:17:03.202Z caller=diskstats_linux.go:264 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2024-02-13T14:17:03.270 INFO:journalctl@ceph.node-exporter.b.smithi148.stdout:Feb 13 14:17:03 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-node-exporter-b[63998]: ts=2024-02-13T14:17:03.202Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2024-02-13T14:17:03.270 INFO:journalctl@ceph.node-exporter.b.smithi148.stdout:Feb 13 14:17:03 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-node-exporter-b[63998]: ts=2024-02-13T14:17:03.202Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-02-13T14:17:03.270 INFO:journalctl@ceph.node-exporter.b.smithi148.stdout:Feb 13 14:17:03 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-node-exporter-b[63998]: ts=2024-02-13T14:17:03.203Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2024-02-13T14:17:03.270 INFO:journalctl@ceph.node-exporter.b.smithi148.stdout:Feb 13 14:17:03 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-node-exporter-b[63998]: ts=2024-02-13T14:17:03.203Z caller=node_exporter.go:117 level=info collector=arp 2024-02-13T14:17:03.270 INFO:journalctl@ceph.node-exporter.b.smithi148.stdout:Feb 13 14:17:03 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-node-exporter-b[63998]: ts=2024-02-13T14:17:03.203Z caller=node_exporter.go:117 level=info collector=bcache 2024-02-13T14:17:03.270 INFO:journalctl@ceph.node-exporter.b.smithi148.stdout:Feb 13 14:17:03 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-node-exporter-b[63998]: ts=2024-02-13T14:17:03.203Z caller=node_exporter.go:117 level=info collector=bonding 2024-02-13T14:17:03.270 INFO:journalctl@ceph.node-exporter.b.smithi148.stdout:Feb 13 14:17:03 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-node-exporter-b[63998]: ts=2024-02-13T14:17:03.203Z caller=node_exporter.go:117 level=info collector=btrfs 2024-02-13T14:17:03.270 INFO:journalctl@ceph.node-exporter.b.smithi148.stdout:Feb 13 14:17:03 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-node-exporter-b[63998]: ts=2024-02-13T14:17:03.203Z caller=node_exporter.go:117 level=info collector=conntrack 2024-02-13T14:17:03.270 INFO:journalctl@ceph.node-exporter.b.smithi148.stdout:Feb 13 14:17:03 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-node-exporter-b[63998]: ts=2024-02-13T14:17:03.203Z caller=node_exporter.go:117 level=info collector=cpu 2024-02-13T14:17:03.270 INFO:journalctl@ceph.node-exporter.b.smithi148.stdout:Feb 13 14:17:03 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-node-exporter-b[63998]: ts=2024-02-13T14:17:03.203Z caller=node_exporter.go:117 level=info collector=cpufreq 2024-02-13T14:17:03.270 INFO:journalctl@ceph.node-exporter.b.smithi148.stdout:Feb 13 14:17:03 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-node-exporter-b[63998]: ts=2024-02-13T14:17:03.203Z caller=node_exporter.go:117 level=info collector=diskstats 2024-02-13T14:17:03.270 INFO:journalctl@ceph.node-exporter.b.smithi148.stdout:Feb 13 14:17:03 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-node-exporter-b[63998]: ts=2024-02-13T14:17:03.203Z caller=node_exporter.go:117 level=info collector=dmi 2024-02-13T14:17:03.271 INFO:journalctl@ceph.node-exporter.b.smithi148.stdout:Feb 13 14:17:03 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-node-exporter-b[63998]: ts=2024-02-13T14:17:03.203Z caller=node_exporter.go:117 level=info collector=edac 2024-02-13T14:17:03.271 INFO:journalctl@ceph.node-exporter.b.smithi148.stdout:Feb 13 14:17:03 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-node-exporter-b[63998]: ts=2024-02-13T14:17:03.203Z caller=node_exporter.go:117 level=info collector=entropy 2024-02-13T14:17:03.271 INFO:journalctl@ceph.node-exporter.b.smithi148.stdout:Feb 13 14:17:03 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-node-exporter-b[63998]: ts=2024-02-13T14:17:03.203Z caller=node_exporter.go:117 level=info collector=fibrechannel 2024-02-13T14:17:03.271 INFO:journalctl@ceph.node-exporter.b.smithi148.stdout:Feb 13 14:17:03 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-node-exporter-b[63998]: ts=2024-02-13T14:17:03.203Z caller=node_exporter.go:117 level=info collector=filefd 2024-02-13T14:17:03.271 INFO:journalctl@ceph.node-exporter.b.smithi148.stdout:Feb 13 14:17:03 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-node-exporter-b[63998]: ts=2024-02-13T14:17:03.203Z caller=node_exporter.go:117 level=info collector=filesystem 2024-02-13T14:17:03.271 INFO:journalctl@ceph.node-exporter.b.smithi148.stdout:Feb 13 14:17:03 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-node-exporter-b[63998]: ts=2024-02-13T14:17:03.203Z caller=node_exporter.go:117 level=info collector=hwmon 2024-02-13T14:17:03.271 INFO:journalctl@ceph.node-exporter.b.smithi148.stdout:Feb 13 14:17:03 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-node-exporter-b[63998]: ts=2024-02-13T14:17:03.203Z caller=node_exporter.go:117 level=info collector=infiniband 2024-02-13T14:17:03.271 INFO:journalctl@ceph.node-exporter.b.smithi148.stdout:Feb 13 14:17:03 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-node-exporter-b[63998]: ts=2024-02-13T14:17:03.203Z caller=node_exporter.go:117 level=info collector=ipvs 2024-02-13T14:17:03.271 INFO:journalctl@ceph.node-exporter.b.smithi148.stdout:Feb 13 14:17:03 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-node-exporter-b[63998]: ts=2024-02-13T14:17:03.203Z caller=node_exporter.go:117 level=info collector=loadavg 2024-02-13T14:17:03.271 INFO:journalctl@ceph.node-exporter.b.smithi148.stdout:Feb 13 14:17:03 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-node-exporter-b[63998]: ts=2024-02-13T14:17:03.203Z caller=node_exporter.go:117 level=info collector=mdadm 2024-02-13T14:17:03.271 INFO:journalctl@ceph.node-exporter.b.smithi148.stdout:Feb 13 14:17:03 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-node-exporter-b[63998]: ts=2024-02-13T14:17:03.203Z caller=node_exporter.go:117 level=info collector=meminfo 2024-02-13T14:17:03.271 INFO:journalctl@ceph.node-exporter.b.smithi148.stdout:Feb 13 14:17:03 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-node-exporter-b[63998]: ts=2024-02-13T14:17:03.203Z caller=node_exporter.go:117 level=info collector=netclass 2024-02-13T14:17:03.271 INFO:journalctl@ceph.node-exporter.b.smithi148.stdout:Feb 13 14:17:03 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-node-exporter-b[63998]: ts=2024-02-13T14:17:03.203Z caller=node_exporter.go:117 level=info collector=netdev 2024-02-13T14:17:03.271 INFO:journalctl@ceph.node-exporter.b.smithi148.stdout:Feb 13 14:17:03 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-node-exporter-b[63998]: ts=2024-02-13T14:17:03.203Z caller=node_exporter.go:117 level=info collector=netstat 2024-02-13T14:17:03.272 INFO:journalctl@ceph.node-exporter.b.smithi148.stdout:Feb 13 14:17:03 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-node-exporter-b[63998]: ts=2024-02-13T14:17:03.203Z caller=node_exporter.go:117 level=info collector=nfs 2024-02-13T14:17:03.272 INFO:journalctl@ceph.node-exporter.b.smithi148.stdout:Feb 13 14:17:03 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-node-exporter-b[63998]: ts=2024-02-13T14:17:03.203Z caller=node_exporter.go:117 level=info collector=nfsd 2024-02-13T14:17:03.272 INFO:journalctl@ceph.node-exporter.b.smithi148.stdout:Feb 13 14:17:03 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-node-exporter-b[63998]: ts=2024-02-13T14:17:03.203Z caller=node_exporter.go:117 level=info collector=nvme 2024-02-13T14:17:03.272 INFO:journalctl@ceph.node-exporter.b.smithi148.stdout:Feb 13 14:17:03 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-node-exporter-b[63998]: ts=2024-02-13T14:17:03.203Z caller=node_exporter.go:117 level=info collector=os 2024-02-13T14:17:03.272 INFO:journalctl@ceph.node-exporter.b.smithi148.stdout:Feb 13 14:17:03 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-node-exporter-b[63998]: ts=2024-02-13T14:17:03.203Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2024-02-13T14:17:03.272 INFO:journalctl@ceph.node-exporter.b.smithi148.stdout:Feb 13 14:17:03 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-node-exporter-b[63998]: ts=2024-02-13T14:17:03.203Z caller=node_exporter.go:117 level=info collector=pressure 2024-02-13T14:17:03.272 INFO:journalctl@ceph.node-exporter.b.smithi148.stdout:Feb 13 14:17:03 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-node-exporter-b[63998]: ts=2024-02-13T14:17:03.203Z caller=node_exporter.go:117 level=info collector=rapl 2024-02-13T14:17:03.272 INFO:journalctl@ceph.node-exporter.b.smithi148.stdout:Feb 13 14:17:03 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-node-exporter-b[63998]: ts=2024-02-13T14:17:03.203Z caller=node_exporter.go:117 level=info collector=schedstat 2024-02-13T14:17:03.272 INFO:journalctl@ceph.node-exporter.b.smithi148.stdout:Feb 13 14:17:03 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-node-exporter-b[63998]: ts=2024-02-13T14:17:03.203Z caller=node_exporter.go:117 level=info collector=selinux 2024-02-13T14:17:03.273 INFO:journalctl@ceph.node-exporter.b.smithi148.stdout:Feb 13 14:17:03 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-node-exporter-b[63998]: ts=2024-02-13T14:17:03.203Z caller=node_exporter.go:117 level=info collector=sockstat 2024-02-13T14:17:03.273 INFO:journalctl@ceph.node-exporter.b.smithi148.stdout:Feb 13 14:17:03 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-node-exporter-b[63998]: ts=2024-02-13T14:17:03.203Z caller=node_exporter.go:117 level=info collector=softnet 2024-02-13T14:17:03.273 INFO:journalctl@ceph.node-exporter.b.smithi148.stdout:Feb 13 14:17:03 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-node-exporter-b[63998]: ts=2024-02-13T14:17:03.203Z caller=node_exporter.go:117 level=info collector=stat 2024-02-13T14:17:03.273 INFO:journalctl@ceph.node-exporter.b.smithi148.stdout:Feb 13 14:17:03 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-node-exporter-b[63998]: ts=2024-02-13T14:17:03.203Z caller=node_exporter.go:117 level=info collector=tapestats 2024-02-13T14:17:03.273 INFO:journalctl@ceph.node-exporter.b.smithi148.stdout:Feb 13 14:17:03 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-node-exporter-b[63998]: ts=2024-02-13T14:17:03.203Z caller=node_exporter.go:117 level=info collector=textfile 2024-02-13T14:17:03.273 INFO:journalctl@ceph.node-exporter.b.smithi148.stdout:Feb 13 14:17:03 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-node-exporter-b[63998]: ts=2024-02-13T14:17:03.203Z caller=node_exporter.go:117 level=info collector=thermal_zone 2024-02-13T14:17:03.273 INFO:journalctl@ceph.node-exporter.b.smithi148.stdout:Feb 13 14:17:03 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-node-exporter-b[63998]: ts=2024-02-13T14:17:03.203Z caller=node_exporter.go:117 level=info collector=time 2024-02-13T14:17:03.273 INFO:journalctl@ceph.node-exporter.b.smithi148.stdout:Feb 13 14:17:03 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-node-exporter-b[63998]: ts=2024-02-13T14:17:03.203Z caller=node_exporter.go:117 level=info collector=udp_queues 2024-02-13T14:17:03.273 INFO:journalctl@ceph.node-exporter.b.smithi148.stdout:Feb 13 14:17:03 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-node-exporter-b[63998]: ts=2024-02-13T14:17:03.203Z caller=node_exporter.go:117 level=info collector=uname 2024-02-13T14:17:03.273 INFO:journalctl@ceph.node-exporter.b.smithi148.stdout:Feb 13 14:17:03 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-node-exporter-b[63998]: ts=2024-02-13T14:17:03.203Z caller=node_exporter.go:117 level=info collector=vmstat 2024-02-13T14:17:03.273 INFO:journalctl@ceph.node-exporter.b.smithi148.stdout:Feb 13 14:17:03 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-node-exporter-b[63998]: ts=2024-02-13T14:17:03.203Z caller=node_exporter.go:117 level=info collector=xfs 2024-02-13T14:17:03.273 INFO:journalctl@ceph.node-exporter.b.smithi148.stdout:Feb 13 14:17:03 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-node-exporter-b[63998]: ts=2024-02-13T14:17:03.203Z caller=node_exporter.go:117 level=info collector=zfs 2024-02-13T14:17:03.273 INFO:journalctl@ceph.node-exporter.b.smithi148.stdout:Feb 13 14:17:03 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-node-exporter-b[63998]: ts=2024-02-13T14:17:03.203Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9100 2024-02-13T14:17:03.274 INFO:journalctl@ceph.node-exporter.b.smithi148.stdout:Feb 13 14:17:03 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-node-exporter-b[63998]: ts=2024-02-13T14:17:03.204Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9100 2024-02-13T14:17:03.274 INFO:journalctl@ceph.node-exporter.b.smithi148.stdout:Feb 13 14:17:03 smithi148 bash[63946]: d7d208c49b58e9e0c15345a8dd874a1b8e1f3222f758b3be3e848a81fd094bc1 2024-02-13T14:17:03.491 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:17:03 smithi087 ceph-mon[27372]: pgmap v37: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-13T14:17:03.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:17:03 smithi087 ceph-mon[39120]: pgmap v37: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-13T14:17:03.618 INFO:journalctl@ceph.node-exporter.b.smithi148.stdout:Feb 13 14:17:03 smithi148 systemd[1]: Started Ceph node-exporter.b for 5c04b074-ca79-11ee-95b9-87774f69a715. 2024-02-13T14:17:04.573 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:17:04 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:17:04.573 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:17:04 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:17:04.574 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:17:04 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:17:04.574 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:17:04 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:17:04.574 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:17:04 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:17:04.574 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:17:04 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:17:04.574 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:17:04 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:17:04.574 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:17:04 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:17:04.574 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:17:04 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:17:04.574 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:17:04 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:17:04.574 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:17:04 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:17:04.574 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:17:04 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:17:04.574 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:17:04 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:17:04.574 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:17:04 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:17:04.605 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:17:04 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:17:04.605 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:17:04 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:17:04.605 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:17:04 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:17:04.605 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:17:04 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:17:04.605 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:17:04 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:17:04.605 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:17:04 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:17:04.605 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:17:04 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:17:04.868 INFO:journalctl@ceph.iscsi.iscsi.a.smithi148.stdout:Feb 13 14:17:04 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-iscsi-iscsi-a[59644]: debug there is no tcmu-runner data available 2024-02-13T14:17:05.618 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:17:05 smithi087 ceph-mon[27372]: pgmap v38: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-13T14:17:05.618 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:17:05 smithi087 ceph-mon[39120]: pgmap v38: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-13T14:17:05.618 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:17:05 smithi148 ceph-mon[31198]: pgmap v38: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-13T14:17:06.491 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:17:06 smithi087 ceph-mon[27372]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:17:06.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:17:06 smithi087 ceph-mon[39120]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:17:06.618 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:17:06 smithi148 ceph-mon[31198]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:17:07.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:17:07 smithi087 ceph-mon[27372]: pgmap v39: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:17:07.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:17:07 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:17:07.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:17:07 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:17:07.492 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:17:07 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: ::ffff:172.21.15.148 - - [13/Feb/2024:14:17:07] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-13T14:17:07.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:17:07 smithi087 ceph-mon[39120]: pgmap v39: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:17:07.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:17:07 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:17:07.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:17:07 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:17:07.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:17:07 smithi148 ceph-mon[31198]: pgmap v39: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:17:07.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:17:07 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:17:07.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:17:07 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:17:09.118 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:17:08 smithi148 ceph-mon[31198]: pgmap v40: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-13T14:17:09.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:17:08 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:17:09.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:17:08 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:17:09.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:17:08 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:17:09.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:17:08 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:17:09.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:17:08 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:17:09.207 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:17:08 smithi087 ceph-mon[39120]: pgmap v40: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-13T14:17:09.207 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:17:08 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:17:09.207 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:17:08 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:17:09.207 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:17:08 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:17:09.208 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:17:08 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:17:09.208 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:17:08 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:17:09.208 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:17:08 smithi087 ceph-mon[27372]: pgmap v40: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-13T14:17:09.208 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:17:08 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:17:09.208 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:17:08 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:17:09.208 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:17:08 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:17:09.208 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:17:08 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:17:09.208 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:17:08 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:17:10.241 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:17:09 smithi087 ceph-mon[27372]: Reconfiguring alertmanager.a (dependencies changed)... 2024-02-13T14:17:10.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:17:09 smithi087 ceph-mon[27372]: Reconfiguring daemon alertmanager.a on smithi087 2024-02-13T14:17:10.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:17:09 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-13T14:17:10.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:17:09 smithi087 ceph-mon[39120]: Reconfiguring alertmanager.a (dependencies changed)... 2024-02-13T14:17:10.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:17:09 smithi087 ceph-mon[39120]: Reconfiguring daemon alertmanager.a on smithi087 2024-02-13T14:17:10.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:17:09 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-13T14:17:10.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:17:09 smithi148 ceph-mon[31198]: Reconfiguring alertmanager.a (dependencies changed)... 2024-02-13T14:17:10.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:17:09 smithi148 ceph-mon[31198]: Reconfiguring daemon alertmanager.a on smithi087 2024-02-13T14:17:10.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:17:09 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-13T14:17:11.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:17:10 smithi087 ceph-mon[27372]: pgmap v41: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:17:11.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:17:10 smithi087 ceph-mon[39120]: pgmap v41: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:17:11.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:17:10 smithi148 ceph-mon[31198]: pgmap v41: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:17:12.162 INFO:journalctl@ceph.alertmanager.a.smithi087.stdout:Feb 13 14:17:11 smithi087 systemd[1]: Stopping Ceph alertmanager.a for 5c04b074-ca79-11ee-95b9-87774f69a715... 2024-02-13T14:17:12.476 INFO:journalctl@ceph.alertmanager.a.smithi087.stdout:Feb 13 14:17:12 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-alertmanager-a[76658]: ts=2024-02-13T14:17:12.160Z caller=main.go:583 level=info msg="Received SIGTERM, exiting gracefully..." 2024-02-13T14:17:12.476 INFO:journalctl@ceph.alertmanager.a.smithi087.stdout:Feb 13 14:17:12 smithi087 podman[78061]: 2024-02-13 14:17:12.181961893 +0000 UTC m=+0.141365279 container died 689561ab5d2d12cfe733ccf3a3d77c114b936f8b532c2b8c5fde4bdc05b6f4fe (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-02-13T14:17:12.741 INFO:journalctl@ceph.alertmanager.a.smithi087.stdout:Feb 13 14:17:12 smithi087 podman[78061]: 2024-02-13 14:17:12.691830326 +0000 UTC m=+0.651233705 container cleanup 689561ab5d2d12cfe733ccf3a3d77c114b936f8b532c2b8c5fde4bdc05b6f4fe (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-02-13T14:17:12.742 INFO:journalctl@ceph.alertmanager.a.smithi087.stdout:Feb 13 14:17:12 smithi087 bash[78061]: ceph-5c04b074-ca79-11ee-95b9-87774f69a715-alertmanager-a 2024-02-13T14:17:13.145 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:17:12 smithi087 ceph-mon[27372]: pgmap v42: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-13T14:17:13.146 INFO:journalctl@ceph.alertmanager.a.smithi087.stdout:Feb 13 14:17:12 smithi087 podman[78080]: 2024-02-13 14:17:12.901592637 +0000 UTC m=+0.722305961 container remove 689561ab5d2d12cfe733ccf3a3d77c114b936f8b532c2b8c5fde4bdc05b6f4fe (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-02-13T14:17:13.146 INFO:journalctl@ceph.alertmanager.a.smithi087.stdout:Feb 13 14:17:13 smithi087 podman[78080]: 2024-02-13 14:17:13.002235697 +0000 UTC m=+0.822949019 volume remove 940eb425956a5fe4ad554c8d929f3386aec685a7d9b95e212ee50c8e5de62c13 2024-02-13T14:17:13.146 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:17:12 smithi087 ceph-mon[39120]: pgmap v42: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-13T14:17:13.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:17:12 smithi148 ceph-mon[31198]: pgmap v42: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-13T14:17:13.492 INFO:journalctl@ceph.alertmanager.a.smithi087.stdout:Feb 13 14:17:13 smithi087 systemd[1]: ceph-5c04b074-ca79-11ee-95b9-87774f69a715@alertmanager.a.service: Deactivated successfully. 2024-02-13T14:17:13.492 INFO:journalctl@ceph.alertmanager.a.smithi087.stdout:Feb 13 14:17:13 smithi087 systemd[1]: Stopped Ceph alertmanager.a for 5c04b074-ca79-11ee-95b9-87774f69a715. 2024-02-13T14:17:13.492 INFO:journalctl@ceph.alertmanager.a.smithi087.stdout:Feb 13 14:17:13 smithi087 systemd[1]: Starting Ceph alertmanager.a for 5c04b074-ca79-11ee-95b9-87774f69a715... 2024-02-13T14:17:13.492 INFO:journalctl@ceph.alertmanager.a.smithi087.stdout:Feb 13 14:17:13 smithi087 podman[78183]: 2024-02-13 14:17:13.27529221 +0000 UTC m=+0.018376490 image pull c8568f914cd25b2062c44e9f79f9c18da6e3b85fe0c47a12a2191c61426c2b19 quay.io/prometheus/alertmanager:v0.25.0 2024-02-13T14:17:13.492 INFO:journalctl@ceph.alertmanager.a.smithi087.stdout:Feb 13 14:17:13 smithi087 podman[78183]: 2024-02-13 14:17:13.386645677 +0000 UTC m=+0.129729959 volume create a271d9f9aefb5aeb0c3be1410be445f36ff9ee7b92445149df260944afcf4da5 2024-02-13T14:17:13.755 INFO:journalctl@ceph.alertmanager.a.smithi087.stdout:Feb 13 14:17:13 smithi087 podman[78183]: 2024-02-13 14:17:13.4955229 +0000 UTC m=+0.238607200 container create 5ff9960e3532efa0da23b7e5ec58abb52f1db388d33b0836d78fde2c36a85582 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-02-13T14:17:14.242 INFO:journalctl@ceph.alertmanager.a.smithi087.stdout:Feb 13 14:17:13 smithi087 podman[78183]: 2024-02-13 14:17:13.873656261 +0000 UTC m=+0.616740560 container init 5ff9960e3532efa0da23b7e5ec58abb52f1db388d33b0836d78fde2c36a85582 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-02-13T14:17:14.242 INFO:journalctl@ceph.alertmanager.a.smithi087.stdout:Feb 13 14:17:13 smithi087 podman[78183]: 2024-02-13 14:17:13.879718806 +0000 UTC m=+0.622803090 container start 5ff9960e3532efa0da23b7e5ec58abb52f1db388d33b0836d78fde2c36a85582 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-02-13T14:17:14.242 INFO:journalctl@ceph.alertmanager.a.smithi087.stdout:Feb 13 14:17:13 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-alertmanager-a[78198]: ts=2024-02-13T14:17:13.891Z caller=main.go:240 level=info msg="Starting Alertmanager" version="(version=0.25.0, branch=HEAD, revision=258fab7cdd551f2cf251ed0348f0ad7289aee789)" 2024-02-13T14:17:14.242 INFO:journalctl@ceph.alertmanager.a.smithi087.stdout:Feb 13 14:17:13 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-alertmanager-a[78198]: ts=2024-02-13T14:17:13.891Z caller=main.go:241 level=info build_context="(go=go1.19.4, user=root@abe866dd5717, date=20221222-14:51:36)" 2024-02-13T14:17:14.242 INFO:journalctl@ceph.alertmanager.a.smithi087.stdout:Feb 13 14:17:13 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-alertmanager-a[78198]: ts=2024-02-13T14:17:13.892Z caller=cluster.go:185 level=info component=cluster msg="setting advertise address explicitly" addr=172.21.15.87 port=9094 2024-02-13T14:17:14.242 INFO:journalctl@ceph.alertmanager.a.smithi087.stdout:Feb 13 14:17:13 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-alertmanager-a[78198]: ts=2024-02-13T14:17:13.893Z caller=cluster.go:681 level=info component=cluster msg="Waiting for gossip to settle..." interval=2s 2024-02-13T14:17:14.242 INFO:journalctl@ceph.alertmanager.a.smithi087.stdout:Feb 13 14:17:13 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-alertmanager-a[78198]: ts=2024-02-13T14:17:13.916Z caller=coordinator.go:113 level=info component=configuration msg="Loading configuration file" file=/etc/alertmanager/alertmanager.yml 2024-02-13T14:17:14.242 INFO:journalctl@ceph.alertmanager.a.smithi087.stdout:Feb 13 14:17:13 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-alertmanager-a[78198]: ts=2024-02-13T14:17:13.917Z caller=coordinator.go:126 level=info component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/alertmanager.yml 2024-02-13T14:17:14.242 INFO:journalctl@ceph.alertmanager.a.smithi087.stdout:Feb 13 14:17:13 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-alertmanager-a[78198]: ts=2024-02-13T14:17:13.919Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9093 2024-02-13T14:17:14.243 INFO:journalctl@ceph.alertmanager.a.smithi087.stdout:Feb 13 14:17:13 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-alertmanager-a[78198]: ts=2024-02-13T14:17:13.919Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9093 2024-02-13T14:17:14.243 INFO:journalctl@ceph.alertmanager.a.smithi087.stdout:Feb 13 14:17:13 smithi087 bash[78183]: 5ff9960e3532efa0da23b7e5ec58abb52f1db388d33b0836d78fde2c36a85582 2024-02-13T14:17:14.243 INFO:journalctl@ceph.alertmanager.a.smithi087.stdout:Feb 13 14:17:13 smithi087 systemd[1]: Started Ceph alertmanager.a for 5c04b074-ca79-11ee-95b9-87774f69a715. 2024-02-13T14:17:15.116 INFO:journalctl@ceph.iscsi.iscsi.a.smithi148.stdout:Feb 13 14:17:14 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-iscsi-iscsi-a[59644]: debug there is no tcmu-runner data available 2024-02-13T14:17:15.116 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:17:15 smithi148 ceph-mon[31198]: pgmap v43: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-13T14:17:15.116 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:17:15 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:17:15.116 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:17:15 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:17:15.116 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:17:15 smithi148 ceph-mon[31198]: Reconfiguring prometheus.a (dependencies changed)... 2024-02-13T14:17:15.116 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:17:15 smithi148 ceph-mon[31198]: Reconfiguring daemon prometheus.a on smithi148 2024-02-13T14:17:15.491 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:17:15 smithi087 ceph-mon[39120]: pgmap v43: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-13T14:17:15.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:17:15 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:17:15.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:17:15 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:17:15.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:17:15 smithi087 ceph-mon[39120]: Reconfiguring prometheus.a (dependencies changed)... 2024-02-13T14:17:15.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:17:15 smithi087 ceph-mon[39120]: Reconfiguring daemon prometheus.a on smithi148 2024-02-13T14:17:15.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:17:15 smithi087 ceph-mon[27372]: pgmap v43: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-13T14:17:15.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:17:15 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:17:15.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:17:15 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:17:15.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:17:15 smithi087 ceph-mon[27372]: Reconfiguring prometheus.a (dependencies changed)... 2024-02-13T14:17:15.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:17:15 smithi087 ceph-mon[27372]: Reconfiguring daemon prometheus.a on smithi148 2024-02-13T14:17:16.241 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:17:16 smithi087 ceph-mon[27372]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:17:16.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:17:16 smithi087 ceph-mon[39120]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:17:16.242 INFO:journalctl@ceph.alertmanager.a.smithi087.stdout:Feb 13 14:17:15 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-alertmanager-a[78198]: ts=2024-02-13T14:17:15.894Z caller=cluster.go:706 level=info component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.00071987s 2024-02-13T14:17:16.288 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:17:16 smithi148 ceph-mon[31198]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:17:17.277 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:17:17 smithi148 ceph-mon[31198]: pgmap v44: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:17:17.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:17:17 smithi087 ceph-mon[27372]: pgmap v44: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:17:17.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:17:17 smithi087 ceph-mon[39120]: pgmap v44: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:17:17.492 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:17:17 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: ::ffff:172.21.15.148 - - [13/Feb/2024:14:17:17] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-13T14:17:17.600 INFO:journalctl@ceph.prometheus.a.smithi148.stdout:Feb 13 14:17:17 smithi148 systemd[1]: Stopping Ceph prometheus.a for 5c04b074-ca79-11ee-95b9-87774f69a715... 2024-02-13T14:17:17.869 INFO:journalctl@ceph.prometheus.a.smithi148.stdout:Feb 13 14:17:17 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-prometheus-a[61663]: ts=2024-02-13T14:17:17.598Z caller=main.go:840 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-02-13T14:17:17.869 INFO:journalctl@ceph.prometheus.a.smithi148.stdout:Feb 13 14:17:17 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-prometheus-a[61663]: ts=2024-02-13T14:17:17.599Z caller=main.go:864 level=info msg="Stopping scrape discovery manager..." 2024-02-13T14:17:17.869 INFO:journalctl@ceph.prometheus.a.smithi148.stdout:Feb 13 14:17:17 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-prometheus-a[61663]: ts=2024-02-13T14:17:17.599Z caller=main.go:878 level=info msg="Stopping notify discovery manager..." 2024-02-13T14:17:17.869 INFO:journalctl@ceph.prometheus.a.smithi148.stdout:Feb 13 14:17:17 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-prometheus-a[61663]: ts=2024-02-13T14:17:17.599Z caller=manager.go:988 level=info component="rule manager" msg="Stopping rule manager..." 2024-02-13T14:17:17.869 INFO:journalctl@ceph.prometheus.a.smithi148.stdout:Feb 13 14:17:17 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-prometheus-a[61663]: ts=2024-02-13T14:17:17.599Z caller=main.go:860 level=info msg="Scrape discovery manager stopped" 2024-02-13T14:17:17.869 INFO:journalctl@ceph.prometheus.a.smithi148.stdout:Feb 13 14:17:17 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-prometheus-a[61663]: ts=2024-02-13T14:17:17.599Z caller=main.go:874 level=info msg="Notify discovery manager stopped" 2024-02-13T14:17:17.869 INFO:journalctl@ceph.prometheus.a.smithi148.stdout:Feb 13 14:17:17 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-prometheus-a[61663]: ts=2024-02-13T14:17:17.599Z caller=manager.go:998 level=info component="rule manager" msg="Rule manager stopped" 2024-02-13T14:17:17.869 INFO:journalctl@ceph.prometheus.a.smithi148.stdout:Feb 13 14:17:17 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-prometheus-a[61663]: ts=2024-02-13T14:17:17.599Z caller=main.go:915 level=info msg="Stopping scrape manager..." 2024-02-13T14:17:17.870 INFO:journalctl@ceph.prometheus.a.smithi148.stdout:Feb 13 14:17:17 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-prometheus-a[61663]: ts=2024-02-13T14:17:17.599Z caller=main.go:907 level=info msg="Scrape manager stopped" 2024-02-13T14:17:17.870 INFO:journalctl@ceph.prometheus.a.smithi148.stdout:Feb 13 14:17:17 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-prometheus-a[61663]: ts=2024-02-13T14:17:17.599Z caller=notifier.go:606 level=info component=notifier msg="Stopping notification manager..." 2024-02-13T14:17:17.870 INFO:journalctl@ceph.prometheus.a.smithi148.stdout:Feb 13 14:17:17 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-prometheus-a[61663]: ts=2024-02-13T14:17:17.599Z caller=main.go:1135 level=info msg="Notifier manager stopped" 2024-02-13T14:17:17.870 INFO:journalctl@ceph.prometheus.a.smithi148.stdout:Feb 13 14:17:17 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-prometheus-a[61663]: ts=2024-02-13T14:17:17.599Z caller=main.go:1147 level=info msg="See you next time!" 2024-02-13T14:17:17.870 INFO:journalctl@ceph.prometheus.a.smithi148.stdout:Feb 13 14:17:17 smithi148 podman[65026]: 2024-02-13 14:17:17.609457022 +0000 UTC m=+0.141187922 container died 73aa09db99e18dfa227340e972c88a5eee1f238d191d1b6934605dde7b54cb4b (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-02-13T14:17:18.265 INFO:journalctl@ceph.prometheus.a.smithi148.stdout:Feb 13 14:17:17 smithi148 podman[65026]: 2024-02-13 14:17:17.962828509 +0000 UTC m=+0.494559396 container cleanup 73aa09db99e18dfa227340e972c88a5eee1f238d191d1b6934605dde7b54cb4b (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-02-13T14:17:18.265 INFO:journalctl@ceph.prometheus.a.smithi148.stdout:Feb 13 14:17:17 smithi148 bash[65026]: ceph-5c04b074-ca79-11ee-95b9-87774f69a715-prometheus-a 2024-02-13T14:17:18.599 INFO:journalctl@ceph.prometheus.a.smithi148.stdout:Feb 13 14:17:18 smithi148 podman[65045]: 2024-02-13 14:17:18.26370188 +0000 UTC m=+0.652656564 container remove 73aa09db99e18dfa227340e972c88a5eee1f238d191d1b6934605dde7b54cb4b (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-02-13T14:17:18.599 INFO:journalctl@ceph.prometheus.a.smithi148.stdout:Feb 13 14:17:18 smithi148 systemd[1]: ceph-5c04b074-ca79-11ee-95b9-87774f69a715@prometheus.a.service: Deactivated successfully. 2024-02-13T14:17:18.599 INFO:journalctl@ceph.prometheus.a.smithi148.stdout:Feb 13 14:17:18 smithi148 systemd[1]: Stopped Ceph prometheus.a for 5c04b074-ca79-11ee-95b9-87774f69a715. 2024-02-13T14:17:18.599 INFO:journalctl@ceph.prometheus.a.smithi148.stdout:Feb 13 14:17:18 smithi148 systemd[1]: ceph-5c04b074-ca79-11ee-95b9-87774f69a715@prometheus.a.service: Consumed 1.141s CPU time. 2024-02-13T14:17:18.599 INFO:journalctl@ceph.prometheus.a.smithi148.stdout:Feb 13 14:17:18 smithi148 systemd[1]: Starting Ceph prometheus.a for 5c04b074-ca79-11ee-95b9-87774f69a715... 2024-02-13T14:17:18.862 INFO:journalctl@ceph.prometheus.a.smithi148.stdout:Feb 13 14:17:18 smithi148 podman[65150]: 2024-02-13 14:17:18.598125673 +0000 UTC m=+0.113695609 container create 9e8219ab6c3626065eb826335a46d80b0311ba678f304b58499cbf5213ec454e (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-02-13T14:17:18.862 INFO:journalctl@ceph.prometheus.a.smithi148.stdout:Feb 13 14:17:18 smithi148 podman[65150]: 2024-02-13 14:17:18.50386069 +0000 UTC m=+0.019430635 image pull a07b618ecd1dce142bce4c52f0e80982eaf1f14265a2415c2d35978ccaa0a464 quay.io/prometheus/prometheus:v2.43.0 2024-02-13T14:17:18.862 INFO:journalctl@ceph.prometheus.a.smithi148.stdout:Feb 13 14:17:18 smithi148 podman[65150]: 2024-02-13 14:17:18.808770145 +0000 UTC m=+0.324340094 container init 9e8219ab6c3626065eb826335a46d80b0311ba678f304b58499cbf5213ec454e (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-02-13T14:17:18.863 INFO:journalctl@ceph.prometheus.a.smithi148.stdout:Feb 13 14:17:18 smithi148 podman[65150]: 2024-02-13 14:17:18.814047471 +0000 UTC m=+0.329617408 container start 9e8219ab6c3626065eb826335a46d80b0311ba678f304b58499cbf5213ec454e (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-02-13T14:17:18.863 INFO:journalctl@ceph.prometheus.a.smithi148.stdout:Feb 13 14:17:18 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-prometheus-a[65166]: ts=2024-02-13T14:17:18.837Z caller=main.go:564 level=info msg="Starting Prometheus Server" mode=server version="(version=2.43.0, branch=HEAD, revision=edfc3bcd025dd6fe296c167a14a216cab1e552ee)" 2024-02-13T14:17:18.863 INFO:journalctl@ceph.prometheus.a.smithi148.stdout:Feb 13 14:17:18 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-prometheus-a[65166]: ts=2024-02-13T14:17:18.837Z caller=main.go:569 level=info build_context="(go=go1.19.7, platform=linux/amd64, user=root@8a0ee342e522, date=20230321-12:56:07, tags=netgo,builtinassets)" 2024-02-13T14:17:18.863 INFO:journalctl@ceph.prometheus.a.smithi148.stdout:Feb 13 14:17:18 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-prometheus-a[65166]: ts=2024-02-13T14:17:18.837Z caller=main.go:570 level=info host_details="(Linux 5.14.0-419.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Wed Feb 7 23:01:41 UTC 2024 x86_64 smithi148 (none))" 2024-02-13T14:17:18.863 INFO:journalctl@ceph.prometheus.a.smithi148.stdout:Feb 13 14:17:18 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-prometheus-a[65166]: ts=2024-02-13T14:17:18.837Z caller=main.go:571 level=info fd_limits="(soft=1048576, hard=1048576)" 2024-02-13T14:17:18.863 INFO:journalctl@ceph.prometheus.a.smithi148.stdout:Feb 13 14:17:18 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-prometheus-a[65166]: ts=2024-02-13T14:17:18.837Z caller=main.go:572 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2024-02-13T14:17:18.863 INFO:journalctl@ceph.prometheus.a.smithi148.stdout:Feb 13 14:17:18 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-prometheus-a[65166]: ts=2024-02-13T14:17:18.839Z caller=web.go:561 level=info component=web msg="Start listening for connections" address=:9095 2024-02-13T14:17:18.863 INFO:journalctl@ceph.prometheus.a.smithi148.stdout:Feb 13 14:17:18 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-prometheus-a[65166]: ts=2024-02-13T14:17:18.839Z caller=main.go:1005 level=info msg="Starting TSDB ..." 2024-02-13T14:17:18.863 INFO:journalctl@ceph.prometheus.a.smithi148.stdout:Feb 13 14:17:18 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-prometheus-a[65166]: ts=2024-02-13T14:17:18.840Z caller=tls_config.go:232 level=info component=web msg="Listening on" address=[::]:9095 2024-02-13T14:17:18.863 INFO:journalctl@ceph.prometheus.a.smithi148.stdout:Feb 13 14:17:18 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-prometheus-a[65166]: ts=2024-02-13T14:17:18.840Z caller=tls_config.go:235 level=info component=web msg="TLS is disabled." http2=false address=[::]:9095 2024-02-13T14:17:18.863 INFO:journalctl@ceph.prometheus.a.smithi148.stdout:Feb 13 14:17:18 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-prometheus-a[65166]: ts=2024-02-13T14:17:18.842Z caller=head.go:587 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2024-02-13T14:17:18.863 INFO:journalctl@ceph.prometheus.a.smithi148.stdout:Feb 13 14:17:18 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-prometheus-a[65166]: ts=2024-02-13T14:17:18.842Z caller=head.go:658 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=1.513µs 2024-02-13T14:17:18.863 INFO:journalctl@ceph.prometheus.a.smithi148.stdout:Feb 13 14:17:18 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-prometheus-a[65166]: ts=2024-02-13T14:17:18.842Z caller=head.go:664 level=info component=tsdb msg="Replaying WAL, this may take a while" 2024-02-13T14:17:18.863 INFO:journalctl@ceph.prometheus.a.smithi148.stdout:Feb 13 14:17:18 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-prometheus-a[65166]: ts=2024-02-13T14:17:18.842Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=1 2024-02-13T14:17:18.864 INFO:journalctl@ceph.prometheus.a.smithi148.stdout:Feb 13 14:17:18 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-prometheus-a[65166]: ts=2024-02-13T14:17:18.843Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=1 maxSegment=1 2024-02-13T14:17:18.864 INFO:journalctl@ceph.prometheus.a.smithi148.stdout:Feb 13 14:17:18 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-prometheus-a[65166]: ts=2024-02-13T14:17:18.843Z caller=head.go:772 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=17.347µs wal_replay_duration=1.574807ms wbl_replay_duration=140ns total_replay_duration=1.609782ms 2024-02-13T14:17:18.864 INFO:journalctl@ceph.prometheus.a.smithi148.stdout:Feb 13 14:17:18 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-prometheus-a[65166]: ts=2024-02-13T14:17:18.844Z caller=main.go:1026 level=info fs_type=XFS_SUPER_MAGIC 2024-02-13T14:17:18.864 INFO:journalctl@ceph.prometheus.a.smithi148.stdout:Feb 13 14:17:18 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-prometheus-a[65166]: ts=2024-02-13T14:17:18.844Z caller=main.go:1029 level=info msg="TSDB started" 2024-02-13T14:17:18.864 INFO:journalctl@ceph.prometheus.a.smithi148.stdout:Feb 13 14:17:18 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-prometheus-a[65166]: ts=2024-02-13T14:17:18.844Z caller=main.go:1209 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2024-02-13T14:17:18.864 INFO:journalctl@ceph.prometheus.a.smithi148.stdout:Feb 13 14:17:18 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-prometheus-a[65166]: ts=2024-02-13T14:17:18.861Z caller=main.go:1246 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=16.487172ms db_storage=695ns remote_storage=1.026µs web_handler=406ns query_engine=488ns scrape=204.779µs scrape_sd=73.258µs notify=10.045µs notify_sd=9.588µs rules=15.914495ms tracing=4.993µs 2024-02-13T14:17:18.864 INFO:journalctl@ceph.prometheus.a.smithi148.stdout:Feb 13 14:17:18 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-prometheus-a[65166]: ts=2024-02-13T14:17:18.861Z caller=main.go:990 level=info msg="Server is ready to receive web requests." 2024-02-13T14:17:18.864 INFO:journalctl@ceph.prometheus.a.smithi148.stdout:Feb 13 14:17:18 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-prometheus-a[65166]: ts=2024-02-13T14:17:18.861Z caller=manager.go:974 level=info component="rule manager" msg="Starting rule manager..." 2024-02-13T14:17:19.119 INFO:journalctl@ceph.prometheus.a.smithi148.stdout:Feb 13 14:17:18 smithi148 bash[65150]: 9e8219ab6c3626065eb826335a46d80b0311ba678f304b58499cbf5213ec454e 2024-02-13T14:17:19.119 INFO:journalctl@ceph.prometheus.a.smithi148.stdout:Feb 13 14:17:18 smithi148 systemd[1]: Started Ceph prometheus.a for 5c04b074-ca79-11ee-95b9-87774f69a715. 2024-02-13T14:17:19.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:17:19 smithi148 ceph-mon[31198]: pgmap v45: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-13T14:17:19.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:17:19 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:17:19.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:17:19 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:17:19.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:17:19 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-13T14:17:19.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:17:19 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi087.front.sepia.ceph.com:9093"}]: dispatch 2024-02-13T14:17:19.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:17:19 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:17:19.120 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:17:19 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-13T14:17:19.120 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:17:19 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi148.front.sepia.ceph.com:9095"}]: dispatch 2024-02-13T14:17:19.120 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:17:19 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:17:19.120 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:17:19 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-13T14:17:19.120 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:17:19 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi148.front.sepia.ceph.com:3000"}]: dispatch 2024-02-13T14:17:19.120 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:17:19 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:17:19.238 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:17:19 smithi087 ceph-mon[39120]: pgmap v45: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-13T14:17:19.238 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:17:19 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:17:19.238 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:17:19 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:17:19.238 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:17:19 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-13T14:17:19.238 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:17:19 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi087.front.sepia.ceph.com:9093"}]: dispatch 2024-02-13T14:17:19.238 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:17:19 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:17:19.238 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:17:19 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-13T14:17:19.238 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:17:19 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi148.front.sepia.ceph.com:9095"}]: dispatch 2024-02-13T14:17:19.238 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:17:19 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:17:19.239 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:17:19 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-13T14:17:19.239 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:17:19 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi148.front.sepia.ceph.com:3000"}]: dispatch 2024-02-13T14:17:19.239 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:17:19 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:17:19.239 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:17:19 smithi087 ceph-mon[27372]: pgmap v45: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-13T14:17:19.239 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:17:19 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:17:19.239 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:17:19 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:17:19.239 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:17:19 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-13T14:17:19.239 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:17:19 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi087.front.sepia.ceph.com:9093"}]: dispatch 2024-02-13T14:17:19.239 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:17:19 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:17:19.239 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:17:19 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-13T14:17:19.239 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:17:19 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi148.front.sepia.ceph.com:9095"}]: dispatch 2024-02-13T14:17:19.240 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:17:19 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:17:19.240 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:17:19 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-13T14:17:19.240 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:17:19 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi148.front.sepia.ceph.com:3000"}]: dispatch 2024-02-13T14:17:19.240 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:17:19 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:17:19.240 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:17:18 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: [13/Feb/2024:14:17:18] ENGINE Bus STOPPING 2024-02-13T14:17:19.240 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:17:19 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: [13/Feb/2024:14:17:19] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-02-13T14:17:19.240 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:17:19 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: [13/Feb/2024:14:17:19] ENGINE Bus STOPPED 2024-02-13T14:17:19.240 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:17:19 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: [13/Feb/2024:14:17:19] ENGINE Bus STARTING 2024-02-13T14:17:19.240 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:17:19 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: [13/Feb/2024:14:17:19] ENGINE Serving on http://:::9283 2024-02-13T14:17:19.240 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:17:19 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: [13/Feb/2024:14:17:19] ENGINE Bus STARTED 2024-02-13T14:17:19.241 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:17:19 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: [13/Feb/2024:14:17:19] ENGINE Bus STOPPING 2024-02-13T14:17:19.241 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:17:19 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: [13/Feb/2024:14:17:19] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-02-13T14:17:19.241 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:17:19 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: [13/Feb/2024:14:17:19] ENGINE Bus STOPPED 2024-02-13T14:17:19.241 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:17:19 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: [13/Feb/2024:14:17:19] ENGINE Bus STARTING 2024-02-13T14:17:19.241 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:17:19 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: [13/Feb/2024:14:17:19] ENGINE Serving on http://:::9283 2024-02-13T14:17:19.241 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:17:19 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: [13/Feb/2024:14:17:19] ENGINE Bus STARTED 2024-02-13T14:17:19.241 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:17:19 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: [13/Feb/2024:14:17:19] ENGINE Bus STOPPING 2024-02-13T14:17:19.241 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:17:19 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: [13/Feb/2024:14:17:19] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-02-13T14:17:19.241 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:17:19 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: [13/Feb/2024:14:17:19] ENGINE Bus STOPPED 2024-02-13T14:17:19.241 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:17:19 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: [13/Feb/2024:14:17:19] ENGINE Bus STARTING 2024-02-13T14:17:19.492 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:17:19 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: [13/Feb/2024:14:17:19] ENGINE Serving on http://:::9283 2024-02-13T14:17:19.492 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:17:19 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: [13/Feb/2024:14:17:19] ENGINE Bus STARTED 2024-02-13T14:17:20.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:17:20 smithi148 ceph-mon[31198]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-13T14:17:20.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:17:20 smithi148 ceph-mon[31198]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi087.front.sepia.ceph.com:9093"}]: dispatch 2024-02-13T14:17:20.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:17:20 smithi148 ceph-mon[31198]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-13T14:17:20.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:17:20 smithi148 ceph-mon[31198]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi148.front.sepia.ceph.com:9095"}]: dispatch 2024-02-13T14:17:20.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:17:20 smithi148 ceph-mon[31198]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-13T14:17:20.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:17:20 smithi148 ceph-mon[31198]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi148.front.sepia.ceph.com:3000"}]: dispatch 2024-02-13T14:17:20.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:17:20 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:17:20.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:17:20 smithi087 ceph-mon[27372]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-13T14:17:20.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:17:20 smithi087 ceph-mon[27372]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi087.front.sepia.ceph.com:9093"}]: dispatch 2024-02-13T14:17:20.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:17:20 smithi087 ceph-mon[27372]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-13T14:17:20.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:17:20 smithi087 ceph-mon[27372]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi148.front.sepia.ceph.com:9095"}]: dispatch 2024-02-13T14:17:20.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:17:20 smithi087 ceph-mon[27372]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-13T14:17:20.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:17:20 smithi087 ceph-mon[27372]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi148.front.sepia.ceph.com:3000"}]: dispatch 2024-02-13T14:17:20.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:17:20 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:17:20.243 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:17:20 smithi087 ceph-mon[39120]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-13T14:17:20.243 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:17:20 smithi087 ceph-mon[39120]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi087.front.sepia.ceph.com:9093"}]: dispatch 2024-02-13T14:17:20.243 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:17:20 smithi087 ceph-mon[39120]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-13T14:17:20.243 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:17:20 smithi087 ceph-mon[39120]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi148.front.sepia.ceph.com:9095"}]: dispatch 2024-02-13T14:17:20.243 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:17:20 smithi087 ceph-mon[39120]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-13T14:17:20.243 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:17:20 smithi087 ceph-mon[39120]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi148.front.sepia.ceph.com:3000"}]: dispatch 2024-02-13T14:17:20.243 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:17:20 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:17:21.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:17:21 smithi148 ceph-mon[31198]: pgmap v46: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:17:21.420 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:17:21 smithi087 ceph-mon[27372]: pgmap v46: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:17:21.420 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:17:21 smithi087 ceph-mon[39120]: pgmap v46: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:17:23.618 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:17:23 smithi148 ceph-mon[31198]: pgmap v47: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-13T14:17:23.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:17:23 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:17:23.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:17:23 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:17:23.741 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:17:23 smithi087 ceph-mon[39120]: pgmap v47: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-13T14:17:23.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:17:23 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:17:23.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:17:23 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:17:23.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:17:23 smithi087 ceph-mon[27372]: pgmap v47: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-13T14:17:23.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:17:23 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:17:23.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:17:23 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:17:24.241 INFO:journalctl@ceph.alertmanager.a.smithi087.stdout:Feb 13 14:17:23 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-alertmanager-a[78198]: ts=2024-02-13T14:17:23.896Z caller=cluster.go:698 level=info component=cluster msg="gossip settled; proceeding" elapsed=10.003331815s 2024-02-13T14:17:24.618 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:17:24 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-13T14:17:24.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:17:24 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-13T14:17:24.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:17:24 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-13T14:17:25.118 INFO:journalctl@ceph.iscsi.iscsi.a.smithi148.stdout:Feb 13 14:17:24 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-iscsi-iscsi-a[59644]: debug there is no tcmu-runner data available 2024-02-13T14:17:25.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:17:25 smithi148 ceph-mon[31198]: pgmap v48: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-13T14:17:25.741 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:17:25 smithi087 ceph-mon[27372]: pgmap v48: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-13T14:17:25.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:17:25 smithi087 ceph-mon[39120]: pgmap v48: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-13T14:17:26.741 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:17:26 smithi087 ceph-mon[27372]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:17:26.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:17:26 smithi087 ceph-mon[39120]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:17:26.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:17:26 smithi148 ceph-mon[31198]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:17:27.491 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:17:27 smithi087 ceph-mon[27372]: pgmap v49: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:17:27.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:17:27 smithi087 ceph-mon[39120]: pgmap v49: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:17:27.492 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:17:27 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: ::ffff:172.21.15.148 - - [13/Feb/2024:14:17:27] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-13T14:17:27.791 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:17:27 smithi148 ceph-mon[31198]: pgmap v49: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:17:29.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:17:29 smithi087 ceph-mon[39120]: pgmap v50: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-13T14:17:29.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:17:29 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:17:29.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:17:29 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:17:29.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:17:29 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:17:29.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:17:29 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:17:29.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:17:29 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:17:29.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:17:29 smithi087 ceph-mon[27372]: pgmap v50: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-13T14:17:29.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:17:29 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:17:29.743 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:17:29 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:17:29.743 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:17:29 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:17:29.743 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:17:29 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:17:29.743 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:17:29 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:17:29.868 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:17:29 smithi148 ceph-mon[31198]: pgmap v50: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-13T14:17:29.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:17:29 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:17:29.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:17:29 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:17:29.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:17:29 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:17:29.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:17:29 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:17:29.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:17:29 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:17:30.741 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:17:30 smithi087 ceph-mon[39120]: pgmap v51: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:17:30.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:17:30 smithi087 ceph-mon[27372]: pgmap v51: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:17:30.868 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:17:30 smithi148 ceph-mon[31198]: pgmap v51: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:17:32.814 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:17:32 smithi087 ceph-mon[39120]: pgmap v52: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-13T14:17:33.118 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:17:32 smithi148 ceph-mon[31198]: pgmap v52: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-13T14:17:33.241 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:17:32 smithi087 ceph-mon[27372]: pgmap v52: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-13T14:17:35.118 INFO:journalctl@ceph.iscsi.iscsi.a.smithi148.stdout:Feb 13 14:17:34 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-iscsi-iscsi-a[59644]: debug there is no tcmu-runner data available 2024-02-13T14:17:35.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:17:34 smithi148 ceph-mon[31198]: pgmap v53: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-13T14:17:35.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:17:34 smithi087 ceph-mon[39120]: pgmap v53: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-13T14:17:35.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:17:34 smithi087 ceph-mon[27372]: pgmap v53: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-13T14:17:36.118 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:17:35 smithi148 ceph-mon[31198]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:17:36.241 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:17:35 smithi087 ceph-mon[27372]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:17:36.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:17:35 smithi087 ceph-mon[39120]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:17:37.118 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:17:36 smithi148 ceph-mon[31198]: pgmap v54: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:17:37.220 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:17:36 smithi087 ceph-mon[39120]: pgmap v54: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:17:37.221 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:17:36 smithi087 ceph-mon[27372]: pgmap v54: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:17:37.491 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:17:37 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: ::ffff:172.21.15.148 - - [13/Feb/2024:14:17:37] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-13T14:17:39.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:17:38 smithi148 ceph-mon[31198]: pgmap v55: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-13T14:17:39.241 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:17:38 smithi087 ceph-mon[39120]: pgmap v55: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-13T14:17:39.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:17:38 smithi087 ceph-mon[27372]: pgmap v55: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-13T14:17:40.118 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:17:39 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-13T14:17:40.241 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:17:39 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-13T14:17:40.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:17:39 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-13T14:17:41.118 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:17:40 smithi148 ceph-mon[31198]: pgmap v56: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:17:41.241 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:17:40 smithi087 ceph-mon[39120]: pgmap v56: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:17:41.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:17:40 smithi087 ceph-mon[27372]: pgmap v56: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:17:43.118 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:17:42 smithi148 ceph-mon[31198]: pgmap v57: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-13T14:17:43.241 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:17:42 smithi087 ceph-mon[27372]: pgmap v57: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-13T14:17:43.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:17:42 smithi087 ceph-mon[39120]: pgmap v57: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-13T14:17:45.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:17:44 smithi148 ceph-mon[31198]: pgmap v58: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-13T14:17:45.119 INFO:journalctl@ceph.iscsi.iscsi.a.smithi148.stdout:Feb 13 14:17:44 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-iscsi-iscsi-a[59644]: debug there is no tcmu-runner data available 2024-02-13T14:17:45.241 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:17:44 smithi087 ceph-mon[39120]: pgmap v58: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-13T14:17:45.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:17:44 smithi087 ceph-mon[27372]: pgmap v58: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-13T14:17:46.118 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:17:45 smithi148 ceph-mon[31198]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:17:46.241 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:17:45 smithi087 ceph-mon[27372]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:17:46.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:17:45 smithi087 ceph-mon[39120]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:17:47.118 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:17:46 smithi148 ceph-mon[31198]: pgmap v59: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:17:47.221 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:17:46 smithi087 ceph-mon[27372]: pgmap v59: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:17:47.221 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:17:46 smithi087 ceph-mon[39120]: pgmap v59: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:17:47.491 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:17:47 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: ::ffff:172.21.15.148 - - [13/Feb/2024:14:17:47] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-13T14:17:49.118 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:17:48 smithi148 ceph-mon[31198]: pgmap v60: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-13T14:17:49.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:17:48 smithi087 ceph-mon[27372]: pgmap v60: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-13T14:17:49.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:17:48 smithi087 ceph-mon[39120]: pgmap v60: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-13T14:17:51.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:17:50 smithi148 ceph-mon[31198]: pgmap v61: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:17:51.241 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:17:50 smithi087 ceph-mon[27372]: pgmap v61: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:17:51.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:17:50 smithi087 ceph-mon[39120]: pgmap v61: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:17:53.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:17:52 smithi148 ceph-mon[31198]: pgmap v62: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-13T14:17:53.241 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:17:52 smithi087 ceph-mon[27372]: pgmap v62: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-13T14:17:53.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:17:52 smithi087 ceph-mon[39120]: pgmap v62: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-13T14:17:53.531 INFO:tasks.workunit.client.0.smithi087.stderr:Note: switching to 'e1b79d5a4193a484176863ab96c0ca28c00e1662'. 2024-02-13T14:17:53.531 INFO:tasks.workunit.client.0.smithi087.stderr: 2024-02-13T14:17:53.531 INFO:tasks.workunit.client.0.smithi087.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-02-13T14:17:53.531 INFO:tasks.workunit.client.0.smithi087.stderr:changes and commit them, and you can discard any commits you make in this 2024-02-13T14:17:53.531 INFO:tasks.workunit.client.0.smithi087.stderr:state without impacting any branches by switching back to a branch. 2024-02-13T14:17:53.531 INFO:tasks.workunit.client.0.smithi087.stderr: 2024-02-13T14:17:53.531 INFO:tasks.workunit.client.0.smithi087.stderr:If you want to create a new branch to retain commits you create, you may 2024-02-13T14:17:53.531 INFO:tasks.workunit.client.0.smithi087.stderr:do so (now or later) by using -c with the switch command. Example: 2024-02-13T14:17:53.531 INFO:tasks.workunit.client.0.smithi087.stderr: 2024-02-13T14:17:53.532 INFO:tasks.workunit.client.0.smithi087.stderr: git switch -c 2024-02-13T14:17:53.532 INFO:tasks.workunit.client.0.smithi087.stderr: 2024-02-13T14:17:53.532 INFO:tasks.workunit.client.0.smithi087.stderr:Or undo this operation with: 2024-02-13T14:17:53.532 INFO:tasks.workunit.client.0.smithi087.stderr: 2024-02-13T14:17:53.532 INFO:tasks.workunit.client.0.smithi087.stderr: git switch - 2024-02-13T14:17:53.532 INFO:tasks.workunit.client.0.smithi087.stderr: 2024-02-13T14:17:53.532 INFO:tasks.workunit.client.0.smithi087.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-02-13T14:17:53.532 INFO:tasks.workunit.client.0.smithi087.stderr: 2024-02-13T14:17:53.532 INFO:tasks.workunit.client.0.smithi087.stderr:HEAD is now at e1b79d5a419 Merge branch 'jjm-cephadm-smb-svc-spec' of https://github.com/phlogistonjohn/ceph into wip-phlogistonjohn-testing-2024-02-12-1654 2024-02-13T14:17:53.534 DEBUG:teuthology.orchestra.run.smithi087:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-02-13T14:17:53.556 INFO:tasks.workunit.client.0.smithi087.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-02-13T14:17:53.559 INFO:tasks.workunit.client.0.smithi087.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-02-13T14:17:53.559 INFO:tasks.workunit.client.0.smithi087.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-02-13T14:17:54.288 INFO:tasks.workunit.client.0.smithi087.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-02-13T14:17:54.357 INFO:tasks.workunit.client.0.smithi087.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-02-13T14:17:54.397 INFO:tasks.workunit.client.0.smithi087.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-02-13T14:17:54.398 INFO:tasks.workunit.client.0.smithi087.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-02-13T14:17:54.398 INFO:tasks.workunit.client.0.smithi087.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-02-13T14:17:54.437 INFO:tasks.workunit.client.0.smithi087.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-02-13T14:17:54.440 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-02-13T14:17:54.440 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-02-13T14:17:54.497 INFO:tasks.workunit:Running workunits matching rados/test_python.sh on client.0... 2024-02-13T14:17:54.498 INFO:tasks.workunit:Running workunit rados/test_python.sh... 2024-02-13T14:17:54.498 DEBUG:teuthology.orchestra.run.smithi087:workunit test rados/test_python.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=e1b79d5a4193a484176863ab96c0ca28c00e1662 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 1h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test_python.sh 2024-02-13T14:17:54.560 INFO:tasks.workunit.client.0.smithi087.stderr:+ ceph osd pool create rbd 2024-02-13T14:17:55.119 INFO:journalctl@ceph.iscsi.iscsi.a.smithi148.stdout:Feb 13 14:17:54 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-iscsi-iscsi-a[59644]: debug there is no tcmu-runner data available 2024-02-13T14:17:55.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:17:55 smithi148 ceph-mon[31198]: pgmap v63: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-13T14:17:55.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:17:55 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-13T14:17:55.222 INFO:tasks.workunit.client.0.smithi087.stderr:pool 'rbd' already exists 2024-02-13T14:17:55.234 INFO:tasks.workunit.client.0.smithi087.stderr:++ dirname /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test_python.sh 2024-02-13T14:17:55.234 INFO:tasks.workunit.client.0.smithi087.stderr:+ python3 -m pytest -v /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/../../../src/test/pybind/test_rados.py 2024-02-13T14:17:55.402 INFO:tasks.workunit.client.0.smithi087.stdout:============================= test session starts ============================== 2024-02-13T14:17:55.403 INFO:tasks.workunit.client.0.smithi087.stdout:platform linux -- Python 3.9.17, pytest-6.2.2, py-1.10.0, pluggy-0.13.1 -- /usr/bin/python3 2024-02-13T14:17:55.403 INFO:tasks.workunit.client.0.smithi087.stdout:cachedir: .pytest_cache 2024-02-13T14:17:55.403 INFO:tasks.workunit.client.0.smithi087.stdout:rootdir: /home/ubuntu/cephtest/clone.client.0/src/test/pybind, configfile: pytest.ini 2024-02-13T14:17:55.491 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:17:55 smithi087 ceph-mon[27372]: pgmap v63: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-13T14:17:55.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:17:55 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-13T14:17:55.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:17:55 smithi087 ceph-mon[39120]: pgmap v63: 132 pgs: 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-13T14:17:55.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:17:55 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-13T14:17:55.659 INFO:tasks.workunit.client.0.smithi087.stdout:collecting ... collected 90 items 2024-02-13T14:17:55.659 INFO:tasks.workunit.client.0.smithi087.stdout: 2024-02-13T14:17:55.668 INFO:tasks.workunit.client.0.smithi087.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::test_rados_init_error PASSED [ 1%] 2024-02-13T14:17:55.718 INFO:tasks.workunit.client.0.smithi087.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::test_rados_init PASSED [ 2%] 2024-02-13T14:17:55.733 INFO:tasks.workunit.client.0.smithi087.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::test_ioctx_context_manager PASSED [ 3%] 2024-02-13T14:17:55.745 INFO:tasks.workunit.client.0.smithi087.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::test_parse_argv PASSED [ 4%] 2024-02-13T14:17:55.751 INFO:tasks.workunit.client.0.smithi087.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::test_parse_argv_empty_str PASSED [ 5%] 2024-02-13T14:17:55.759 INFO:tasks.workunit.client.0.smithi087.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestRadosStateError::test_configuring PASSED [ 6%] 2024-02-13T14:17:55.773 INFO:tasks.workunit.client.0.smithi087.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestRadosStateError::test_connected PASSED [ 7%] 2024-02-13T14:17:55.786 INFO:tasks.workunit.client.0.smithi087.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestRadosStateError::test_shutdown PASSED [ 8%] 2024-02-13T14:17:55.807 INFO:tasks.workunit.client.0.smithi087.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestRados::test_ping_monitor PASSED [ 10%] 2024-02-13T14:17:55.825 INFO:tasks.workunit.client.0.smithi087.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestRados::test_annotations PASSED [ 11%] 2024-02-13T14:17:56.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:17:56 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/4197708862' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "rbd"}]: dispatch 2024-02-13T14:17:56.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:17:56 smithi148 ceph-mon[31198]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "rbd"}]: dispatch 2024-02-13T14:17:56.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:17:56 smithi148 ceph-mon[31198]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:17:56.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:17:56 smithi148 ceph-mon[31198]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "rbd"}]': finished 2024-02-13T14:17:56.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:17:56 smithi148 ceph-mon[31198]: osdmap e75: 8 total, 8 up, 8 in 2024-02-13T14:17:56.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:17:56 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/4197708862' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "rbd"}]: dispatch 2024-02-13T14:17:56.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:17:56 smithi148 ceph-mon[31198]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "rbd"}]: dispatch 2024-02-13T14:17:56.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:17:56 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/1558917467' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-13T14:17:56.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:17:56 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/4197708862' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "rbd"}]: dispatch 2024-02-13T14:17:56.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:17:56 smithi087 ceph-mon[27372]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "rbd"}]: dispatch 2024-02-13T14:17:56.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:17:56 smithi087 ceph-mon[27372]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:17:56.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:17:56 smithi087 ceph-mon[27372]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "rbd"}]': finished 2024-02-13T14:17:56.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:17:56 smithi087 ceph-mon[27372]: osdmap e75: 8 total, 8 up, 8 in 2024-02-13T14:17:56.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:17:56 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/4197708862' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "rbd"}]: dispatch 2024-02-13T14:17:56.493 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:17:56 smithi087 ceph-mon[27372]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "rbd"}]: dispatch 2024-02-13T14:17:56.493 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:17:56 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/1558917467' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-13T14:17:56.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:17:56 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/4197708862' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "rbd"}]: dispatch 2024-02-13T14:17:56.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:17:56 smithi087 ceph-mon[39120]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "rbd"}]: dispatch 2024-02-13T14:17:56.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:17:56 smithi087 ceph-mon[39120]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:17:56.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:17:56 smithi087 ceph-mon[39120]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "rbd"}]': finished 2024-02-13T14:17:56.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:17:56 smithi087 ceph-mon[39120]: osdmap e75: 8 total, 8 up, 8 in 2024-02-13T14:17:56.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:17:56 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/4197708862' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "rbd"}]: dispatch 2024-02-13T14:17:56.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:17:56 smithi087 ceph-mon[39120]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "rbd"}]: dispatch 2024-02-13T14:17:56.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:17:56 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/1558917467' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-13T14:17:57.128 INFO:tasks.workunit.client.0.smithi087.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestRados::test_create PASSED [ 12%] 2024-02-13T14:17:57.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:17:57 smithi148 ceph-mon[31198]: pgmap v65: 164 pgs: 32 unknown, 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-13T14:17:57.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:17:57 smithi148 ceph-mon[31198]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T14:17:57.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:17:57 smithi148 ceph-mon[31198]: osdmap e76: 8 total, 8 up, 8 in 2024-02-13T14:17:57.491 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:17:57 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: ::ffff:172.21.15.148 - - [13/Feb/2024:14:17:57] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-13T14:17:57.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:17:57 smithi087 ceph-mon[27372]: pgmap v65: 164 pgs: 32 unknown, 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-13T14:17:57.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:17:57 smithi087 ceph-mon[27372]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T14:17:57.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:17:57 smithi087 ceph-mon[27372]: osdmap e76: 8 total, 8 up, 8 in 2024-02-13T14:17:57.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:17:57 smithi087 ceph-mon[39120]: pgmap v65: 164 pgs: 32 unknown, 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-13T14:17:57.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:17:57 smithi087 ceph-mon[39120]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T14:17:57.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:17:57 smithi087 ceph-mon[39120]: osdmap e76: 8 total, 8 up, 8 in 2024-02-13T14:17:58.491 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:17:58 smithi087 ceph-mon[27372]: osdmap e77: 8 total, 8 up, 8 in 2024-02-13T14:17:58.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:17:58 smithi087 ceph-mon[39120]: osdmap e77: 8 total, 8 up, 8 in 2024-02-13T14:17:58.618 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:17:58 smithi148 ceph-mon[31198]: osdmap e77: 8 total, 8 up, 8 in 2024-02-13T14:17:59.136 INFO:tasks.workunit.client.0.smithi087.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestRados::test_create_utf8 PASSED [ 13%] 2024-02-13T14:17:59.491 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:17:59 smithi087 ceph-mon[27372]: pgmap v68: 164 pgs: 32 unknown, 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-13T14:17:59.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:17:59 smithi087 ceph-mon[27372]: osdmap e78: 8 total, 8 up, 8 in 2024-02-13T14:17:59.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:17:59 smithi087 ceph-mon[39120]: pgmap v68: 164 pgs: 32 unknown, 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-13T14:17:59.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:17:59 smithi087 ceph-mon[39120]: osdmap e78: 8 total, 8 up, 8 in 2024-02-13T14:17:59.618 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:17:59 smithi148 ceph-mon[31198]: pgmap v68: 164 pgs: 32 unknown, 132 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-13T14:17:59.618 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:17:59 smithi148 ceph-mon[31198]: osdmap e78: 8 total, 8 up, 8 in 2024-02-13T14:18:00.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:00 smithi087 ceph-mon[27372]: osdmap e79: 8 total, 8 up, 8 in 2024-02-13T14:18:00.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:00 smithi087 ceph-mon[39120]: osdmap e79: 8 total, 8 up, 8 in 2024-02-13T14:18:00.618 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:00 smithi148 ceph-mon[31198]: osdmap e79: 8 total, 8 up, 8 in 2024-02-13T14:18:01.154 INFO:tasks.workunit.client.0.smithi087.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestRados::test_pool_lookup_utf8 PASSED [ 14%] 2024-02-13T14:18:01.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:01 smithi087 ceph-mon[27372]: pgmap v71: 164 pgs: 164 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:18:01.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:01 smithi087 ceph-mon[27372]: osdmap e80: 8 total, 8 up, 8 in 2024-02-13T14:18:01.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:01 smithi087 ceph-mon[39120]: pgmap v71: 164 pgs: 164 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:18:01.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:01 smithi087 ceph-mon[39120]: osdmap e80: 8 total, 8 up, 8 in 2024-02-13T14:18:01.618 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:01 smithi148 ceph-mon[31198]: pgmap v71: 164 pgs: 164 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:18:01.618 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:01 smithi148 ceph-mon[31198]: osdmap e80: 8 total, 8 up, 8 in 2024-02-13T14:18:02.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:02 smithi087 ceph-mon[27372]: osdmap e81: 8 total, 8 up, 8 in 2024-02-13T14:18:02.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:02 smithi087 ceph-mon[39120]: osdmap e81: 8 total, 8 up, 8 in 2024-02-13T14:18:02.618 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:02 smithi148 ceph-mon[31198]: osdmap e81: 8 total, 8 up, 8 in 2024-02-13T14:18:03.172 INFO:tasks.workunit.client.0.smithi087.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestRados::test_eexist PASSED [ 15%] 2024-02-13T14:18:03.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:03 smithi087 ceph-mon[27372]: pgmap v74: 164 pgs: 164 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:18:03.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:03 smithi087 ceph-mon[27372]: osdmap e82: 8 total, 8 up, 8 in 2024-02-13T14:18:03.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:03 smithi087 ceph-mon[39120]: pgmap v74: 164 pgs: 164 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:18:03.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:03 smithi087 ceph-mon[39120]: osdmap e82: 8 total, 8 up, 8 in 2024-02-13T14:18:03.618 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:03 smithi148 ceph-mon[31198]: pgmap v74: 164 pgs: 164 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:18:03.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:03 smithi148 ceph-mon[31198]: osdmap e82: 8 total, 8 up, 8 in 2024-02-13T14:18:04.491 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:04 smithi087 ceph-mon[27372]: osdmap e83: 8 total, 8 up, 8 in 2024-02-13T14:18:04.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:04 smithi087 ceph-mon[39120]: osdmap e83: 8 total, 8 up, 8 in 2024-02-13T14:18:04.618 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:04 smithi148 ceph-mon[31198]: osdmap e83: 8 total, 8 up, 8 in 2024-02-13T14:18:05.118 INFO:journalctl@ceph.iscsi.iscsi.a.smithi148.stdout:Feb 13 14:18:04 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-iscsi-iscsi-a[59644]: debug there is no tcmu-runner data available 2024-02-13T14:18:05.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:05 smithi087 ceph-mon[27372]: pgmap v77: 164 pgs: 164 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-02-13T14:18:05.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:05 smithi087 ceph-mon[27372]: osdmap e84: 8 total, 8 up, 8 in 2024-02-13T14:18:05.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:05 smithi087 ceph-mon[39120]: pgmap v77: 164 pgs: 164 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-02-13T14:18:05.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:05 smithi087 ceph-mon[39120]: osdmap e84: 8 total, 8 up, 8 in 2024-02-13T14:18:05.618 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:05 smithi148 ceph-mon[31198]: pgmap v77: 164 pgs: 164 active+clean; 582 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-02-13T14:18:05.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:05 smithi148 ceph-mon[31198]: osdmap e84: 8 total, 8 up, 8 in 2024-02-13T14:18:06.491 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:06 smithi087 ceph-mon[39120]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:18:06.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:06 smithi087 ceph-mon[39120]: osdmap e85: 8 total, 8 up, 8 in 2024-02-13T14:18:06.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:06 smithi087 ceph-mon[27372]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:18:06.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:06 smithi087 ceph-mon[27372]: osdmap e85: 8 total, 8 up, 8 in 2024-02-13T14:18:06.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:06 smithi148 ceph-mon[31198]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:18:06.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:06 smithi148 ceph-mon[31198]: osdmap e85: 8 total, 8 up, 8 in 2024-02-13T14:18:07.492 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:18:07 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: ::ffff:172.21.15.148 - - [13/Feb/2024:14:18:07] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-13T14:18:07.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:07 smithi087 ceph-mon[27372]: pgmap v80: 228 pgs: 32 unknown, 32 creating+peering, 164 active+clean; 582 KiB data, 245 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:18:07.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:07 smithi087 ceph-mon[27372]: osdmap e86: 8 total, 8 up, 8 in 2024-02-13T14:18:07.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:07 smithi087 ceph-mon[27372]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T14:18:07.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:07 smithi087 ceph-mon[27372]: osdmap e87: 8 total, 8 up, 8 in 2024-02-13T14:18:07.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:07 smithi087 ceph-mon[39120]: pgmap v80: 228 pgs: 32 unknown, 32 creating+peering, 164 active+clean; 582 KiB data, 245 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:18:07.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:07 smithi087 ceph-mon[39120]: osdmap e86: 8 total, 8 up, 8 in 2024-02-13T14:18:07.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:07 smithi087 ceph-mon[39120]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T14:18:07.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:07 smithi087 ceph-mon[39120]: osdmap e87: 8 total, 8 up, 8 in 2024-02-13T14:18:07.618 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:07 smithi148 ceph-mon[31198]: pgmap v80: 228 pgs: 32 unknown, 32 creating+peering, 164 active+clean; 582 KiB data, 245 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:18:07.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:07 smithi148 ceph-mon[31198]: osdmap e86: 8 total, 8 up, 8 in 2024-02-13T14:18:07.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:07 smithi148 ceph-mon[31198]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T14:18:07.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:07 smithi148 ceph-mon[31198]: osdmap e87: 8 total, 8 up, 8 in 2024-02-13T14:18:09.491 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:09 smithi087 ceph-mon[27372]: pgmap v83: 228 pgs: 64 unknown, 164 active+clean; 582 KiB data, 245 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:18:09.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:09 smithi087 ceph-mon[27372]: osdmap e88: 8 total, 8 up, 8 in 2024-02-13T14:18:09.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:09 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:18:09.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:09 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-13T14:18:09.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:09 smithi087 ceph-mon[39120]: pgmap v83: 228 pgs: 64 unknown, 164 active+clean; 582 KiB data, 245 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:18:09.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:09 smithi087 ceph-mon[39120]: osdmap e88: 8 total, 8 up, 8 in 2024-02-13T14:18:09.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:09 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:18:09.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:09 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-13T14:18:09.618 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:09 smithi148 ceph-mon[31198]: pgmap v83: 228 pgs: 64 unknown, 164 active+clean; 582 KiB data, 245 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:18:09.618 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:09 smithi148 ceph-mon[31198]: osdmap e88: 8 total, 8 up, 8 in 2024-02-13T14:18:09.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:09 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:18:09.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:09 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-13T14:18:10.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:10 smithi087 ceph-mon[27372]: osdmap e89: 8 total, 8 up, 8 in 2024-02-13T14:18:10.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:10 smithi087 ceph-mon[39120]: osdmap e89: 8 total, 8 up, 8 in 2024-02-13T14:18:10.618 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:10 smithi148 ceph-mon[31198]: osdmap e89: 8 total, 8 up, 8 in 2024-02-13T14:18:11.224 INFO:tasks.workunit.client.0.smithi087.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestRados::test_list_pools PASSED [ 16%] 2024-02-13T14:18:11.491 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:11 smithi087 ceph-mon[27372]: pgmap v86: 164 pgs: 164 active+clean; 582 KiB data, 249 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:18:11.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:11 smithi087 ceph-mon[27372]: osdmap e90: 8 total, 8 up, 8 in 2024-02-13T14:18:11.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:11 smithi087 ceph-mon[39120]: pgmap v86: 164 pgs: 164 active+clean; 582 KiB data, 249 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:18:11.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:11 smithi087 ceph-mon[39120]: osdmap e90: 8 total, 8 up, 8 in 2024-02-13T14:18:11.618 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:11 smithi148 ceph-mon[31198]: pgmap v86: 164 pgs: 164 active+clean; 582 KiB data, 249 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:18:11.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:11 smithi148 ceph-mon[31198]: osdmap e90: 8 total, 8 up, 8 in 2024-02-13T14:18:12.491 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:12 smithi087 ceph-mon[27372]: osdmap e91: 8 total, 8 up, 8 in 2024-02-13T14:18:12.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:12 smithi087 ceph-mon[39120]: osdmap e91: 8 total, 8 up, 8 in 2024-02-13T14:18:12.618 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:12 smithi148 ceph-mon[31198]: osdmap e91: 8 total, 8 up, 8 in 2024-02-13T14:18:13.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:13 smithi087 ceph-mon[27372]: pgmap v89: 164 pgs: 164 active+clean; 582 KiB data, 249 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:18:13.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:13 smithi087 ceph-mon[27372]: osdmap e92: 8 total, 8 up, 8 in 2024-02-13T14:18:13.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:13 smithi087 ceph-mon[39120]: pgmap v89: 164 pgs: 164 active+clean; 582 KiB data, 249 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:18:13.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:13 smithi087 ceph-mon[39120]: osdmap e92: 8 total, 8 up, 8 in 2024-02-13T14:18:13.618 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:13 smithi148 ceph-mon[31198]: pgmap v89: 164 pgs: 164 active+clean; 582 KiB data, 249 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:18:13.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:13 smithi148 ceph-mon[31198]: osdmap e92: 8 total, 8 up, 8 in 2024-02-13T14:18:14.618 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:14 smithi148 ceph-mon[31198]: osdmap e93: 8 total, 8 up, 8 in 2024-02-13T14:18:14.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:14 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/934496227' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "foo", "tierpool": "foo-cache", "force_nonempty": ""}]: dispatch 2024-02-13T14:18:14.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:14 smithi148 ceph-mon[31198]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "foo", "tierpool": "foo-cache", "force_nonempty": ""}]: dispatch 2024-02-13T14:18:14.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:14 smithi148 ceph-mon[31198]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "foo", "tierpool": "foo-cache", "force_nonempty": ""}]': finished 2024-02-13T14:18:14.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:14 smithi148 ceph-mon[31198]: osdmap e94: 8 total, 8 up, 8 in 2024-02-13T14:18:14.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:14 smithi087 ceph-mon[27372]: osdmap e93: 8 total, 8 up, 8 in 2024-02-13T14:18:14.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:14 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/934496227' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "foo", "tierpool": "foo-cache", "force_nonempty": ""}]: dispatch 2024-02-13T14:18:14.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:14 smithi087 ceph-mon[27372]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "foo", "tierpool": "foo-cache", "force_nonempty": ""}]: dispatch 2024-02-13T14:18:14.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:14 smithi087 ceph-mon[27372]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "foo", "tierpool": "foo-cache", "force_nonempty": ""}]': finished 2024-02-13T14:18:14.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:14 smithi087 ceph-mon[27372]: osdmap e94: 8 total, 8 up, 8 in 2024-02-13T14:18:14.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:14 smithi087 ceph-mon[39120]: osdmap e93: 8 total, 8 up, 8 in 2024-02-13T14:18:14.743 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:14 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/934496227' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "foo", "tierpool": "foo-cache", "force_nonempty": ""}]: dispatch 2024-02-13T14:18:14.743 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:14 smithi087 ceph-mon[39120]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "foo", "tierpool": "foo-cache", "force_nonempty": ""}]: dispatch 2024-02-13T14:18:14.743 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:14 smithi087 ceph-mon[39120]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "foo", "tierpool": "foo-cache", "force_nonempty": ""}]': finished 2024-02-13T14:18:14.743 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:14 smithi087 ceph-mon[39120]: osdmap e94: 8 total, 8 up, 8 in 2024-02-13T14:18:15.118 INFO:journalctl@ceph.iscsi.iscsi.a.smithi148.stdout:Feb 13 14:18:14 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-iscsi-iscsi-a[59644]: debug there is no tcmu-runner data available 2024-02-13T14:18:15.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:15 smithi148 ceph-mon[31198]: pgmap v92: 228 pgs: 64 unknown, 164 active+clean; 582 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-02-13T14:18:15.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:15 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/934496227' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "foo-cache", "tierpool": "foo-cache", "mode": "readonly", "yes_i_really_mean_it": true}]: dispatch 2024-02-13T14:18:15.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:15 smithi148 ceph-mon[31198]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T14:18:15.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:15 smithi148 ceph-mon[31198]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "foo-cache", "tierpool": "foo-cache", "mode": "readonly", "yes_i_really_mean_it": true}]: dispatch 2024-02-13T14:18:15.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:15 smithi148 ceph-mon[31198]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "foo-cache", "tierpool": "foo-cache", "mode": "readonly", "yes_i_really_mean_it": true}]': finished 2024-02-13T14:18:15.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:15 smithi148 ceph-mon[31198]: osdmap e95: 8 total, 8 up, 8 in 2024-02-13T14:18:15.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:15 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/934496227' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "foo", "tierpool": "foo-cache"}]: dispatch 2024-02-13T14:18:15.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:15 smithi148 ceph-mon[31198]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "foo", "tierpool": "foo-cache"}]: dispatch 2024-02-13T14:18:15.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:15 smithi087 ceph-mon[27372]: pgmap v92: 228 pgs: 64 unknown, 164 active+clean; 582 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-02-13T14:18:15.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:15 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/934496227' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "foo-cache", "tierpool": "foo-cache", "mode": "readonly", "yes_i_really_mean_it": true}]: dispatch 2024-02-13T14:18:15.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:15 smithi087 ceph-mon[27372]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T14:18:15.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:15 smithi087 ceph-mon[27372]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "foo-cache", "tierpool": "foo-cache", "mode": "readonly", "yes_i_really_mean_it": true}]: dispatch 2024-02-13T14:18:15.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:15 smithi087 ceph-mon[27372]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "foo-cache", "tierpool": "foo-cache", "mode": "readonly", "yes_i_really_mean_it": true}]': finished 2024-02-13T14:18:15.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:15 smithi087 ceph-mon[27372]: osdmap e95: 8 total, 8 up, 8 in 2024-02-13T14:18:15.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:15 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/934496227' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "foo", "tierpool": "foo-cache"}]: dispatch 2024-02-13T14:18:15.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:15 smithi087 ceph-mon[27372]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "foo", "tierpool": "foo-cache"}]: dispatch 2024-02-13T14:18:15.743 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:15 smithi087 ceph-mon[39120]: pgmap v92: 228 pgs: 64 unknown, 164 active+clean; 582 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-02-13T14:18:15.743 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:15 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/934496227' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "foo-cache", "tierpool": "foo-cache", "mode": "readonly", "yes_i_really_mean_it": true}]: dispatch 2024-02-13T14:18:15.743 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:15 smithi087 ceph-mon[39120]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T14:18:15.743 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:15 smithi087 ceph-mon[39120]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "foo-cache", "tierpool": "foo-cache", "mode": "readonly", "yes_i_really_mean_it": true}]: dispatch 2024-02-13T14:18:15.743 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:15 smithi087 ceph-mon[39120]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "foo-cache", "tierpool": "foo-cache", "mode": "readonly", "yes_i_really_mean_it": true}]': finished 2024-02-13T14:18:15.743 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:15 smithi087 ceph-mon[39120]: osdmap e95: 8 total, 8 up, 8 in 2024-02-13T14:18:15.743 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:15 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/934496227' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "foo", "tierpool": "foo-cache"}]: dispatch 2024-02-13T14:18:15.743 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:15 smithi087 ceph-mon[39120]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "foo", "tierpool": "foo-cache"}]: dispatch 2024-02-13T14:18:16.618 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:16 smithi148 ceph-mon[31198]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:18:16.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:16 smithi148 ceph-mon[31198]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "foo", "tierpool": "foo-cache"}]': finished 2024-02-13T14:18:16.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:16 smithi148 ceph-mon[31198]: osdmap e96: 8 total, 8 up, 8 in 2024-02-13T14:18:16.741 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:16 smithi087 ceph-mon[39120]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:18:16.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:16 smithi087 ceph-mon[39120]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "foo", "tierpool": "foo-cache"}]': finished 2024-02-13T14:18:16.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:16 smithi087 ceph-mon[39120]: osdmap e96: 8 total, 8 up, 8 in 2024-02-13T14:18:16.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:16 smithi087 ceph-mon[27372]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:18:16.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:16 smithi087 ceph-mon[27372]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "foo", "tierpool": "foo-cache"}]': finished 2024-02-13T14:18:16.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:16 smithi087 ceph-mon[27372]: osdmap e96: 8 total, 8 up, 8 in 2024-02-13T14:18:17.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:17 smithi087 ceph-mon[27372]: pgmap v95: 228 pgs: 228 active+clean; 582 KiB data, 250 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:18:17.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:17 smithi087 ceph-mon[27372]: osdmap e97: 8 total, 8 up, 8 in 2024-02-13T14:18:17.492 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:18:17 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: ::ffff:172.21.15.148 - - [13/Feb/2024:14:18:17] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-13T14:18:17.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:17 smithi087 ceph-mon[39120]: pgmap v95: 228 pgs: 228 active+clean; 582 KiB data, 250 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:18:17.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:17 smithi087 ceph-mon[39120]: osdmap e97: 8 total, 8 up, 8 in 2024-02-13T14:18:17.618 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:17 smithi148 ceph-mon[31198]: pgmap v95: 228 pgs: 228 active+clean; 582 KiB data, 250 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:18:17.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:17 smithi148 ceph-mon[31198]: osdmap e97: 8 total, 8 up, 8 in 2024-02-13T14:18:18.242 INFO:tasks.workunit.client.0.smithi087.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestRados::test_get_pool_base_tier PASSED [ 17%] 2024-02-13T14:18:18.263 INFO:tasks.workunit.client.0.smithi087.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestRados::test_get_fsid PASSED [ 18%] 2024-02-13T14:18:19.268 INFO:tasks.workunit.client.0.smithi087.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestRados::test_blocklist_add PASSED [ 20%] 2024-02-13T14:18:19.287 INFO:tasks.workunit.client.0.smithi087.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestRados::test_get_cluster_stats PASSED [ 21%] 2024-02-13T14:18:19.306 INFO:tasks.workunit.client.0.smithi087.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestRados::test_monitor_log PASSED [ 22%] 2024-02-13T14:18:19.491 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:19 smithi087 ceph-mon[39120]: pgmap v98: 196 pgs: 196 active+clean; 582 KiB data, 250 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:18:19.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:19 smithi087 ceph-mon[39120]: osdmap e98: 8 total, 8 up, 8 in 2024-02-13T14:18:19.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:19 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/1600665618' entity='client.admin' cmd=[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "1.2.3.4/123", "expire": 1.0}]: dispatch 2024-02-13T14:18:19.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:19 smithi087 ceph-mon[39120]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "1.2.3.4/123", "expire": 1.0}]: dispatch 2024-02-13T14:18:19.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:19 smithi087 ceph-mon[27372]: pgmap v98: 196 pgs: 196 active+clean; 582 KiB data, 250 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:18:19.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:19 smithi087 ceph-mon[27372]: osdmap e98: 8 total, 8 up, 8 in 2024-02-13T14:18:19.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:19 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/1600665618' entity='client.admin' cmd=[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "1.2.3.4/123", "expire": 1.0}]: dispatch 2024-02-13T14:18:19.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:19 smithi087 ceph-mon[27372]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "1.2.3.4/123", "expire": 1.0}]: dispatch 2024-02-13T14:18:19.618 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:19 smithi148 ceph-mon[31198]: pgmap v98: 196 pgs: 196 active+clean; 582 KiB data, 250 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:18:19.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:19 smithi148 ceph-mon[31198]: osdmap e98: 8 total, 8 up, 8 in 2024-02-13T14:18:19.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:19 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/1600665618' entity='client.admin' cmd=[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "1.2.3.4/123", "expire": 1.0}]: dispatch 2024-02-13T14:18:19.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:19 smithi148 ceph-mon[31198]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "1.2.3.4/123", "expire": 1.0}]: dispatch 2024-02-13T14:18:20.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:20 smithi148 ceph-mon[31198]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "1.2.3.4/123", "expire": 1.0}]': finished 2024-02-13T14:18:20.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:20 smithi148 ceph-mon[31198]: osdmap e99: 8 total, 8 up, 8 in 2024-02-13T14:18:20.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:20 smithi087 ceph-mon[27372]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "1.2.3.4/123", "expire": 1.0}]': finished 2024-02-13T14:18:20.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:20 smithi087 ceph-mon[27372]: osdmap e99: 8 total, 8 up, 8 in 2024-02-13T14:18:20.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:20 smithi087 ceph-mon[39120]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd blocklist", "blocklistop": "add", "addr": "1.2.3.4/123", "expire": 1.0}]': finished 2024-02-13T14:18:20.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:20 smithi087 ceph-mon[39120]: osdmap e99: 8 total, 8 up, 8 in 2024-02-13T14:18:21.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:21 smithi148 ceph-mon[31198]: pgmap v101: 164 pgs: 164 active+clean; 582 KiB data, 250 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:18:21.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:21 smithi148 ceph-mon[31198]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T14:18:21.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:21 smithi148 ceph-mon[31198]: osdmap e100: 8 total, 8 up, 8 in 2024-02-13T14:18:21.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:21 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/3454145641' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:18:21.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:21 smithi148 ceph-mon[31198]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:18:21.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:21 smithi148 ceph-mon[31198]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:18:21.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:21 smithi148 ceph-mon[31198]: osdmap e101: 8 total, 8 up, 8 in 2024-02-13T14:18:21.643 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:21 smithi087 ceph-mon[27372]: pgmap v101: 164 pgs: 164 active+clean; 582 KiB data, 250 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:18:21.643 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:21 smithi087 ceph-mon[27372]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T14:18:21.643 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:21 smithi087 ceph-mon[27372]: osdmap e100: 8 total, 8 up, 8 in 2024-02-13T14:18:21.643 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:21 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/3454145641' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:18:21.643 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:21 smithi087 ceph-mon[27372]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:18:21.644 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:21 smithi087 ceph-mon[27372]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:18:21.644 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:21 smithi087 ceph-mon[27372]: osdmap e101: 8 total, 8 up, 8 in 2024-02-13T14:18:21.644 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:21 smithi087 ceph-mon[39120]: pgmap v101: 164 pgs: 164 active+clean; 582 KiB data, 250 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:18:21.644 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:21 smithi087 ceph-mon[39120]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T14:18:21.644 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:21 smithi087 ceph-mon[39120]: osdmap e100: 8 total, 8 up, 8 in 2024-02-13T14:18:21.644 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:21 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/3454145641' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:18:21.644 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:21 smithi087 ceph-mon[39120]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:18:21.644 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:21 smithi087 ceph-mon[39120]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:18:21.644 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:21 smithi087 ceph-mon[39120]: osdmap e101: 8 total, 8 up, 8 in 2024-02-13T14:18:21.823 INFO:tasks.workunit.client.0.smithi087.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_get_last_version PASSED [ 23%] 2024-02-13T14:18:23.118 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:22 smithi148 ceph-mon[31198]: pgmap v104: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 250 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:18:23.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:22 smithi148 ceph-mon[31198]: osdmap e102: 8 total, 8 up, 8 in 2024-02-13T14:18:23.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:22 smithi087 ceph-mon[27372]: pgmap v104: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 250 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:18:23.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:22 smithi087 ceph-mon[27372]: osdmap e102: 8 total, 8 up, 8 in 2024-02-13T14:18:23.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:22 smithi087 ceph-mon[39120]: pgmap v104: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 250 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:18:23.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:22 smithi087 ceph-mon[39120]: osdmap e102: 8 total, 8 up, 8 in 2024-02-13T14:18:24.118 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:23 smithi148 ceph-mon[31198]: osdmap e103: 8 total, 8 up, 8 in 2024-02-13T14:18:24.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:23 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/1819670399' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:18:24.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:23 smithi148 ceph-mon[31198]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:18:24.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:23 smithi148 ceph-mon[31198]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:18:24.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:23 smithi148 ceph-mon[31198]: osdmap e104: 8 total, 8 up, 8 in 2024-02-13T14:18:24.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:23 smithi087 ceph-mon[27372]: osdmap e103: 8 total, 8 up, 8 in 2024-02-13T14:18:24.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:23 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/1819670399' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:18:24.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:23 smithi087 ceph-mon[27372]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:18:24.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:23 smithi087 ceph-mon[27372]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:18:24.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:23 smithi087 ceph-mon[27372]: osdmap e104: 8 total, 8 up, 8 in 2024-02-13T14:18:24.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:23 smithi087 ceph-mon[39120]: osdmap e103: 8 total, 8 up, 8 in 2024-02-13T14:18:24.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:23 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/1819670399' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:18:24.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:23 smithi087 ceph-mon[39120]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:18:24.243 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:23 smithi087 ceph-mon[39120]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:18:24.243 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:23 smithi087 ceph-mon[39120]: osdmap e104: 8 total, 8 up, 8 in 2024-02-13T14:18:24.832 INFO:tasks.workunit.client.0.smithi087.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_get_stats PASSED [ 24%] 2024-02-13T14:18:25.119 INFO:journalctl@ceph.iscsi.iscsi.a.smithi148.stdout:Feb 13 14:18:24 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-iscsi-iscsi-a[59644]: debug there is no tcmu-runner data available 2024-02-13T14:18:25.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:24 smithi148 ceph-mon[31198]: pgmap v107: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-02-13T14:18:25.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:24 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-13T14:18:25.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:24 smithi148 ceph-mon[31198]: osdmap e105: 8 total, 8 up, 8 in 2024-02-13T14:18:25.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:24 smithi087 ceph-mon[27372]: pgmap v107: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-02-13T14:18:25.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:24 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-13T14:18:25.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:24 smithi087 ceph-mon[27372]: osdmap e105: 8 total, 8 up, 8 in 2024-02-13T14:18:25.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:24 smithi087 ceph-mon[39120]: pgmap v107: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-02-13T14:18:25.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:24 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-13T14:18:25.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:24 smithi087 ceph-mon[39120]: osdmap e105: 8 total, 8 up, 8 in 2024-02-13T14:18:26.118 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:25 smithi148 ceph-mon[31198]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:18:26.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:25 smithi148 ceph-mon[31198]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T14:18:26.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:25 smithi087 ceph-mon[39120]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:18:26.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:25 smithi087 ceph-mon[39120]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T14:18:26.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:25 smithi087 ceph-mon[27372]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:18:26.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:25 smithi087 ceph-mon[27372]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T14:18:27.118 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:26 smithi148 ceph-mon[31198]: pgmap v110: 164 pgs: 164 active+clean; 582 KiB data, 255 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:18:27.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:26 smithi148 ceph-mon[31198]: osdmap e106: 8 total, 8 up, 8 in 2024-02-13T14:18:27.221 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:26 smithi087 ceph-mon[27372]: pgmap v110: 164 pgs: 164 active+clean; 582 KiB data, 255 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:18:27.221 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:26 smithi087 ceph-mon[27372]: osdmap e106: 8 total, 8 up, 8 in 2024-02-13T14:18:27.222 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:26 smithi087 ceph-mon[39120]: pgmap v110: 164 pgs: 164 active+clean; 582 KiB data, 255 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:18:27.222 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:26 smithi087 ceph-mon[39120]: osdmap e106: 8 total, 8 up, 8 in 2024-02-13T14:18:27.492 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:18:27 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: ::ffff:172.21.15.148 - - [13/Feb/2024:14:18:27] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-13T14:18:28.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:27 smithi148 ceph-mon[31198]: osdmap e107: 8 total, 8 up, 8 in 2024-02-13T14:18:28.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:27 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/2442993882' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:18:28.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:27 smithi148 ceph-mon[31198]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:18:28.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:27 smithi087 ceph-mon[27372]: osdmap e107: 8 total, 8 up, 8 in 2024-02-13T14:18:28.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:27 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/2442993882' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:18:28.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:27 smithi087 ceph-mon[27372]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:18:28.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:27 smithi087 ceph-mon[39120]: osdmap e107: 8 total, 8 up, 8 in 2024-02-13T14:18:28.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:27 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/2442993882' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:18:28.243 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:27 smithi087 ceph-mon[39120]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:18:28.878 INFO:tasks.workunit.client.0.smithi087.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_write PASSED [ 25%] 2024-02-13T14:18:29.241 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:28 smithi087 ceph-mon[27372]: pgmap v113: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 255 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:18:29.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:28 smithi087 ceph-mon[27372]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:18:29.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:28 smithi087 ceph-mon[27372]: osdmap e108: 8 total, 8 up, 8 in 2024-02-13T14:18:29.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:28 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:18:29.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:28 smithi087 ceph-mon[39120]: pgmap v113: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 255 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:18:29.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:28 smithi087 ceph-mon[39120]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:18:29.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:28 smithi087 ceph-mon[39120]: osdmap e108: 8 total, 8 up, 8 in 2024-02-13T14:18:29.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:28 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:18:29.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:28 smithi148 ceph-mon[31198]: pgmap v113: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 255 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:18:29.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:28 smithi148 ceph-mon[31198]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:18:29.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:28 smithi148 ceph-mon[31198]: osdmap e108: 8 total, 8 up, 8 in 2024-02-13T14:18:29.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:28 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:18:30.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:29 smithi087 ceph-mon[39120]: osdmap e109: 8 total, 8 up, 8 in 2024-02-13T14:18:30.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:29 smithi087 ceph-mon[27372]: osdmap e109: 8 total, 8 up, 8 in 2024-02-13T14:18:30.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:29 smithi148 ceph-mon[31198]: osdmap e109: 8 total, 8 up, 8 in 2024-02-13T14:18:31.118 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:30 smithi148 ceph-mon[31198]: pgmap v116: 164 pgs: 164 active+clean; 582 KiB data, 256 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:18:31.118 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:30 smithi148 ceph-mon[31198]: osdmap e110: 8 total, 8 up, 8 in 2024-02-13T14:18:31.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:30 smithi148 ceph-mon[31198]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T14:18:31.241 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:30 smithi087 ceph-mon[27372]: pgmap v116: 164 pgs: 164 active+clean; 582 KiB data, 256 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:18:31.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:30 smithi087 ceph-mon[27372]: osdmap e110: 8 total, 8 up, 8 in 2024-02-13T14:18:31.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:30 smithi087 ceph-mon[27372]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T14:18:31.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:30 smithi087 ceph-mon[39120]: pgmap v116: 164 pgs: 164 active+clean; 582 KiB data, 256 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:18:31.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:30 smithi087 ceph-mon[39120]: osdmap e110: 8 total, 8 up, 8 in 2024-02-13T14:18:31.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:30 smithi087 ceph-mon[39120]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T14:18:32.202 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:31 smithi148 ceph-mon[31198]: osdmap e111: 8 total, 8 up, 8 in 2024-02-13T14:18:32.202 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:31 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/4188334446' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:18:32.202 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:31 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:18:32.202 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:31 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:18:32.241 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:31 smithi087 ceph-mon[39120]: osdmap e111: 8 total, 8 up, 8 in 2024-02-13T14:18:32.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:31 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/4188334446' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:18:32.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:31 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:18:32.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:31 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:18:32.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:31 smithi087 ceph-mon[27372]: osdmap e111: 8 total, 8 up, 8 in 2024-02-13T14:18:32.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:31 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/4188334446' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:18:32.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:31 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:18:32.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:31 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:18:32.906 INFO:tasks.workunit.client.0.smithi087.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_write_full PASSED [ 26%] 2024-02-13T14:18:33.189 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:32 smithi148 ceph-mon[31198]: pgmap v119: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 256 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:18:33.190 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:32 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/4188334446' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:18:33.190 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:32 smithi148 ceph-mon[31198]: osdmap e112: 8 total, 8 up, 8 in 2024-02-13T14:18:33.190 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:32 smithi148 ceph-mon[31198]: osdmap e113: 8 total, 8 up, 8 in 2024-02-13T14:18:33.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:32 smithi087 ceph-mon[27372]: pgmap v119: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 256 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:18:33.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:32 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/4188334446' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:18:33.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:32 smithi087 ceph-mon[27372]: osdmap e112: 8 total, 8 up, 8 in 2024-02-13T14:18:33.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:32 smithi087 ceph-mon[27372]: osdmap e113: 8 total, 8 up, 8 in 2024-02-13T14:18:33.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:32 smithi087 ceph-mon[39120]: pgmap v119: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 256 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:18:33.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:32 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/4188334446' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:18:33.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:32 smithi087 ceph-mon[39120]: osdmap e112: 8 total, 8 up, 8 in 2024-02-13T14:18:33.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:32 smithi087 ceph-mon[39120]: osdmap e113: 8 total, 8 up, 8 in 2024-02-13T14:18:34.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:34 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:18:34.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:34 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:18:34.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:34 smithi087 ceph-mon[27372]: pgmap v122: 164 pgs: 164 active+clean; 582 KiB data, 256 MiB used, 715 GiB / 715 GiB avail 2024-02-13T14:18:34.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:34 smithi087 ceph-mon[27372]: osdmap e114: 8 total, 8 up, 8 in 2024-02-13T14:18:34.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:34 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:18:34.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:34 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:18:34.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:34 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:18:34.743 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:34 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:18:34.743 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:34 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:18:34.743 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:34 smithi087 ceph-mon[39120]: pgmap v122: 164 pgs: 164 active+clean; 582 KiB data, 256 MiB used, 715 GiB / 715 GiB avail 2024-02-13T14:18:34.743 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:34 smithi087 ceph-mon[39120]: osdmap e114: 8 total, 8 up, 8 in 2024-02-13T14:18:34.743 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:34 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:18:34.743 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:34 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:18:34.743 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:34 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:18:34.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:34 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:18:34.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:34 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:18:34.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:34 smithi148 ceph-mon[31198]: pgmap v122: 164 pgs: 164 active+clean; 582 KiB data, 256 MiB used, 715 GiB / 715 GiB avail 2024-02-13T14:18:34.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:34 smithi148 ceph-mon[31198]: osdmap e114: 8 total, 8 up, 8 in 2024-02-13T14:18:34.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:34 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:18:34.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:34 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:18:34.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:34 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:18:35.369 INFO:journalctl@ceph.iscsi.iscsi.a.smithi148.stdout:Feb 13 14:18:34 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-iscsi-iscsi-a[59644]: debug there is no tcmu-runner data available 2024-02-13T14:18:36.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:35 smithi087 ceph-mon[27372]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:18:36.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:35 smithi087 ceph-mon[27372]: osdmap e115: 8 total, 8 up, 8 in 2024-02-13T14:18:36.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:35 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/3466896695' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:18:36.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:35 smithi087 ceph-mon[39120]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:18:36.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:35 smithi087 ceph-mon[39120]: osdmap e115: 8 total, 8 up, 8 in 2024-02-13T14:18:36.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:35 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/3466896695' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:18:36.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:35 smithi148 ceph-mon[31198]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:18:36.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:35 smithi148 ceph-mon[31198]: osdmap e115: 8 total, 8 up, 8 in 2024-02-13T14:18:36.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:35 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/3466896695' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:18:36.936 INFO:tasks.workunit.client.0.smithi087.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_writesame PASSED [ 27%] 2024-02-13T14:18:37.118 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:36 smithi148 ceph-mon[31198]: pgmap v125: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 256 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:18:37.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:36 smithi148 ceph-mon[31198]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T14:18:37.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:36 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/3466896695' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:18:37.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:36 smithi148 ceph-mon[31198]: osdmap e116: 8 total, 8 up, 8 in 2024-02-13T14:18:37.222 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:36 smithi087 ceph-mon[27372]: pgmap v125: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 256 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:18:37.222 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:36 smithi087 ceph-mon[27372]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T14:18:37.222 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:36 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/3466896695' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:18:37.222 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:36 smithi087 ceph-mon[27372]: osdmap e116: 8 total, 8 up, 8 in 2024-02-13T14:18:37.222 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:36 smithi087 ceph-mon[39120]: pgmap v125: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 256 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:18:37.222 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:36 smithi087 ceph-mon[39120]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T14:18:37.222 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:36 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/3466896695' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:18:37.222 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:36 smithi087 ceph-mon[39120]: osdmap e116: 8 total, 8 up, 8 in 2024-02-13T14:18:37.491 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:18:37 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: ::ffff:172.21.15.148 - - [13/Feb/2024:14:18:37] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-13T14:18:38.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:37 smithi087 ceph-mon[27372]: osdmap e117: 8 total, 8 up, 8 in 2024-02-13T14:18:38.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:37 smithi087 ceph-mon[39120]: osdmap e117: 8 total, 8 up, 8 in 2024-02-13T14:18:38.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:37 smithi148 ceph-mon[31198]: osdmap e117: 8 total, 8 up, 8 in 2024-02-13T14:18:39.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:38 smithi087 ceph-mon[27372]: pgmap v128: 164 pgs: 164 active+clean; 582 KiB data, 256 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:18:39.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:38 smithi087 ceph-mon[27372]: osdmap e118: 8 total, 8 up, 8 in 2024-02-13T14:18:39.243 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:38 smithi087 ceph-mon[39120]: pgmap v128: 164 pgs: 164 active+clean; 582 KiB data, 256 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:18:39.243 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:38 smithi087 ceph-mon[39120]: osdmap e118: 8 total, 8 up, 8 in 2024-02-13T14:18:39.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:38 smithi148 ceph-mon[31198]: pgmap v128: 164 pgs: 164 active+clean; 582 KiB data, 256 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:18:39.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:38 smithi148 ceph-mon[31198]: osdmap e118: 8 total, 8 up, 8 in 2024-02-13T14:18:40.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:39 smithi087 ceph-mon[27372]: osdmap e119: 8 total, 8 up, 8 in 2024-02-13T14:18:40.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:39 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-13T14:18:40.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:39 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/1703460019' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:18:40.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:39 smithi087 ceph-mon[39120]: osdmap e119: 8 total, 8 up, 8 in 2024-02-13T14:18:40.243 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:39 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-13T14:18:40.243 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:39 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/1703460019' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:18:40.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:39 smithi148 ceph-mon[31198]: osdmap e119: 8 total, 8 up, 8 in 2024-02-13T14:18:40.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:39 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-13T14:18:40.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:39 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/1703460019' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:18:40.973 INFO:tasks.workunit.client.0.smithi087.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_append PASSED [ 28%] 2024-02-13T14:18:41.241 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:40 smithi087 ceph-mon[27372]: pgmap v131: 196 pgs: 196 active+clean; 582 KiB data, 257 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-13T14:18:41.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:40 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/1703460019' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:18:41.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:40 smithi087 ceph-mon[27372]: osdmap e120: 8 total, 8 up, 8 in 2024-02-13T14:18:41.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:40 smithi087 ceph-mon[39120]: pgmap v131: 196 pgs: 196 active+clean; 582 KiB data, 257 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-13T14:18:41.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:40 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/1703460019' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:18:41.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:40 smithi087 ceph-mon[39120]: osdmap e120: 8 total, 8 up, 8 in 2024-02-13T14:18:41.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:40 smithi148 ceph-mon[31198]: pgmap v131: 196 pgs: 196 active+clean; 582 KiB data, 257 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-13T14:18:41.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:40 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/1703460019' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:18:41.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:40 smithi148 ceph-mon[31198]: osdmap e120: 8 total, 8 up, 8 in 2024-02-13T14:18:42.241 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:41 smithi087 ceph-mon[27372]: osdmap e121: 8 total, 8 up, 8 in 2024-02-13T14:18:42.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:41 smithi087 ceph-mon[39120]: osdmap e121: 8 total, 8 up, 8 in 2024-02-13T14:18:42.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:41 smithi148 ceph-mon[31198]: osdmap e121: 8 total, 8 up, 8 in 2024-02-13T14:18:43.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:42 smithi087 ceph-mon[27372]: pgmap v134: 164 pgs: 164 active+clean; 582 KiB data, 257 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:18:43.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:42 smithi087 ceph-mon[27372]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T14:18:43.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:42 smithi087 ceph-mon[27372]: osdmap e122: 8 total, 8 up, 8 in 2024-02-13T14:18:43.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:42 smithi087 ceph-mon[39120]: pgmap v134: 164 pgs: 164 active+clean; 582 KiB data, 257 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:18:43.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:42 smithi087 ceph-mon[39120]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T14:18:43.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:42 smithi087 ceph-mon[39120]: osdmap e122: 8 total, 8 up, 8 in 2024-02-13T14:18:43.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:42 smithi148 ceph-mon[31198]: pgmap v134: 164 pgs: 164 active+clean; 582 KiB data, 257 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:18:43.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:42 smithi148 ceph-mon[31198]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T14:18:43.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:42 smithi148 ceph-mon[31198]: osdmap e122: 8 total, 8 up, 8 in 2024-02-13T14:18:44.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:43 smithi148 ceph-mon[31198]: osdmap e123: 8 total, 8 up, 8 in 2024-02-13T14:18:44.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:43 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/898261341' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:18:44.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:43 smithi087 ceph-mon[27372]: osdmap e123: 8 total, 8 up, 8 in 2024-02-13T14:18:44.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:43 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/898261341' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:18:44.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:44 smithi087 ceph-mon[39120]: osdmap e123: 8 total, 8 up, 8 in 2024-02-13T14:18:44.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:44 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/898261341' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:18:45.009 INFO:tasks.workunit.client.0.smithi087.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_write_zeros PASSED [ 30%] 2024-02-13T14:18:45.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:45 smithi148 ceph-mon[31198]: pgmap v137: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 257 MiB used, 715 GiB / 715 GiB avail 2024-02-13T14:18:45.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:45 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/898261341' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:18:45.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:45 smithi148 ceph-mon[31198]: osdmap e124: 8 total, 8 up, 8 in 2024-02-13T14:18:45.369 INFO:journalctl@ceph.iscsi.iscsi.a.smithi148.stdout:Feb 13 14:18:44 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-iscsi-iscsi-a[59644]: debug there is no tcmu-runner data available 2024-02-13T14:18:45.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:45 smithi087 ceph-mon[39120]: pgmap v137: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 257 MiB used, 715 GiB / 715 GiB avail 2024-02-13T14:18:45.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:45 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/898261341' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:18:45.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:45 smithi087 ceph-mon[39120]: osdmap e124: 8 total, 8 up, 8 in 2024-02-13T14:18:45.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:45 smithi087 ceph-mon[27372]: pgmap v137: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 257 MiB used, 715 GiB / 715 GiB avail 2024-02-13T14:18:45.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:45 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/898261341' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:18:45.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:45 smithi087 ceph-mon[27372]: osdmap e124: 8 total, 8 up, 8 in 2024-02-13T14:18:46.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:46 smithi148 ceph-mon[31198]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:18:46.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:46 smithi148 ceph-mon[31198]: osdmap e125: 8 total, 8 up, 8 in 2024-02-13T14:18:46.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:46 smithi087 ceph-mon[39120]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:18:46.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:46 smithi087 ceph-mon[39120]: osdmap e125: 8 total, 8 up, 8 in 2024-02-13T14:18:46.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:46 smithi087 ceph-mon[27372]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:18:46.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:46 smithi087 ceph-mon[27372]: osdmap e125: 8 total, 8 up, 8 in 2024-02-13T14:18:47.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:47 smithi148 ceph-mon[31198]: pgmap v140: 164 pgs: 164 active+clean; 582 KiB data, 257 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:18:47.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:47 smithi148 ceph-mon[31198]: osdmap e126: 8 total, 8 up, 8 in 2024-02-13T14:18:47.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:47 smithi087 ceph-mon[27372]: pgmap v140: 164 pgs: 164 active+clean; 582 KiB data, 257 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:18:47.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:47 smithi087 ceph-mon[27372]: osdmap e126: 8 total, 8 up, 8 in 2024-02-13T14:18:47.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:47 smithi087 ceph-mon[39120]: pgmap v140: 164 pgs: 164 active+clean; 582 KiB data, 257 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:18:47.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:47 smithi087 ceph-mon[39120]: osdmap e126: 8 total, 8 up, 8 in 2024-02-13T14:18:47.492 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:18:47 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: ::ffff:172.21.15.148 - - [13/Feb/2024:14:18:47] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-13T14:18:48.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:48 smithi148 ceph-mon[31198]: osdmap e127: 8 total, 8 up, 8 in 2024-02-13T14:18:48.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:48 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/516562961' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:18:48.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:48 smithi148 ceph-mon[31198]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:18:48.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:48 smithi087 ceph-mon[27372]: osdmap e127: 8 total, 8 up, 8 in 2024-02-13T14:18:48.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:48 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/516562961' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:18:48.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:48 smithi087 ceph-mon[27372]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:18:48.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:48 smithi087 ceph-mon[39120]: osdmap e127: 8 total, 8 up, 8 in 2024-02-13T14:18:48.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:48 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/516562961' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:18:48.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:48 smithi087 ceph-mon[39120]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:18:49.042 INFO:tasks.workunit.client.0.smithi087.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_trunc PASSED [ 31%] 2024-02-13T14:18:49.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:49 smithi148 ceph-mon[31198]: pgmap v143: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 257 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:18:49.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:49 smithi148 ceph-mon[31198]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T14:18:49.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:49 smithi148 ceph-mon[31198]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:18:49.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:49 smithi148 ceph-mon[31198]: osdmap e128: 8 total, 8 up, 8 in 2024-02-13T14:18:49.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:49 smithi087 ceph-mon[27372]: pgmap v143: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 257 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:18:49.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:49 smithi087 ceph-mon[27372]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T14:18:49.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:49 smithi087 ceph-mon[27372]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:18:49.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:49 smithi087 ceph-mon[27372]: osdmap e128: 8 total, 8 up, 8 in 2024-02-13T14:18:49.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:49 smithi087 ceph-mon[39120]: pgmap v143: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 257 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:18:49.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:49 smithi087 ceph-mon[39120]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T14:18:49.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:49 smithi087 ceph-mon[39120]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:18:49.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:49 smithi087 ceph-mon[39120]: osdmap e128: 8 total, 8 up, 8 in 2024-02-13T14:18:50.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:50 smithi148 ceph-mon[31198]: osdmap e129: 8 total, 8 up, 8 in 2024-02-13T14:18:50.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:50 smithi087 ceph-mon[27372]: osdmap e129: 8 total, 8 up, 8 in 2024-02-13T14:18:50.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:50 smithi087 ceph-mon[39120]: osdmap e129: 8 total, 8 up, 8 in 2024-02-13T14:18:51.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:51 smithi148 ceph-mon[31198]: pgmap v146: 164 pgs: 164 active+clean; 582 KiB data, 258 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:18:51.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:51 smithi148 ceph-mon[31198]: osdmap e130: 8 total, 8 up, 8 in 2024-02-13T14:18:51.393 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:51 smithi087 ceph-mon[27372]: pgmap v146: 164 pgs: 164 active+clean; 582 KiB data, 258 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:18:51.393 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:51 smithi087 ceph-mon[27372]: osdmap e130: 8 total, 8 up, 8 in 2024-02-13T14:18:51.393 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:51 smithi087 ceph-mon[39120]: pgmap v146: 164 pgs: 164 active+clean; 582 KiB data, 258 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:18:51.393 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:51 smithi087 ceph-mon[39120]: osdmap e130: 8 total, 8 up, 8 in 2024-02-13T14:18:52.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:52 smithi148 ceph-mon[31198]: osdmap e131: 8 total, 8 up, 8 in 2024-02-13T14:18:52.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:52 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/674382774' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:18:52.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:52 smithi148 ceph-mon[31198]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:18:52.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:52 smithi087 ceph-mon[27372]: osdmap e131: 8 total, 8 up, 8 in 2024-02-13T14:18:52.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:52 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/674382774' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:18:52.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:52 smithi087 ceph-mon[27372]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:18:52.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:52 smithi087 ceph-mon[39120]: osdmap e131: 8 total, 8 up, 8 in 2024-02-13T14:18:52.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:52 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/674382774' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:18:52.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:52 smithi087 ceph-mon[39120]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:18:53.068 INFO:tasks.workunit.client.0.smithi087.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_cmpext PASSED [ 32%] 2024-02-13T14:18:53.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:53 smithi148 ceph-mon[31198]: pgmap v149: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 258 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:18:53.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:53 smithi148 ceph-mon[31198]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:18:53.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:53 smithi148 ceph-mon[31198]: osdmap e132: 8 total, 8 up, 8 in 2024-02-13T14:18:53.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:53 smithi087 ceph-mon[27372]: pgmap v149: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 258 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:18:53.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:53 smithi087 ceph-mon[27372]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:18:53.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:53 smithi087 ceph-mon[27372]: osdmap e132: 8 total, 8 up, 8 in 2024-02-13T14:18:53.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:53 smithi087 ceph-mon[39120]: pgmap v149: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 258 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:18:53.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:53 smithi087 ceph-mon[39120]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:18:53.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:53 smithi087 ceph-mon[39120]: osdmap e132: 8 total, 8 up, 8 in 2024-02-13T14:18:54.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:54 smithi148 ceph-mon[31198]: osdmap e133: 8 total, 8 up, 8 in 2024-02-13T14:18:54.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:54 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-13T14:18:54.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:54 smithi087 ceph-mon[27372]: osdmap e133: 8 total, 8 up, 8 in 2024-02-13T14:18:54.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:54 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-13T14:18:54.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:54 smithi087 ceph-mon[39120]: osdmap e133: 8 total, 8 up, 8 in 2024-02-13T14:18:54.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:54 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-13T14:18:55.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:55 smithi148 ceph-mon[31198]: pgmap v152: 164 pgs: 164 active+clean; 582 KiB data, 258 MiB used, 715 GiB / 715 GiB avail 2024-02-13T14:18:55.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:55 smithi148 ceph-mon[31198]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T14:18:55.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:55 smithi148 ceph-mon[31198]: osdmap e134: 8 total, 8 up, 8 in 2024-02-13T14:18:55.369 INFO:journalctl@ceph.iscsi.iscsi.a.smithi148.stdout:Feb 13 14:18:54 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-iscsi-iscsi-a[59644]: debug there is no tcmu-runner data available 2024-02-13T14:18:55.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:55 smithi087 ceph-mon[27372]: pgmap v152: 164 pgs: 164 active+clean; 582 KiB data, 258 MiB used, 715 GiB / 715 GiB avail 2024-02-13T14:18:55.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:55 smithi087 ceph-mon[27372]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T14:18:55.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:55 smithi087 ceph-mon[27372]: osdmap e134: 8 total, 8 up, 8 in 2024-02-13T14:18:55.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:55 smithi087 ceph-mon[39120]: pgmap v152: 164 pgs: 164 active+clean; 582 KiB data, 258 MiB used, 715 GiB / 715 GiB avail 2024-02-13T14:18:55.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:55 smithi087 ceph-mon[39120]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T14:18:55.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:55 smithi087 ceph-mon[39120]: osdmap e134: 8 total, 8 up, 8 in 2024-02-13T14:18:56.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:56 smithi148 ceph-mon[31198]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:18:56.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:56 smithi148 ceph-mon[31198]: osdmap e135: 8 total, 8 up, 8 in 2024-02-13T14:18:56.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:56 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/2213630605' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:18:56.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:56 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/2213630605' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:18:56.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:56 smithi148 ceph-mon[31198]: osdmap e136: 8 total, 8 up, 8 in 2024-02-13T14:18:56.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:56 smithi087 ceph-mon[27372]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:18:56.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:56 smithi087 ceph-mon[27372]: osdmap e135: 8 total, 8 up, 8 in 2024-02-13T14:18:56.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:56 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/2213630605' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:18:56.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:56 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/2213630605' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:18:56.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:56 smithi087 ceph-mon[27372]: osdmap e136: 8 total, 8 up, 8 in 2024-02-13T14:18:56.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:56 smithi087 ceph-mon[39120]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:18:56.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:56 smithi087 ceph-mon[39120]: osdmap e135: 8 total, 8 up, 8 in 2024-02-13T14:18:56.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:56 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/2213630605' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:18:56.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:56 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/2213630605' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:18:56.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:56 smithi087 ceph-mon[39120]: osdmap e136: 8 total, 8 up, 8 in 2024-02-13T14:18:57.087 INFO:tasks.workunit.client.0.smithi087.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_list_objects_empty PASSED [ 33%] 2024-02-13T14:18:57.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:57 smithi148 ceph-mon[31198]: pgmap v155: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 266 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:18:57.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:57 smithi148 ceph-mon[31198]: osdmap e137: 8 total, 8 up, 8 in 2024-02-13T14:18:57.492 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:18:57 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: ::ffff:172.21.15.148 - - [13/Feb/2024:14:18:57] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-13T14:18:57.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:57 smithi087 ceph-mon[27372]: pgmap v155: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 266 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:18:57.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:57 smithi087 ceph-mon[27372]: osdmap e137: 8 total, 8 up, 8 in 2024-02-13T14:18:57.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:57 smithi087 ceph-mon[39120]: pgmap v155: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 266 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:18:57.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:57 smithi087 ceph-mon[39120]: osdmap e137: 8 total, 8 up, 8 in 2024-02-13T14:18:59.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:59 smithi148 ceph-mon[31198]: pgmap v158: 164 pgs: 164 active+clean; 582 KiB data, 266 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:18:59.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:18:59 smithi148 ceph-mon[31198]: osdmap e138: 8 total, 8 up, 8 in 2024-02-13T14:18:59.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:59 smithi087 ceph-mon[27372]: pgmap v158: 164 pgs: 164 active+clean; 582 KiB data, 266 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:18:59.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:18:59 smithi087 ceph-mon[27372]: osdmap e138: 8 total, 8 up, 8 in 2024-02-13T14:18:59.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:59 smithi087 ceph-mon[39120]: pgmap v158: 164 pgs: 164 active+clean; 582 KiB data, 266 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:18:59.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:18:59 smithi087 ceph-mon[39120]: osdmap e138: 8 total, 8 up, 8 in 2024-02-13T14:19:00.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:00 smithi087 ceph-mon[27372]: osdmap e139: 8 total, 8 up, 8 in 2024-02-13T14:19:00.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:00 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/3902773598' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:19:00.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:00 smithi087 ceph-mon[27372]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:19:00.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:00 smithi087 ceph-mon[39120]: osdmap e139: 8 total, 8 up, 8 in 2024-02-13T14:19:00.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:00 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/3902773598' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:19:00.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:00 smithi087 ceph-mon[39120]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:19:00.618 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:00 smithi148 ceph-mon[31198]: osdmap e139: 8 total, 8 up, 8 in 2024-02-13T14:19:00.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:00 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/3902773598' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:19:00.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:00 smithi148 ceph-mon[31198]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:19:01.139 INFO:tasks.workunit.client.0.smithi087.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_list_objects PASSED [ 34%] 2024-02-13T14:19:01.491 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:01 smithi087 ceph-mon[27372]: pgmap v161: 196 pgs: 196 active+clean; 582 KiB data, 271 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-13T14:19:01.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:01 smithi087 ceph-mon[27372]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T14:19:01.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:01 smithi087 ceph-mon[27372]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:19:01.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:01 smithi087 ceph-mon[27372]: osdmap e140: 8 total, 8 up, 8 in 2024-02-13T14:19:01.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:01 smithi087 ceph-mon[39120]: pgmap v161: 196 pgs: 196 active+clean; 582 KiB data, 271 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-13T14:19:01.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:01 smithi087 ceph-mon[39120]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T14:19:01.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:01 smithi087 ceph-mon[39120]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:19:01.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:01 smithi087 ceph-mon[39120]: osdmap e140: 8 total, 8 up, 8 in 2024-02-13T14:19:01.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:01 smithi148 ceph-mon[31198]: pgmap v161: 196 pgs: 196 active+clean; 582 KiB data, 271 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-13T14:19:01.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:01 smithi148 ceph-mon[31198]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T14:19:01.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:01 smithi148 ceph-mon[31198]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:19:01.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:01 smithi148 ceph-mon[31198]: osdmap e140: 8 total, 8 up, 8 in 2024-02-13T14:19:02.491 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:02 smithi087 ceph-mon[27372]: osdmap e141: 8 total, 8 up, 8 in 2024-02-13T14:19:02.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:02 smithi087 ceph-mon[39120]: osdmap e141: 8 total, 8 up, 8 in 2024-02-13T14:19:02.618 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:02 smithi148 ceph-mon[31198]: osdmap e141: 8 total, 8 up, 8 in 2024-02-13T14:19:03.491 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:03 smithi087 ceph-mon[27372]: pgmap v164: 164 pgs: 164 active+clean; 582 KiB data, 271 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:19:03.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:03 smithi087 ceph-mon[27372]: osdmap e142: 8 total, 8 up, 8 in 2024-02-13T14:19:03.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:03 smithi087 ceph-mon[39120]: pgmap v164: 164 pgs: 164 active+clean; 582 KiB data, 271 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:19:03.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:03 smithi087 ceph-mon[39120]: osdmap e142: 8 total, 8 up, 8 in 2024-02-13T14:19:03.618 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:03 smithi148 ceph-mon[31198]: pgmap v164: 164 pgs: 164 active+clean; 582 KiB data, 271 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:19:03.618 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:03 smithi148 ceph-mon[31198]: osdmap e142: 8 total, 8 up, 8 in 2024-02-13T14:19:04.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:04 smithi087 ceph-mon[27372]: osdmap e143: 8 total, 8 up, 8 in 2024-02-13T14:19:04.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:04 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/335964769' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:19:04.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:04 smithi087 ceph-mon[27372]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:19:04.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:04 smithi087 ceph-mon[39120]: osdmap e143: 8 total, 8 up, 8 in 2024-02-13T14:19:04.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:04 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/335964769' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:19:04.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:04 smithi087 ceph-mon[39120]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:19:04.618 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:04 smithi148 ceph-mon[31198]: osdmap e143: 8 total, 8 up, 8 in 2024-02-13T14:19:04.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:04 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/335964769' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:19:04.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:04 smithi148 ceph-mon[31198]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:19:05.156 INFO:journalctl@ceph.iscsi.iscsi.a.smithi148.stdout:Feb 13 14:19:04 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-iscsi-iscsi-a[59644]: debug there is no tcmu-runner data available 2024-02-13T14:19:05.158 INFO:tasks.workunit.client.0.smithi087.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_list_ns_objects PASSED [ 35%] 2024-02-13T14:19:05.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:05 smithi087 ceph-mon[27372]: pgmap v167: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 271 MiB used, 715 GiB / 715 GiB avail 2024-02-13T14:19:05.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:05 smithi087 ceph-mon[27372]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:19:05.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:05 smithi087 ceph-mon[27372]: osdmap e144: 8 total, 8 up, 8 in 2024-02-13T14:19:05.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:05 smithi087 ceph-mon[39120]: pgmap v167: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 271 MiB used, 715 GiB / 715 GiB avail 2024-02-13T14:19:05.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:05 smithi087 ceph-mon[39120]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:19:05.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:05 smithi087 ceph-mon[39120]: osdmap e144: 8 total, 8 up, 8 in 2024-02-13T14:19:05.618 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:05 smithi148 ceph-mon[31198]: pgmap v167: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 271 MiB used, 715 GiB / 715 GiB avail 2024-02-13T14:19:05.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:05 smithi148 ceph-mon[31198]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:19:05.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:05 smithi148 ceph-mon[31198]: osdmap e144: 8 total, 8 up, 8 in 2024-02-13T14:19:06.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:06 smithi087 ceph-mon[39120]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:19:06.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:06 smithi087 ceph-mon[39120]: osdmap e145: 8 total, 8 up, 8 in 2024-02-13T14:19:06.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:06 smithi087 ceph-mon[27372]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:19:06.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:06 smithi087 ceph-mon[27372]: osdmap e145: 8 total, 8 up, 8 in 2024-02-13T14:19:06.618 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:06 smithi148 ceph-mon[31198]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:19:06.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:06 smithi148 ceph-mon[31198]: osdmap e145: 8 total, 8 up, 8 in 2024-02-13T14:19:07.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:07 smithi087 ceph-mon[27372]: pgmap v170: 164 pgs: 164 active+clean; 582 KiB data, 275 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:19:07.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:07 smithi087 ceph-mon[27372]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T14:19:07.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:07 smithi087 ceph-mon[27372]: osdmap e146: 8 total, 8 up, 8 in 2024-02-13T14:19:07.493 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:19:07 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: ::ffff:172.21.15.148 - - [13/Feb/2024:14:19:07] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-13T14:19:07.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:07 smithi087 ceph-mon[39120]: pgmap v170: 164 pgs: 164 active+clean; 582 KiB data, 275 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:19:07.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:07 smithi087 ceph-mon[39120]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T14:19:07.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:07 smithi087 ceph-mon[39120]: osdmap e146: 8 total, 8 up, 8 in 2024-02-13T14:19:07.618 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:07 smithi148 ceph-mon[31198]: pgmap v170: 164 pgs: 164 active+clean; 582 KiB data, 275 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:19:07.618 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:07 smithi148 ceph-mon[31198]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T14:19:07.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:07 smithi148 ceph-mon[31198]: osdmap e146: 8 total, 8 up, 8 in 2024-02-13T14:19:08.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:08 smithi087 ceph-mon[27372]: osdmap e147: 8 total, 8 up, 8 in 2024-02-13T14:19:08.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:08 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/3187965587' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:19:08.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:08 smithi087 ceph-mon[27372]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:19:08.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:08 smithi087 ceph-mon[39120]: osdmap e147: 8 total, 8 up, 8 in 2024-02-13T14:19:08.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:08 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/3187965587' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:19:08.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:08 smithi087 ceph-mon[39120]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:19:08.618 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:08 smithi148 ceph-mon[31198]: osdmap e147: 8 total, 8 up, 8 in 2024-02-13T14:19:08.618 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:08 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/3187965587' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:19:08.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:08 smithi148 ceph-mon[31198]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:19:09.184 INFO:tasks.workunit.client.0.smithi087.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_xattrs PASSED [ 36%] 2024-02-13T14:19:09.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:09 smithi087 ceph-mon[27372]: pgmap v173: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 275 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:19:09.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:09 smithi087 ceph-mon[27372]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:19:09.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:09 smithi087 ceph-mon[27372]: osdmap e148: 8 total, 8 up, 8 in 2024-02-13T14:19:09.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:09 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-13T14:19:09.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:09 smithi087 ceph-mon[39120]: pgmap v173: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 275 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:19:09.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:09 smithi087 ceph-mon[39120]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:19:09.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:09 smithi087 ceph-mon[39120]: osdmap e148: 8 total, 8 up, 8 in 2024-02-13T14:19:09.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:09 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-13T14:19:09.618 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:09 smithi148 ceph-mon[31198]: pgmap v173: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 275 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:19:09.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:09 smithi148 ceph-mon[31198]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:19:09.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:09 smithi148 ceph-mon[31198]: osdmap e148: 8 total, 8 up, 8 in 2024-02-13T14:19:09.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:09 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-13T14:19:10.491 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:10 smithi087 ceph-mon[27372]: osdmap e149: 8 total, 8 up, 8 in 2024-02-13T14:19:10.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:10 smithi087 ceph-mon[39120]: osdmap e149: 8 total, 8 up, 8 in 2024-02-13T14:19:10.618 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:10 smithi148 ceph-mon[31198]: osdmap e149: 8 total, 8 up, 8 in 2024-02-13T14:19:11.491 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:11 smithi087 ceph-mon[27372]: pgmap v176: 164 pgs: 164 active+clean; 582 KiB data, 276 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:19:11.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:11 smithi087 ceph-mon[27372]: osdmap e150: 8 total, 8 up, 8 in 2024-02-13T14:19:11.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:11 smithi087 ceph-mon[39120]: pgmap v176: 164 pgs: 164 active+clean; 582 KiB data, 276 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:19:11.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:11 smithi087 ceph-mon[39120]: osdmap e150: 8 total, 8 up, 8 in 2024-02-13T14:19:11.618 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:11 smithi148 ceph-mon[31198]: pgmap v176: 164 pgs: 164 active+clean; 582 KiB data, 276 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:19:11.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:11 smithi148 ceph-mon[31198]: osdmap e150: 8 total, 8 up, 8 in 2024-02-13T14:19:12.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:12 smithi087 ceph-mon[27372]: osdmap e151: 8 total, 8 up, 8 in 2024-02-13T14:19:12.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:12 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/1752458457' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:19:12.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:12 smithi087 ceph-mon[27372]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:19:12.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:12 smithi087 ceph-mon[39120]: osdmap e151: 8 total, 8 up, 8 in 2024-02-13T14:19:12.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:12 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/1752458457' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:19:12.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:12 smithi087 ceph-mon[39120]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:19:12.618 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:12 smithi148 ceph-mon[31198]: osdmap e151: 8 total, 8 up, 8 in 2024-02-13T14:19:12.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:12 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/1752458457' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:19:12.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:12 smithi148 ceph-mon[31198]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:19:13.225 INFO:tasks.workunit.client.0.smithi087.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_obj_xattrs PASSED [ 37%] 2024-02-13T14:19:13.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:13 smithi087 ceph-mon[27372]: pgmap v179: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 276 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:19:13.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:13 smithi087 ceph-mon[27372]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T14:19:13.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:13 smithi087 ceph-mon[27372]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:19:13.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:13 smithi087 ceph-mon[27372]: osdmap e152: 8 total, 8 up, 8 in 2024-02-13T14:19:13.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:13 smithi087 ceph-mon[39120]: pgmap v179: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 276 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:19:13.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:13 smithi087 ceph-mon[39120]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T14:19:13.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:13 smithi087 ceph-mon[39120]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:19:13.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:13 smithi087 ceph-mon[39120]: osdmap e152: 8 total, 8 up, 8 in 2024-02-13T14:19:13.618 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:13 smithi148 ceph-mon[31198]: pgmap v179: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 276 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:19:13.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:13 smithi148 ceph-mon[31198]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T14:19:13.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:13 smithi148 ceph-mon[31198]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:19:13.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:13 smithi148 ceph-mon[31198]: osdmap e152: 8 total, 8 up, 8 in 2024-02-13T14:19:14.491 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:14 smithi087 ceph-mon[27372]: osdmap e153: 8 total, 8 up, 8 in 2024-02-13T14:19:14.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:14 smithi087 ceph-mon[39120]: osdmap e153: 8 total, 8 up, 8 in 2024-02-13T14:19:14.618 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:14 smithi148 ceph-mon[31198]: osdmap e153: 8 total, 8 up, 8 in 2024-02-13T14:19:15.235 INFO:journalctl@ceph.iscsi.iscsi.a.smithi148.stdout:Feb 13 14:19:14 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-iscsi-iscsi-a[59644]: debug there is no tcmu-runner data available 2024-02-13T14:19:15.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:15 smithi087 ceph-mon[27372]: pgmap v182: 164 pgs: 164 active+clean; 582 KiB data, 276 MiB used, 715 GiB / 715 GiB avail 2024-02-13T14:19:15.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:15 smithi087 ceph-mon[27372]: osdmap e154: 8 total, 8 up, 8 in 2024-02-13T14:19:15.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:15 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/613721642' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:19:15.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:15 smithi087 ceph-mon[39120]: pgmap v182: 164 pgs: 164 active+clean; 582 KiB data, 276 MiB used, 715 GiB / 715 GiB avail 2024-02-13T14:19:15.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:15 smithi087 ceph-mon[39120]: osdmap e154: 8 total, 8 up, 8 in 2024-02-13T14:19:15.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:15 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/613721642' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:19:15.618 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:15 smithi148 ceph-mon[31198]: pgmap v182: 164 pgs: 164 active+clean; 582 KiB data, 276 MiB used, 715 GiB / 715 GiB avail 2024-02-13T14:19:15.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:15 smithi148 ceph-mon[31198]: osdmap e154: 8 total, 8 up, 8 in 2024-02-13T14:19:15.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:15 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/613721642' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:19:16.232 INFO:tasks.workunit.client.0.smithi087.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_get_pool_id PASSED [ 38%] 2024-02-13T14:19:16.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:16 smithi087 ceph-mon[27372]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:19:16.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:16 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/613721642' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:19:16.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:16 smithi087 ceph-mon[27372]: osdmap e155: 8 total, 8 up, 8 in 2024-02-13T14:19:16.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:16 smithi087 ceph-mon[27372]: osdmap e156: 8 total, 8 up, 8 in 2024-02-13T14:19:16.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:16 smithi087 ceph-mon[39120]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:19:16.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:16 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/613721642' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:19:16.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:16 smithi087 ceph-mon[39120]: osdmap e155: 8 total, 8 up, 8 in 2024-02-13T14:19:16.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:16 smithi087 ceph-mon[39120]: osdmap e156: 8 total, 8 up, 8 in 2024-02-13T14:19:16.620 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:16 smithi148 ceph-mon[31198]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:19:16.620 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:16 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/613721642' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:19:16.620 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:16 smithi148 ceph-mon[31198]: osdmap e155: 8 total, 8 up, 8 in 2024-02-13T14:19:16.620 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:16 smithi148 ceph-mon[31198]: osdmap e156: 8 total, 8 up, 8 in 2024-02-13T14:19:17.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:17 smithi087 ceph-mon[27372]: pgmap v185: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 280 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:19:17.492 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:19:17 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: ::ffff:172.21.15.148 - - [13/Feb/2024:14:19:17] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-13T14:19:17.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:17 smithi087 ceph-mon[39120]: pgmap v185: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 280 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:19:17.618 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:17 smithi148 ceph-mon[31198]: pgmap v185: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 280 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:19:18.618 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:18 smithi148 ceph-mon[31198]: osdmap e157: 8 total, 8 up, 8 in 2024-02-13T14:19:18.618 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:18 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/2637504573' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:19:18.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:18 smithi087 ceph-mon[27372]: osdmap e157: 8 total, 8 up, 8 in 2024-02-13T14:19:18.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:18 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/2637504573' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:19:18.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:18 smithi087 ceph-mon[39120]: osdmap e157: 8 total, 8 up, 8 in 2024-02-13T14:19:18.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:18 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/2637504573' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:19:19.266 INFO:tasks.workunit.client.0.smithi087.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_get_pool_name PASSED [ 40%] 2024-02-13T14:19:19.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:19 smithi148 ceph-mon[31198]: pgmap v188: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 280 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:19:19.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:19 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/2637504573' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:19:19.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:19 smithi148 ceph-mon[31198]: osdmap e158: 8 total, 8 up, 8 in 2024-02-13T14:19:19.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:19 smithi087 ceph-mon[27372]: pgmap v188: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 280 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:19:19.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:19 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/2637504573' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:19:19.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:19 smithi087 ceph-mon[27372]: osdmap e158: 8 total, 8 up, 8 in 2024-02-13T14:19:19.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:19 smithi087 ceph-mon[39120]: pgmap v188: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 280 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:19:19.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:19 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/2637504573' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:19:19.743 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:19 smithi087 ceph-mon[39120]: osdmap e158: 8 total, 8 up, 8 in 2024-02-13T14:19:20.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:20 smithi148 ceph-mon[31198]: osdmap e159: 8 total, 8 up, 8 in 2024-02-13T14:19:20.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:20 smithi087 ceph-mon[27372]: osdmap e159: 8 total, 8 up, 8 in 2024-02-13T14:19:20.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:20 smithi087 ceph-mon[39120]: osdmap e159: 8 total, 8 up, 8 in 2024-02-13T14:19:21.618 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:21 smithi148 ceph-mon[31198]: pgmap v191: 164 pgs: 164 active+clean; 582 KiB data, 285 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:19:21.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:21 smithi148 ceph-mon[31198]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T14:19:21.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:21 smithi148 ceph-mon[31198]: osdmap e160: 8 total, 8 up, 8 in 2024-02-13T14:19:21.643 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:21 smithi087 ceph-mon[27372]: pgmap v191: 164 pgs: 164 active+clean; 582 KiB data, 285 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:19:21.643 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:21 smithi087 ceph-mon[27372]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T14:19:21.643 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:21 smithi087 ceph-mon[27372]: osdmap e160: 8 total, 8 up, 8 in 2024-02-13T14:19:21.644 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:21 smithi087 ceph-mon[39120]: pgmap v191: 164 pgs: 164 active+clean; 582 KiB data, 285 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:19:21.644 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:21 smithi087 ceph-mon[39120]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T14:19:21.644 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:21 smithi087 ceph-mon[39120]: osdmap e160: 8 total, 8 up, 8 in 2024-02-13T14:19:22.618 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:22 smithi148 ceph-mon[31198]: osdmap e161: 8 total, 8 up, 8 in 2024-02-13T14:19:22.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:22 smithi087 ceph-mon[39120]: osdmap e161: 8 total, 8 up, 8 in 2024-02-13T14:19:22.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:22 smithi087 ceph-mon[27372]: osdmap e161: 8 total, 8 up, 8 in 2024-02-13T14:19:23.618 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:23 smithi148 ceph-mon[31198]: pgmap v194: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 285 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:19:23.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:23 smithi148 ceph-mon[31198]: osdmap e162: 8 total, 8 up, 8 in 2024-02-13T14:19:23.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:23 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/4201142829' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:19:23.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:23 smithi148 ceph-mon[31198]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:19:23.741 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:23 smithi087 ceph-mon[27372]: pgmap v194: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 285 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:19:23.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:23 smithi087 ceph-mon[27372]: osdmap e162: 8 total, 8 up, 8 in 2024-02-13T14:19:23.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:23 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/4201142829' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:19:23.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:23 smithi087 ceph-mon[27372]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:19:23.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:23 smithi087 ceph-mon[39120]: pgmap v194: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 285 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:19:23.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:23 smithi087 ceph-mon[39120]: osdmap e162: 8 total, 8 up, 8 in 2024-02-13T14:19:23.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:23 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/4201142829' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:19:23.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:23 smithi087 ceph-mon[39120]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:19:24.315 INFO:tasks.workunit.client.0.smithi087.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_create_snap PASSED [ 41%] 2024-02-13T14:19:24.618 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:24 smithi148 ceph-mon[31198]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:19:24.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:24 smithi148 ceph-mon[31198]: osdmap e163: 8 total, 8 up, 8 in 2024-02-13T14:19:24.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:24 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-13T14:19:24.741 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:24 smithi087 ceph-mon[27372]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:19:24.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:24 smithi087 ceph-mon[27372]: osdmap e163: 8 total, 8 up, 8 in 2024-02-13T14:19:24.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:24 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-13T14:19:24.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:24 smithi087 ceph-mon[39120]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:19:24.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:24 smithi087 ceph-mon[39120]: osdmap e163: 8 total, 8 up, 8 in 2024-02-13T14:19:24.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:24 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-13T14:19:25.311 INFO:journalctl@ceph.iscsi.iscsi.a.smithi148.stdout:Feb 13 14:19:24 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-iscsi-iscsi-a[59644]: debug there is no tcmu-runner data available 2024-02-13T14:19:25.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:25 smithi148 ceph-mon[31198]: pgmap v197: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 285 MiB used, 715 GiB / 715 GiB avail 2024-02-13T14:19:25.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:25 smithi148 ceph-mon[31198]: osdmap e164: 8 total, 8 up, 8 in 2024-02-13T14:19:25.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:25 smithi087 ceph-mon[27372]: pgmap v197: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 285 MiB used, 715 GiB / 715 GiB avail 2024-02-13T14:19:25.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:25 smithi087 ceph-mon[27372]: osdmap e164: 8 total, 8 up, 8 in 2024-02-13T14:19:25.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:25 smithi087 ceph-mon[39120]: pgmap v197: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 285 MiB used, 715 GiB / 715 GiB avail 2024-02-13T14:19:25.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:25 smithi087 ceph-mon[39120]: osdmap e164: 8 total, 8 up, 8 in 2024-02-13T14:19:26.618 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:26 smithi148 ceph-mon[31198]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:19:26.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:26 smithi148 ceph-mon[31198]: osdmap e165: 8 total, 8 up, 8 in 2024-02-13T14:19:26.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:26 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/3927863667' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:19:26.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:26 smithi148 ceph-mon[31198]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:19:26.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:26 smithi148 ceph-mon[31198]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T14:19:26.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:26 smithi087 ceph-mon[27372]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:19:26.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:26 smithi087 ceph-mon[27372]: osdmap e165: 8 total, 8 up, 8 in 2024-02-13T14:19:26.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:26 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/3927863667' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:19:26.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:26 smithi087 ceph-mon[27372]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:19:26.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:26 smithi087 ceph-mon[27372]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T14:19:26.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:26 smithi087 ceph-mon[39120]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:19:26.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:26 smithi087 ceph-mon[39120]: osdmap e165: 8 total, 8 up, 8 in 2024-02-13T14:19:26.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:26 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/3927863667' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:19:26.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:26 smithi087 ceph-mon[39120]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:19:26.743 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:26 smithi087 ceph-mon[39120]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T14:19:27.336 INFO:tasks.workunit.client.0.smithi087.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_list_snaps_empty PASSED [ 42%] 2024-02-13T14:19:27.491 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:19:27 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: ::ffff:172.21.15.148 - - [13/Feb/2024:14:19:27] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-13T14:19:27.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:27 smithi087 ceph-mon[27372]: pgmap v200: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 286 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:19:27.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:27 smithi087 ceph-mon[27372]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:19:27.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:27 smithi087 ceph-mon[27372]: osdmap e166: 8 total, 8 up, 8 in 2024-02-13T14:19:27.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:27 smithi087 ceph-mon[39120]: pgmap v200: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 286 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:19:27.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:27 smithi087 ceph-mon[39120]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:19:27.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:27 smithi087 ceph-mon[39120]: osdmap e166: 8 total, 8 up, 8 in 2024-02-13T14:19:27.618 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:27 smithi148 ceph-mon[31198]: pgmap v200: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 286 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:19:27.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:27 smithi148 ceph-mon[31198]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:19:27.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:27 smithi148 ceph-mon[31198]: osdmap e166: 8 total, 8 up, 8 in 2024-02-13T14:19:28.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:28 smithi148 ceph-mon[31198]: osdmap e167: 8 total, 8 up, 8 in 2024-02-13T14:19:28.741 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:28 smithi087 ceph-mon[27372]: osdmap e167: 8 total, 8 up, 8 in 2024-02-13T14:19:28.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:28 smithi087 ceph-mon[39120]: osdmap e167: 8 total, 8 up, 8 in 2024-02-13T14:19:29.618 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:29 smithi148 ceph-mon[31198]: pgmap v203: 164 pgs: 164 active+clean; 582 KiB data, 286 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:19:29.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:29 smithi148 ceph-mon[31198]: osdmap e168: 8 total, 8 up, 8 in 2024-02-13T14:19:29.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:29 smithi087 ceph-mon[27372]: pgmap v203: 164 pgs: 164 active+clean; 582 KiB data, 286 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:19:29.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:29 smithi087 ceph-mon[27372]: osdmap e168: 8 total, 8 up, 8 in 2024-02-13T14:19:29.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:29 smithi087 ceph-mon[39120]: pgmap v203: 164 pgs: 164 active+clean; 582 KiB data, 286 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:19:29.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:29 smithi087 ceph-mon[39120]: osdmap e168: 8 total, 8 up, 8 in 2024-02-13T14:19:30.618 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:30 smithi148 ceph-mon[31198]: osdmap e169: 8 total, 8 up, 8 in 2024-02-13T14:19:30.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:30 smithi148 ceph-mon[31198]: osdmap e170: 8 total, 8 up, 8 in 2024-02-13T14:19:30.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:30 smithi087 ceph-mon[27372]: osdmap e169: 8 total, 8 up, 8 in 2024-02-13T14:19:30.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:30 smithi087 ceph-mon[27372]: osdmap e170: 8 total, 8 up, 8 in 2024-02-13T14:19:30.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:30 smithi087 ceph-mon[39120]: osdmap e169: 8 total, 8 up, 8 in 2024-02-13T14:19:30.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:30 smithi087 ceph-mon[39120]: osdmap e170: 8 total, 8 up, 8 in 2024-02-13T14:19:31.618 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:31 smithi148 ceph-mon[31198]: pgmap v206: 196 pgs: 196 active+clean; 582 KiB data, 286 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:19:31.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:31 smithi148 ceph-mon[31198]: osdmap e171: 8 total, 8 up, 8 in 2024-02-13T14:19:31.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:31 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/2620432788' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:19:31.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:31 smithi148 ceph-mon[31198]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:19:31.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:31 smithi087 ceph-mon[27372]: pgmap v206: 196 pgs: 196 active+clean; 582 KiB data, 286 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:19:31.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:31 smithi087 ceph-mon[27372]: osdmap e171: 8 total, 8 up, 8 in 2024-02-13T14:19:31.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:31 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/2620432788' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:19:31.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:31 smithi087 ceph-mon[27372]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:19:31.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:31 smithi087 ceph-mon[39120]: pgmap v206: 196 pgs: 196 active+clean; 582 KiB data, 286 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:19:31.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:31 smithi087 ceph-mon[39120]: osdmap e171: 8 total, 8 up, 8 in 2024-02-13T14:19:31.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:31 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/2620432788' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:19:31.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:31 smithi087 ceph-mon[39120]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:19:33.364 INFO:tasks.workunit.client.0.smithi087.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_list_snaps PASSED [ 43%] 2024-02-13T14:19:33.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:33 smithi148 ceph-mon[31198]: pgmap v209: 196 pgs: 196 active+clean; 582 KiB data, 286 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:19:33.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:33 smithi148 ceph-mon[31198]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:19:33.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:33 smithi148 ceph-mon[31198]: osdmap e172: 8 total, 8 up, 8 in 2024-02-13T14:19:33.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:33 smithi087 ceph-mon[27372]: pgmap v209: 196 pgs: 196 active+clean; 582 KiB data, 286 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:19:33.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:33 smithi087 ceph-mon[27372]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:19:33.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:33 smithi087 ceph-mon[27372]: osdmap e172: 8 total, 8 up, 8 in 2024-02-13T14:19:33.743 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:33 smithi087 ceph-mon[39120]: pgmap v209: 196 pgs: 196 active+clean; 582 KiB data, 286 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:19:33.743 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:33 smithi087 ceph-mon[39120]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:19:33.743 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:33 smithi087 ceph-mon[39120]: osdmap e172: 8 total, 8 up, 8 in 2024-02-13T14:19:34.741 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:34 smithi087 ceph-mon[27372]: osdmap e173: 8 total, 8 up, 8 in 2024-02-13T14:19:34.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:34 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:19:34.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:34 smithi087 ceph-mon[39120]: osdmap e173: 8 total, 8 up, 8 in 2024-02-13T14:19:34.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:34 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:19:34.868 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:34 smithi148 ceph-mon[31198]: osdmap e173: 8 total, 8 up, 8 in 2024-02-13T14:19:34.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:34 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:19:35.369 INFO:journalctl@ceph.iscsi.iscsi.a.smithi148.stdout:Feb 13 14:19:34 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-iscsi-iscsi-a[59644]: debug there is no tcmu-runner data available 2024-02-13T14:19:35.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:35 smithi087 ceph-mon[27372]: pgmap v212: 164 pgs: 164 active+clean; 582 KiB data, 286 MiB used, 715 GiB / 715 GiB avail 2024-02-13T14:19:35.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:35 smithi087 ceph-mon[27372]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T14:19:35.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:35 smithi087 ceph-mon[27372]: osdmap e174: 8 total, 8 up, 8 in 2024-02-13T14:19:35.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:35 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:19:35.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:35 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:19:35.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:35 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:19:35.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:35 smithi087 ceph-mon[39120]: pgmap v212: 164 pgs: 164 active+clean; 582 KiB data, 286 MiB used, 715 GiB / 715 GiB avail 2024-02-13T14:19:35.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:35 smithi087 ceph-mon[39120]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T14:19:35.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:35 smithi087 ceph-mon[39120]: osdmap e174: 8 total, 8 up, 8 in 2024-02-13T14:19:35.743 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:35 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:19:35.743 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:35 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:19:35.743 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:35 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:19:35.872 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:35 smithi148 ceph-mon[31198]: pgmap v212: 164 pgs: 164 active+clean; 582 KiB data, 286 MiB used, 715 GiB / 715 GiB avail 2024-02-13T14:19:35.872 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:35 smithi148 ceph-mon[31198]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T14:19:35.872 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:35 smithi148 ceph-mon[31198]: osdmap e174: 8 total, 8 up, 8 in 2024-02-13T14:19:35.872 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:35 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:19:35.872 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:35 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:19:35.872 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:35 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:19:36.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:36 smithi087 ceph-mon[27372]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:19:36.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:36 smithi087 ceph-mon[27372]: osdmap e175: 8 total, 8 up, 8 in 2024-02-13T14:19:36.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:36 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/3489427628' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:19:36.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:36 smithi087 ceph-mon[27372]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:19:36.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:36 smithi087 ceph-mon[39120]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:19:36.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:36 smithi087 ceph-mon[39120]: osdmap e175: 8 total, 8 up, 8 in 2024-02-13T14:19:36.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:36 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/3489427628' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:19:36.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:36 smithi087 ceph-mon[39120]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:19:36.868 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:36 smithi148 ceph-mon[31198]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:19:36.868 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:36 smithi148 ceph-mon[31198]: osdmap e175: 8 total, 8 up, 8 in 2024-02-13T14:19:36.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:36 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/3489427628' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:19:36.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:36 smithi148 ceph-mon[31198]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:19:37.391 INFO:tasks.workunit.client.0.smithi087.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_lookup_snap PASSED [ 44%] 2024-02-13T14:19:37.491 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:19:37 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: ::ffff:172.21.15.148 - - [13/Feb/2024:14:19:37] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-13T14:19:37.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:37 smithi087 ceph-mon[39120]: pgmap v215: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 287 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:19:37.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:37 smithi087 ceph-mon[39120]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:19:37.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:37 smithi087 ceph-mon[39120]: osdmap e176: 8 total, 8 up, 8 in 2024-02-13T14:19:37.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:37 smithi087 ceph-mon[27372]: pgmap v215: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 287 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:19:37.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:37 smithi087 ceph-mon[27372]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:19:37.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:37 smithi087 ceph-mon[27372]: osdmap e176: 8 total, 8 up, 8 in 2024-02-13T14:19:37.868 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:37 smithi148 ceph-mon[31198]: pgmap v215: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 287 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:19:37.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:37 smithi148 ceph-mon[31198]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:19:37.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:37 smithi148 ceph-mon[31198]: osdmap e176: 8 total, 8 up, 8 in 2024-02-13T14:19:38.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:38 smithi087 ceph-mon[27372]: osdmap e177: 8 total, 8 up, 8 in 2024-02-13T14:19:38.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:38 smithi087 ceph-mon[39120]: osdmap e177: 8 total, 8 up, 8 in 2024-02-13T14:19:38.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:38 smithi148 ceph-mon[31198]: osdmap e177: 8 total, 8 up, 8 in 2024-02-13T14:19:39.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:39 smithi087 ceph-mon[39120]: pgmap v218: 164 pgs: 164 active+clean; 582 KiB data, 287 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:19:39.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:39 smithi087 ceph-mon[39120]: osdmap e178: 8 total, 8 up, 8 in 2024-02-13T14:19:39.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:39 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-13T14:19:39.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:39 smithi087 ceph-mon[27372]: pgmap v218: 164 pgs: 164 active+clean; 582 KiB data, 287 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:19:39.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:39 smithi087 ceph-mon[27372]: osdmap e178: 8 total, 8 up, 8 in 2024-02-13T14:19:39.743 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:39 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-13T14:19:39.868 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:39 smithi148 ceph-mon[31198]: pgmap v218: 164 pgs: 164 active+clean; 582 KiB data, 287 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:19:39.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:39 smithi148 ceph-mon[31198]: osdmap e178: 8 total, 8 up, 8 in 2024-02-13T14:19:39.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:39 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-13T14:19:40.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:40 smithi087 ceph-mon[39120]: osdmap e179: 8 total, 8 up, 8 in 2024-02-13T14:19:40.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:40 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/1651602373' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:19:40.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:40 smithi087 ceph-mon[39120]: pgmap v221: 196 pgs: 196 active+clean; 582 KiB data, 291 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:19:40.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:40 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/1651602373' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:19:40.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:40 smithi087 ceph-mon[39120]: osdmap e180: 8 total, 8 up, 8 in 2024-02-13T14:19:40.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:40 smithi087 ceph-mon[27372]: osdmap e179: 8 total, 8 up, 8 in 2024-02-13T14:19:40.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:40 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/1651602373' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:19:40.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:40 smithi087 ceph-mon[27372]: pgmap v221: 196 pgs: 196 active+clean; 582 KiB data, 291 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:19:40.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:40 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/1651602373' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:19:40.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:40 smithi087 ceph-mon[27372]: osdmap e180: 8 total, 8 up, 8 in 2024-02-13T14:19:40.868 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:40 smithi148 ceph-mon[31198]: osdmap e179: 8 total, 8 up, 8 in 2024-02-13T14:19:40.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:40 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/1651602373' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:19:40.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:40 smithi148 ceph-mon[31198]: pgmap v221: 196 pgs: 196 active+clean; 582 KiB data, 291 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:19:40.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:40 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/1651602373' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:19:40.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:40 smithi148 ceph-mon[31198]: osdmap e180: 8 total, 8 up, 8 in 2024-02-13T14:19:41.410 INFO:tasks.workunit.client.0.smithi087.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_snap_timestamp PASSED [ 45%] 2024-02-13T14:19:41.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:41 smithi087 ceph-mon[39120]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T14:19:41.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:41 smithi087 ceph-mon[39120]: osdmap e181: 8 total, 8 up, 8 in 2024-02-13T14:19:41.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:41 smithi087 ceph-mon[27372]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T14:19:41.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:41 smithi087 ceph-mon[27372]: osdmap e181: 8 total, 8 up, 8 in 2024-02-13T14:19:41.868 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:41 smithi148 ceph-mon[31198]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T14:19:41.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:41 smithi148 ceph-mon[31198]: osdmap e181: 8 total, 8 up, 8 in 2024-02-13T14:19:42.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:42 smithi087 ceph-mon[27372]: pgmap v224: 164 pgs: 164 active+clean; 582 KiB data, 291 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:19:42.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:42 smithi087 ceph-mon[39120]: pgmap v224: 164 pgs: 164 active+clean; 582 KiB data, 291 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:19:42.868 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:42 smithi148 ceph-mon[31198]: pgmap v224: 164 pgs: 164 active+clean; 582 KiB data, 291 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:19:43.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:43 smithi087 ceph-mon[27372]: osdmap e182: 8 total, 8 up, 8 in 2024-02-13T14:19:43.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:43 smithi087 ceph-mon[39120]: osdmap e182: 8 total, 8 up, 8 in 2024-02-13T14:19:43.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:43 smithi148 ceph-mon[31198]: osdmap e182: 8 total, 8 up, 8 in 2024-02-13T14:19:44.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:44 smithi087 ceph-mon[27372]: osdmap e183: 8 total, 8 up, 8 in 2024-02-13T14:19:44.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:44 smithi087 ceph-mon[27372]: pgmap v227: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 291 MiB used, 715 GiB / 715 GiB avail 2024-02-13T14:19:44.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:44 smithi087 ceph-mon[39120]: osdmap e183: 8 total, 8 up, 8 in 2024-02-13T14:19:44.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:44 smithi087 ceph-mon[39120]: pgmap v227: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 291 MiB used, 715 GiB / 715 GiB avail 2024-02-13T14:19:44.868 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:44 smithi148 ceph-mon[31198]: osdmap e183: 8 total, 8 up, 8 in 2024-02-13T14:19:44.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:44 smithi148 ceph-mon[31198]: pgmap v227: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 291 MiB used, 715 GiB / 715 GiB avail 2024-02-13T14:19:45.368 INFO:journalctl@ceph.iscsi.iscsi.a.smithi148.stdout:Feb 13 14:19:44 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-iscsi-iscsi-a[59644]: debug there is no tcmu-runner data available 2024-02-13T14:19:45.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:45 smithi087 ceph-mon[27372]: osdmap e184: 8 total, 8 up, 8 in 2024-02-13T14:19:45.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:45 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/893069856' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:19:45.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:45 smithi087 ceph-mon[27372]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:19:45.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:45 smithi087 ceph-mon[27372]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:19:45.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:45 smithi087 ceph-mon[39120]: osdmap e184: 8 total, 8 up, 8 in 2024-02-13T14:19:45.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:45 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/893069856' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:19:45.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:45 smithi087 ceph-mon[39120]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:19:45.743 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:45 smithi087 ceph-mon[39120]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:19:45.868 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:45 smithi148 ceph-mon[31198]: osdmap e184: 8 total, 8 up, 8 in 2024-02-13T14:19:45.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:45 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/893069856' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:19:45.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:45 smithi148 ceph-mon[31198]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:19:45.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:45 smithi148 ceph-mon[31198]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:19:46.458 INFO:tasks.workunit.client.0.smithi087.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_remove_snap PASSED [ 46%] 2024-02-13T14:19:46.741 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:46 smithi087 ceph-mon[27372]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:19:46.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:46 smithi087 ceph-mon[27372]: osdmap e185: 8 total, 8 up, 8 in 2024-02-13T14:19:46.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:46 smithi087 ceph-mon[27372]: pgmap v230: 196 pgs: 196 active+clean; 582 KiB data, 292 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:19:46.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:46 smithi087 ceph-mon[27372]: osdmap e186: 8 total, 8 up, 8 in 2024-02-13T14:19:46.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:46 smithi087 ceph-mon[39120]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:19:46.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:46 smithi087 ceph-mon[39120]: osdmap e185: 8 total, 8 up, 8 in 2024-02-13T14:19:46.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:46 smithi087 ceph-mon[39120]: pgmap v230: 196 pgs: 196 active+clean; 582 KiB data, 292 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:19:46.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:46 smithi087 ceph-mon[39120]: osdmap e186: 8 total, 8 up, 8 in 2024-02-13T14:19:46.868 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:46 smithi148 ceph-mon[31198]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:19:46.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:46 smithi148 ceph-mon[31198]: osdmap e185: 8 total, 8 up, 8 in 2024-02-13T14:19:46.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:46 smithi148 ceph-mon[31198]: pgmap v230: 196 pgs: 196 active+clean; 582 KiB data, 292 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:19:46.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:46 smithi148 ceph-mon[31198]: osdmap e186: 8 total, 8 up, 8 in 2024-02-13T14:19:47.492 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:19:47 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: ::ffff:172.21.15.148 - - [13/Feb/2024:14:19:47] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-13T14:19:48.741 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:48 smithi087 ceph-mon[27372]: osdmap e187: 8 total, 8 up, 8 in 2024-02-13T14:19:48.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:48 smithi087 ceph-mon[27372]: pgmap v233: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 292 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:19:48.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:48 smithi087 ceph-mon[39120]: osdmap e187: 8 total, 8 up, 8 in 2024-02-13T14:19:48.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:48 smithi087 ceph-mon[39120]: pgmap v233: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 292 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:19:48.868 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:48 smithi148 ceph-mon[31198]: osdmap e187: 8 total, 8 up, 8 in 2024-02-13T14:19:48.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:48 smithi148 ceph-mon[31198]: pgmap v233: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 292 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:19:49.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:49 smithi087 ceph-mon[27372]: osdmap e188: 8 total, 8 up, 8 in 2024-02-13T14:19:49.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:49 smithi087 ceph-mon[39120]: osdmap e188: 8 total, 8 up, 8 in 2024-02-13T14:19:49.868 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:49 smithi148 ceph-mon[31198]: osdmap e188: 8 total, 8 up, 8 in 2024-02-13T14:19:50.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:50 smithi087 ceph-mon[39120]: osdmap e189: 8 total, 8 up, 8 in 2024-02-13T14:19:50.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:50 smithi087 ceph-mon[39120]: pgmap v236: 196 pgs: 196 active+clean; 582 KiB data, 293 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-13T14:19:50.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:50 smithi087 ceph-mon[27372]: osdmap e189: 8 total, 8 up, 8 in 2024-02-13T14:19:50.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:50 smithi087 ceph-mon[27372]: pgmap v236: 196 pgs: 196 active+clean; 582 KiB data, 293 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-13T14:19:50.868 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:50 smithi148 ceph-mon[31198]: osdmap e189: 8 total, 8 up, 8 in 2024-02-13T14:19:50.868 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:50 smithi148 ceph-mon[31198]: pgmap v236: 196 pgs: 196 active+clean; 582 KiB data, 293 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-13T14:19:51.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:51 smithi087 ceph-mon[27372]: osdmap e190: 8 total, 8 up, 8 in 2024-02-13T14:19:51.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:51 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/1215446633' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:19:51.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:51 smithi087 ceph-mon[39120]: osdmap e190: 8 total, 8 up, 8 in 2024-02-13T14:19:51.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:51 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/1215446633' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:19:51.868 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:51 smithi148 ceph-mon[31198]: osdmap e190: 8 total, 8 up, 8 in 2024-02-13T14:19:51.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:51 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/1215446633' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:19:52.492 INFO:tasks.workunit.client.0.smithi087.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_snap_rollback PASSED [ 47%] 2024-02-13T14:19:52.868 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:52 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/1215446633' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:19:52.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:52 smithi148 ceph-mon[31198]: osdmap e191: 8 total, 8 up, 8 in 2024-02-13T14:19:52.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:52 smithi148 ceph-mon[31198]: pgmap v239: 196 pgs: 196 active+clean; 582 KiB data, 293 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-13T14:19:52.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:52 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/1215446633' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:19:52.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:52 smithi087 ceph-mon[39120]: osdmap e191: 8 total, 8 up, 8 in 2024-02-13T14:19:52.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:52 smithi087 ceph-mon[39120]: pgmap v239: 196 pgs: 196 active+clean; 582 KiB data, 293 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-13T14:19:52.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:52 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/1215446633' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:19:52.993 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:52 smithi087 ceph-mon[27372]: osdmap e191: 8 total, 8 up, 8 in 2024-02-13T14:19:52.993 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:52 smithi087 ceph-mon[27372]: pgmap v239: 196 pgs: 196 active+clean; 582 KiB data, 293 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-13T14:19:53.868 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:53 smithi148 ceph-mon[31198]: osdmap e192: 8 total, 8 up, 8 in 2024-02-13T14:19:53.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:53 smithi087 ceph-mon[27372]: osdmap e192: 8 total, 8 up, 8 in 2024-02-13T14:19:53.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:53 smithi087 ceph-mon[39120]: osdmap e192: 8 total, 8 up, 8 in 2024-02-13T14:19:54.868 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:54 smithi148 ceph-mon[31198]: osdmap e193: 8 total, 8 up, 8 in 2024-02-13T14:19:54.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:54 smithi148 ceph-mon[31198]: pgmap v242: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 293 MiB used, 715 GiB / 715 GiB avail 2024-02-13T14:19:54.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:54 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-13T14:19:54.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:54 smithi087 ceph-mon[27372]: osdmap e193: 8 total, 8 up, 8 in 2024-02-13T14:19:54.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:54 smithi087 ceph-mon[27372]: pgmap v242: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 293 MiB used, 715 GiB / 715 GiB avail 2024-02-13T14:19:54.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:54 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-13T14:19:54.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:54 smithi087 ceph-mon[39120]: osdmap e193: 8 total, 8 up, 8 in 2024-02-13T14:19:54.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:54 smithi087 ceph-mon[39120]: pgmap v242: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 293 MiB used, 715 GiB / 715 GiB avail 2024-02-13T14:19:54.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:54 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-13T14:19:55.368 INFO:journalctl@ceph.iscsi.iscsi.a.smithi148.stdout:Feb 13 14:19:54 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-iscsi-iscsi-a[59644]: debug there is no tcmu-runner data available 2024-02-13T14:19:55.868 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:55 smithi148 ceph-mon[31198]: osdmap e194: 8 total, 8 up, 8 in 2024-02-13T14:19:55.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:55 smithi148 ceph-mon[31198]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:19:55.991 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:55 smithi087 ceph-mon[27372]: osdmap e194: 8 total, 8 up, 8 in 2024-02-13T14:19:55.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:55 smithi087 ceph-mon[27372]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:19:55.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:55 smithi087 ceph-mon[39120]: osdmap e194: 8 total, 8 up, 8 in 2024-02-13T14:19:55.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:55 smithi087 ceph-mon[39120]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:19:56.868 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:56 smithi148 ceph-mon[31198]: osdmap e195: 8 total, 8 up, 8 in 2024-02-13T14:19:56.882 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:56 smithi148 ceph-mon[31198]: pgmap v245: 196 pgs: 196 active+clean; 582 KiB data, 293 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-13T14:19:56.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:56 smithi087 ceph-mon[27372]: osdmap e195: 8 total, 8 up, 8 in 2024-02-13T14:19:56.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:56 smithi087 ceph-mon[27372]: pgmap v245: 196 pgs: 196 active+clean; 582 KiB data, 293 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-13T14:19:56.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:56 smithi087 ceph-mon[39120]: osdmap e195: 8 total, 8 up, 8 in 2024-02-13T14:19:56.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:56 smithi087 ceph-mon[39120]: pgmap v245: 196 pgs: 196 active+clean; 582 KiB data, 293 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-13T14:19:57.492 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:19:57 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: ::ffff:172.21.15.148 - - [13/Feb/2024:14:19:57] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-13T14:19:57.868 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:57 smithi148 ceph-mon[31198]: osdmap e196: 8 total, 8 up, 8 in 2024-02-13T14:19:57.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:57 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/3367120927' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:19:57.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:57 smithi148 ceph-mon[31198]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:19:57.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:57 smithi087 ceph-mon[39120]: osdmap e196: 8 total, 8 up, 8 in 2024-02-13T14:19:57.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:57 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/3367120927' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:19:57.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:57 smithi087 ceph-mon[39120]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:19:57.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:57 smithi087 ceph-mon[27372]: osdmap e196: 8 total, 8 up, 8 in 2024-02-13T14:19:57.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:57 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/3367120927' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:19:57.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:57 smithi087 ceph-mon[27372]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:19:58.553 INFO:tasks.workunit.client.0.smithi087.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_snap_rollback_removed PASSED [ 48%] 2024-02-13T14:19:58.868 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:58 smithi148 ceph-mon[31198]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:19:58.868 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:58 smithi148 ceph-mon[31198]: osdmap e197: 8 total, 8 up, 8 in 2024-02-13T14:19:58.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:58 smithi148 ceph-mon[31198]: pgmap v248: 196 pgs: 196 active+clean; 582 KiB data, 293 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-13T14:19:58.991 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:58 smithi087 ceph-mon[39120]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:19:58.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:58 smithi087 ceph-mon[39120]: osdmap e197: 8 total, 8 up, 8 in 2024-02-13T14:19:58.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:58 smithi087 ceph-mon[39120]: pgmap v248: 196 pgs: 196 active+clean; 582 KiB data, 293 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-13T14:19:58.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:58 smithi087 ceph-mon[27372]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:19:58.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:58 smithi087 ceph-mon[27372]: osdmap e197: 8 total, 8 up, 8 in 2024-02-13T14:19:58.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:58 smithi087 ceph-mon[27372]: pgmap v248: 196 pgs: 196 active+clean; 582 KiB data, 293 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-13T14:19:59.868 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:19:59 smithi148 ceph-mon[31198]: osdmap e198: 8 total, 8 up, 8 in 2024-02-13T14:19:59.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:19:59 smithi087 ceph-mon[27372]: osdmap e198: 8 total, 8 up, 8 in 2024-02-13T14:19:59.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:19:59 smithi087 ceph-mon[39120]: osdmap e198: 8 total, 8 up, 8 in 2024-02-13T14:20:00.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:00 smithi148 ceph-mon[31198]: osdmap e199: 8 total, 8 up, 8 in 2024-02-13T14:20:00.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:00 smithi148 ceph-mon[31198]: pgmap v251: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 294 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:20:00.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:00 smithi148 ceph-mon[31198]: Health detail: HEALTH_WARN 2 pool(s) do not have an application enabled 2024-02-13T14:20:00.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:00 smithi148 ceph-mon[31198]: [WRN] POOL_APP_NOT_ENABLED: 2 pool(s) do not have an application enabled 2024-02-13T14:20:00.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:00 smithi148 ceph-mon[31198]: application not enabled on pool 'rbd' 2024-02-13T14:20:00.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:00 smithi148 ceph-mon[31198]: application not enabled on pool 'test_pool' 2024-02-13T14:20:00.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:00 smithi148 ceph-mon[31198]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-02-13T14:20:00.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:00 smithi148 ceph-mon[31198]: osdmap e200: 8 total, 8 up, 8 in 2024-02-13T14:20:00.991 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:00 smithi087 ceph-mon[39120]: osdmap e199: 8 total, 8 up, 8 in 2024-02-13T14:20:00.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:00 smithi087 ceph-mon[39120]: pgmap v251: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 294 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:20:00.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:00 smithi087 ceph-mon[39120]: Health detail: HEALTH_WARN 2 pool(s) do not have an application enabled 2024-02-13T14:20:00.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:00 smithi087 ceph-mon[39120]: [WRN] POOL_APP_NOT_ENABLED: 2 pool(s) do not have an application enabled 2024-02-13T14:20:00.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:00 smithi087 ceph-mon[39120]: application not enabled on pool 'rbd' 2024-02-13T14:20:00.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:00 smithi087 ceph-mon[39120]: application not enabled on pool 'test_pool' 2024-02-13T14:20:00.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:00 smithi087 ceph-mon[39120]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-02-13T14:20:00.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:00 smithi087 ceph-mon[39120]: osdmap e200: 8 total, 8 up, 8 in 2024-02-13T14:20:00.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:00 smithi087 ceph-mon[27372]: osdmap e199: 8 total, 8 up, 8 in 2024-02-13T14:20:00.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:00 smithi087 ceph-mon[27372]: pgmap v251: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 294 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:20:00.993 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:00 smithi087 ceph-mon[27372]: Health detail: HEALTH_WARN 2 pool(s) do not have an application enabled 2024-02-13T14:20:00.993 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:00 smithi087 ceph-mon[27372]: [WRN] POOL_APP_NOT_ENABLED: 2 pool(s) do not have an application enabled 2024-02-13T14:20:00.993 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:00 smithi087 ceph-mon[27372]: application not enabled on pool 'rbd' 2024-02-13T14:20:00.993 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:00 smithi087 ceph-mon[27372]: application not enabled on pool 'test_pool' 2024-02-13T14:20:00.993 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:00 smithi087 ceph-mon[27372]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-02-13T14:20:00.993 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:00 smithi087 ceph-mon[27372]: osdmap e200: 8 total, 8 up, 8 in 2024-02-13T14:20:02.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:02 smithi148 ceph-mon[31198]: osdmap e201: 8 total, 8 up, 8 in 2024-02-13T14:20:02.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:02 smithi148 ceph-mon[31198]: pgmap v254: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 294 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:20:02.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:02 smithi087 ceph-mon[27372]: osdmap e201: 8 total, 8 up, 8 in 2024-02-13T14:20:02.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:02 smithi087 ceph-mon[27372]: pgmap v254: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 294 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:20:02.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:02 smithi087 ceph-mon[39120]: osdmap e201: 8 total, 8 up, 8 in 2024-02-13T14:20:02.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:02 smithi087 ceph-mon[39120]: pgmap v254: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 294 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:20:03.868 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:03 smithi148 ceph-mon[31198]: osdmap e202: 8 total, 8 up, 8 in 2024-02-13T14:20:03.868 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:03 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/2224050578' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:20:03.868 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:03 smithi148 ceph-mon[31198]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:20:03.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:03 smithi087 ceph-mon[39120]: osdmap e202: 8 total, 8 up, 8 in 2024-02-13T14:20:03.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:03 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/2224050578' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:20:03.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:03 smithi087 ceph-mon[39120]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:20:03.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:03 smithi087 ceph-mon[27372]: osdmap e202: 8 total, 8 up, 8 in 2024-02-13T14:20:03.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:03 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/2224050578' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:20:03.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:03 smithi087 ceph-mon[27372]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:20:04.592 INFO:tasks.workunit.client.0.smithi087.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_snap_read PASSED [ 50%] 2024-02-13T14:20:04.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:04 smithi148 ceph-mon[31198]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:20:04.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:04 smithi148 ceph-mon[31198]: osdmap e203: 8 total, 8 up, 8 in 2024-02-13T14:20:04.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:04 smithi148 ceph-mon[31198]: pgmap v257: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 294 MiB used, 715 GiB / 715 GiB avail 2024-02-13T14:20:04.991 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:04 smithi087 ceph-mon[27372]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:20:04.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:04 smithi087 ceph-mon[27372]: osdmap e203: 8 total, 8 up, 8 in 2024-02-13T14:20:04.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:04 smithi087 ceph-mon[27372]: pgmap v257: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 294 MiB used, 715 GiB / 715 GiB avail 2024-02-13T14:20:04.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:04 smithi087 ceph-mon[39120]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:20:04.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:04 smithi087 ceph-mon[39120]: osdmap e203: 8 total, 8 up, 8 in 2024-02-13T14:20:04.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:04 smithi087 ceph-mon[39120]: pgmap v257: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 294 MiB used, 715 GiB / 715 GiB avail 2024-02-13T14:20:05.368 INFO:journalctl@ceph.iscsi.iscsi.a.smithi148.stdout:Feb 13 14:20:04 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-iscsi-iscsi-a[59644]: debug there is no tcmu-runner data available 2024-02-13T14:20:05.868 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:05 smithi148 ceph-mon[31198]: osdmap e204: 8 total, 8 up, 8 in 2024-02-13T14:20:05.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:05 smithi148 ceph-mon[31198]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:20:05.991 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:05 smithi087 ceph-mon[27372]: osdmap e204: 8 total, 8 up, 8 in 2024-02-13T14:20:05.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:05 smithi087 ceph-mon[27372]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:20:05.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:05 smithi087 ceph-mon[39120]: osdmap e204: 8 total, 8 up, 8 in 2024-02-13T14:20:05.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:05 smithi087 ceph-mon[39120]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:20:06.868 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:06 smithi148 ceph-mon[31198]: osdmap e205: 8 total, 8 up, 8 in 2024-02-13T14:20:06.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:06 smithi148 ceph-mon[31198]: pgmap v260: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 294 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:20:06.991 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:06 smithi087 ceph-mon[27372]: osdmap e205: 8 total, 8 up, 8 in 2024-02-13T14:20:06.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:06 smithi087 ceph-mon[27372]: pgmap v260: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 294 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:20:06.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:06 smithi087 ceph-mon[39120]: osdmap e205: 8 total, 8 up, 8 in 2024-02-13T14:20:06.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:06 smithi087 ceph-mon[39120]: pgmap v260: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 294 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:20:07.491 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:20:07 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: ::ffff:172.21.15.148 - - [13/Feb/2024:14:20:07] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-13T14:20:07.868 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:07 smithi148 ceph-mon[31198]: osdmap e206: 8 total, 8 up, 8 in 2024-02-13T14:20:07.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:07 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/1079857832' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:20:07.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:07 smithi087 ceph-mon[39120]: osdmap e206: 8 total, 8 up, 8 in 2024-02-13T14:20:07.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:07 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/1079857832' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:20:07.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:07 smithi087 ceph-mon[27372]: osdmap e206: 8 total, 8 up, 8 in 2024-02-13T14:20:07.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:07 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/1079857832' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:20:08.607 INFO:tasks.workunit.client.0.smithi087.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_set_omap PASSED [ 51%] 2024-02-13T14:20:08.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:08 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/1079857832' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:20:08.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:08 smithi148 ceph-mon[31198]: osdmap e207: 8 total, 8 up, 8 in 2024-02-13T14:20:08.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:08 smithi148 ceph-mon[31198]: pgmap v263: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 294 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:20:08.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:08 smithi148 ceph-mon[31198]: osdmap e208: 8 total, 8 up, 8 in 2024-02-13T14:20:08.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:08 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/1079857832' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:20:08.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:08 smithi087 ceph-mon[27372]: osdmap e207: 8 total, 8 up, 8 in 2024-02-13T14:20:08.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:08 smithi087 ceph-mon[27372]: pgmap v263: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 294 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:20:08.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:08 smithi087 ceph-mon[27372]: osdmap e208: 8 total, 8 up, 8 in 2024-02-13T14:20:08.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:08 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/1079857832' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:20:08.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:08 smithi087 ceph-mon[39120]: osdmap e207: 8 total, 8 up, 8 in 2024-02-13T14:20:08.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:08 smithi087 ceph-mon[39120]: pgmap v263: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 294 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:20:08.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:08 smithi087 ceph-mon[39120]: osdmap e208: 8 total, 8 up, 8 in 2024-02-13T14:20:09.868 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:09 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-13T14:20:09.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:09 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-13T14:20:09.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:09 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-13T14:20:10.991 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:10 smithi087 ceph-mon[27372]: osdmap e209: 8 total, 8 up, 8 in 2024-02-13T14:20:10.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:10 smithi087 ceph-mon[27372]: pgmap v266: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 299 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:20:10.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:10 smithi087 ceph-mon[39120]: osdmap e209: 8 total, 8 up, 8 in 2024-02-13T14:20:10.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:10 smithi087 ceph-mon[39120]: pgmap v266: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 299 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:20:11.118 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:10 smithi148 ceph-mon[31198]: osdmap e209: 8 total, 8 up, 8 in 2024-02-13T14:20:11.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:10 smithi148 ceph-mon[31198]: pgmap v266: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 299 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:20:11.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:11 smithi087 ceph-mon[27372]: osdmap e210: 8 total, 8 up, 8 in 2024-02-13T14:20:11.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:11 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/2192553537' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:20:11.993 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:11 smithi087 ceph-mon[39120]: osdmap e210: 8 total, 8 up, 8 in 2024-02-13T14:20:11.993 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:11 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/2192553537' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:20:12.118 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:11 smithi148 ceph-mon[31198]: osdmap e210: 8 total, 8 up, 8 in 2024-02-13T14:20:12.118 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:11 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/2192553537' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:20:12.644 INFO:tasks.workunit.client.0.smithi087.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_set_omap_aio PASSED [ 52%] 2024-02-13T14:20:12.991 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:12 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/2192553537' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:20:12.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:12 smithi087 ceph-mon[27372]: osdmap e211: 8 total, 8 up, 8 in 2024-02-13T14:20:12.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:12 smithi087 ceph-mon[27372]: pgmap v269: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 299 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:20:12.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:12 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/2192553537' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:20:12.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:12 smithi087 ceph-mon[39120]: osdmap e211: 8 total, 8 up, 8 in 2024-02-13T14:20:12.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:12 smithi087 ceph-mon[39120]: pgmap v269: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 299 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:20:13.118 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:12 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/2192553537' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:20:13.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:12 smithi148 ceph-mon[31198]: osdmap e211: 8 total, 8 up, 8 in 2024-02-13T14:20:13.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:12 smithi148 ceph-mon[31198]: pgmap v269: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 299 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:20:13.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:13 smithi087 ceph-mon[27372]: osdmap e212: 8 total, 8 up, 8 in 2024-02-13T14:20:13.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:13 smithi087 ceph-mon[39120]: osdmap e212: 8 total, 8 up, 8 in 2024-02-13T14:20:14.118 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:13 smithi148 ceph-mon[31198]: osdmap e212: 8 total, 8 up, 8 in 2024-02-13T14:20:14.970 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:14 smithi148 ceph-mon[31198]: osdmap e213: 8 total, 8 up, 8 in 2024-02-13T14:20:14.970 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:14 smithi148 ceph-mon[31198]: pgmap v272: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 299 MiB used, 715 GiB / 715 GiB avail 2024-02-13T14:20:14.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:14 smithi087 ceph-mon[27372]: osdmap e213: 8 total, 8 up, 8 in 2024-02-13T14:20:14.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:14 smithi087 ceph-mon[27372]: pgmap v272: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 299 MiB used, 715 GiB / 715 GiB avail 2024-02-13T14:20:14.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:14 smithi087 ceph-mon[39120]: osdmap e213: 8 total, 8 up, 8 in 2024-02-13T14:20:14.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:14 smithi087 ceph-mon[39120]: pgmap v272: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 299 MiB used, 715 GiB / 715 GiB avail 2024-02-13T14:20:15.368 INFO:journalctl@ceph.iscsi.iscsi.a.smithi148.stdout:Feb 13 14:20:14 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-iscsi-iscsi-a[59644]: debug there is no tcmu-runner data available 2024-02-13T14:20:15.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:15 smithi087 ceph-mon[27372]: osdmap e214: 8 total, 8 up, 8 in 2024-02-13T14:20:15.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:15 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/698982557' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:20:15.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:15 smithi087 ceph-mon[27372]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:20:15.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:15 smithi087 ceph-mon[39120]: osdmap e214: 8 total, 8 up, 8 in 2024-02-13T14:20:15.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:15 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/698982557' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:20:15.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:15 smithi087 ceph-mon[39120]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:20:16.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:15 smithi148 ceph-mon[31198]: osdmap e214: 8 total, 8 up, 8 in 2024-02-13T14:20:16.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:15 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/698982557' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:20:16.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:15 smithi148 ceph-mon[31198]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:20:16.679 INFO:tasks.workunit.client.0.smithi087.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_write_ops PASSED [ 53%] 2024-02-13T14:20:16.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:16 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/698982557' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:20:16.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:16 smithi087 ceph-mon[27372]: osdmap e215: 8 total, 8 up, 8 in 2024-02-13T14:20:16.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:16 smithi087 ceph-mon[27372]: pgmap v275: 196 pgs: 196 active+clean; 582 KiB data, 300 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-02-13T14:20:16.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:16 smithi087 ceph-mon[27372]: osdmap e216: 8 total, 8 up, 8 in 2024-02-13T14:20:16.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:16 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/698982557' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:20:16.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:16 smithi087 ceph-mon[39120]: osdmap e215: 8 total, 8 up, 8 in 2024-02-13T14:20:16.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:16 smithi087 ceph-mon[39120]: pgmap v275: 196 pgs: 196 active+clean; 582 KiB data, 300 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-02-13T14:20:16.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:16 smithi087 ceph-mon[39120]: osdmap e216: 8 total, 8 up, 8 in 2024-02-13T14:20:17.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:16 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/698982557' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:20:17.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:16 smithi148 ceph-mon[31198]: osdmap e215: 8 total, 8 up, 8 in 2024-02-13T14:20:17.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:16 smithi148 ceph-mon[31198]: pgmap v275: 196 pgs: 196 active+clean; 582 KiB data, 300 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-02-13T14:20:17.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:16 smithi148 ceph-mon[31198]: osdmap e216: 8 total, 8 up, 8 in 2024-02-13T14:20:17.491 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:20:17 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: ::ffff:172.21.15.148 - - [13/Feb/2024:14:20:17] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-13T14:20:18.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:18 smithi087 ceph-mon[27372]: osdmap e217: 8 total, 8 up, 8 in 2024-02-13T14:20:18.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:18 smithi087 ceph-mon[27372]: pgmap v278: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 300 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:20:18.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:18 smithi087 ceph-mon[39120]: osdmap e217: 8 total, 8 up, 8 in 2024-02-13T14:20:18.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:18 smithi087 ceph-mon[39120]: pgmap v278: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 300 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:20:19.118 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:18 smithi148 ceph-mon[31198]: osdmap e217: 8 total, 8 up, 8 in 2024-02-13T14:20:19.118 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:18 smithi148 ceph-mon[31198]: pgmap v278: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 300 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:20:19.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:19 smithi087 ceph-mon[27372]: osdmap e218: 8 total, 8 up, 8 in 2024-02-13T14:20:19.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:19 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/321247625' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:20:19.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:19 smithi087 ceph-mon[39120]: osdmap e218: 8 total, 8 up, 8 in 2024-02-13T14:20:19.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:19 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/321247625' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:20:20.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:19 smithi148 ceph-mon[31198]: osdmap e218: 8 total, 8 up, 8 in 2024-02-13T14:20:20.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:19 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/321247625' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:20:20.703 INFO:tasks.workunit.client.0.smithi087.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_execute_op PASSED [ 54%] 2024-02-13T14:20:20.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:20 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/321247625' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:20:20.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:20 smithi087 ceph-mon[39120]: osdmap e219: 8 total, 8 up, 8 in 2024-02-13T14:20:20.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:20 smithi087 ceph-mon[39120]: pgmap v281: 196 pgs: 196 active+clean; 582 KiB data, 300 MiB used, 715 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-02-13T14:20:20.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:20 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/321247625' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:20:20.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:20 smithi087 ceph-mon[27372]: osdmap e219: 8 total, 8 up, 8 in 2024-02-13T14:20:20.993 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:20 smithi087 ceph-mon[27372]: pgmap v281: 196 pgs: 196 active+clean; 582 KiB data, 300 MiB used, 715 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-02-13T14:20:21.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:20 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/321247625' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:20:21.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:20 smithi148 ceph-mon[31198]: osdmap e219: 8 total, 8 up, 8 in 2024-02-13T14:20:21.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:20 smithi148 ceph-mon[31198]: pgmap v281: 196 pgs: 196 active+clean; 582 KiB data, 300 MiB used, 715 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-02-13T14:20:21.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:21 smithi087 ceph-mon[27372]: osdmap e220: 8 total, 8 up, 8 in 2024-02-13T14:20:21.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:21 smithi087 ceph-mon[39120]: osdmap e220: 8 total, 8 up, 8 in 2024-02-13T14:20:22.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:21 smithi148 ceph-mon[31198]: osdmap e220: 8 total, 8 up, 8 in 2024-02-13T14:20:22.991 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:22 smithi087 ceph-mon[27372]: osdmap e221: 8 total, 8 up, 8 in 2024-02-13T14:20:22.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:22 smithi087 ceph-mon[27372]: pgmap v284: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 300 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:20:22.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:22 smithi087 ceph-mon[39120]: osdmap e221: 8 total, 8 up, 8 in 2024-02-13T14:20:22.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:22 smithi087 ceph-mon[39120]: pgmap v284: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 300 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:20:23.118 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:22 smithi148 ceph-mon[31198]: osdmap e221: 8 total, 8 up, 8 in 2024-02-13T14:20:23.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:22 smithi148 ceph-mon[31198]: pgmap v284: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 300 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:20:23.991 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:23 smithi087 ceph-mon[27372]: osdmap e222: 8 total, 8 up, 8 in 2024-02-13T14:20:23.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:23 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/2501826155' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:20:23.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:23 smithi087 ceph-mon[27372]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:20:23.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:23 smithi087 ceph-mon[39120]: osdmap e222: 8 total, 8 up, 8 in 2024-02-13T14:20:23.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:23 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/2501826155' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:20:23.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:23 smithi087 ceph-mon[39120]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:20:24.118 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:23 smithi148 ceph-mon[31198]: osdmap e222: 8 total, 8 up, 8 in 2024-02-13T14:20:24.118 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:23 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/2501826155' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:20:24.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:23 smithi148 ceph-mon[31198]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:20:24.727 INFO:tasks.workunit.client.0.smithi087.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_writesame_op PASSED [ 55%] 2024-02-13T14:20:24.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:24 smithi087 ceph-mon[39120]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:20:24.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:24 smithi087 ceph-mon[39120]: osdmap e223: 8 total, 8 up, 8 in 2024-02-13T14:20:24.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:24 smithi087 ceph-mon[39120]: pgmap v287: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 300 MiB used, 715 GiB / 715 GiB avail 2024-02-13T14:20:24.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:24 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-13T14:20:24.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:24 smithi087 ceph-mon[39120]: osdmap e224: 8 total, 8 up, 8 in 2024-02-13T14:20:24.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:24 smithi087 ceph-mon[27372]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:20:24.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:24 smithi087 ceph-mon[27372]: osdmap e223: 8 total, 8 up, 8 in 2024-02-13T14:20:24.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:24 smithi087 ceph-mon[27372]: pgmap v287: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 300 MiB used, 715 GiB / 715 GiB avail 2024-02-13T14:20:24.993 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:24 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-13T14:20:24.993 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:24 smithi087 ceph-mon[27372]: osdmap e224: 8 total, 8 up, 8 in 2024-02-13T14:20:25.118 INFO:journalctl@ceph.iscsi.iscsi.a.smithi148.stdout:Feb 13 14:20:24 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-iscsi-iscsi-a[59644]: debug there is no tcmu-runner data available 2024-02-13T14:20:25.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:24 smithi148 ceph-mon[31198]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:20:25.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:24 smithi148 ceph-mon[31198]: osdmap e223: 8 total, 8 up, 8 in 2024-02-13T14:20:25.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:24 smithi148 ceph-mon[31198]: pgmap v287: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 300 MiB used, 715 GiB / 715 GiB avail 2024-02-13T14:20:25.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:24 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-13T14:20:25.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:24 smithi148 ceph-mon[31198]: osdmap e224: 8 total, 8 up, 8 in 2024-02-13T14:20:25.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:25 smithi087 ceph-mon[27372]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:20:25.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:25 smithi087 ceph-mon[39120]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:20:26.020 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:25 smithi148 ceph-mon[31198]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:20:27.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:26 smithi148 ceph-mon[31198]: osdmap e225: 8 total, 8 up, 8 in 2024-02-13T14:20:27.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:26 smithi148 ceph-mon[31198]: pgmap v290: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 309 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:20:27.222 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:26 smithi087 ceph-mon[27372]: osdmap e225: 8 total, 8 up, 8 in 2024-02-13T14:20:27.222 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:26 smithi087 ceph-mon[27372]: pgmap v290: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 309 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:20:27.222 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:26 smithi087 ceph-mon[39120]: osdmap e225: 8 total, 8 up, 8 in 2024-02-13T14:20:27.222 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:26 smithi087 ceph-mon[39120]: pgmap v290: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 309 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:20:27.491 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:20:27 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: ::ffff:172.21.15.148 - - [13/Feb/2024:14:20:27] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-13T14:20:28.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:27 smithi148 ceph-mon[31198]: osdmap e226: 8 total, 8 up, 8 in 2024-02-13T14:20:28.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:27 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/2544465960' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:20:28.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:27 smithi148 ceph-mon[31198]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:20:28.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:27 smithi087 ceph-mon[27372]: osdmap e226: 8 total, 8 up, 8 in 2024-02-13T14:20:28.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:27 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/2544465960' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:20:28.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:27 smithi087 ceph-mon[27372]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:20:28.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:27 smithi087 ceph-mon[39120]: osdmap e226: 8 total, 8 up, 8 in 2024-02-13T14:20:28.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:27 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/2544465960' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:20:28.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:27 smithi087 ceph-mon[39120]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:20:28.778 INFO:tasks.workunit.client.0.smithi087.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_get_omap_vals_by_keys PASSED [ 56%] 2024-02-13T14:20:29.118 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:28 smithi148 ceph-mon[31198]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:20:29.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:28 smithi148 ceph-mon[31198]: osdmap e227: 8 total, 8 up, 8 in 2024-02-13T14:20:29.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:28 smithi148 ceph-mon[31198]: pgmap v293: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 309 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:20:29.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:28 smithi087 ceph-mon[27372]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:20:29.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:28 smithi087 ceph-mon[27372]: osdmap e227: 8 total, 8 up, 8 in 2024-02-13T14:20:29.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:28 smithi087 ceph-mon[27372]: pgmap v293: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 309 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:20:29.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:28 smithi087 ceph-mon[39120]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:20:29.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:28 smithi087 ceph-mon[39120]: osdmap e227: 8 total, 8 up, 8 in 2024-02-13T14:20:29.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:28 smithi087 ceph-mon[39120]: pgmap v293: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 309 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:20:30.118 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:29 smithi148 ceph-mon[31198]: osdmap e228: 8 total, 8 up, 8 in 2024-02-13T14:20:30.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:29 smithi087 ceph-mon[27372]: osdmap e228: 8 total, 8 up, 8 in 2024-02-13T14:20:30.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:29 smithi087 ceph-mon[39120]: osdmap e228: 8 total, 8 up, 8 in 2024-02-13T14:20:31.118 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:30 smithi148 ceph-mon[31198]: osdmap e229: 8 total, 8 up, 8 in 2024-02-13T14:20:31.118 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:30 smithi148 ceph-mon[31198]: pgmap v296: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 310 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:20:31.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:30 smithi087 ceph-mon[27372]: osdmap e229: 8 total, 8 up, 8 in 2024-02-13T14:20:31.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:30 smithi087 ceph-mon[27372]: pgmap v296: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 310 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:20:31.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:30 smithi087 ceph-mon[39120]: osdmap e229: 8 total, 8 up, 8 in 2024-02-13T14:20:31.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:30 smithi087 ceph-mon[39120]: pgmap v296: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 310 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:20:32.118 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:31 smithi148 ceph-mon[31198]: osdmap e230: 8 total, 8 up, 8 in 2024-02-13T14:20:32.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:31 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/3935418915' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:20:32.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:31 smithi148 ceph-mon[31198]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:20:32.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:31 smithi087 ceph-mon[27372]: osdmap e230: 8 total, 8 up, 8 in 2024-02-13T14:20:32.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:31 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/3935418915' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:20:32.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:31 smithi087 ceph-mon[27372]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:20:32.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:31 smithi087 ceph-mon[39120]: osdmap e230: 8 total, 8 up, 8 in 2024-02-13T14:20:32.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:31 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/3935418915' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:20:32.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:31 smithi087 ceph-mon[39120]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:20:32.806 INFO:tasks.workunit.client.0.smithi087.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_get_omap_keys PASSED [ 57%] 2024-02-13T14:20:33.118 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:32 smithi148 ceph-mon[31198]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:20:33.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:32 smithi148 ceph-mon[31198]: osdmap e231: 8 total, 8 up, 8 in 2024-02-13T14:20:33.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:32 smithi148 ceph-mon[31198]: pgmap v299: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 310 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:20:33.241 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:32 smithi087 ceph-mon[27372]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:20:33.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:32 smithi087 ceph-mon[27372]: osdmap e231: 8 total, 8 up, 8 in 2024-02-13T14:20:33.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:32 smithi087 ceph-mon[27372]: pgmap v299: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 310 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:20:33.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:32 smithi087 ceph-mon[39120]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:20:33.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:32 smithi087 ceph-mon[39120]: osdmap e231: 8 total, 8 up, 8 in 2024-02-13T14:20:33.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:32 smithi087 ceph-mon[39120]: pgmap v299: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 310 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:20:33.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:33 smithi087 ceph-mon[27372]: osdmap e232: 8 total, 8 up, 8 in 2024-02-13T14:20:33.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:33 smithi087 ceph-mon[39120]: osdmap e232: 8 total, 8 up, 8 in 2024-02-13T14:20:34.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:33 smithi148 ceph-mon[31198]: osdmap e232: 8 total, 8 up, 8 in 2024-02-13T14:20:35.118 INFO:journalctl@ceph.iscsi.iscsi.a.smithi148.stdout:Feb 13 14:20:34 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-iscsi-iscsi-a[59644]: debug there is no tcmu-runner data available 2024-02-13T14:20:35.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:34 smithi148 ceph-mon[31198]: osdmap e233: 8 total, 8 up, 8 in 2024-02-13T14:20:35.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:34 smithi148 ceph-mon[31198]: pgmap v302: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 310 MiB used, 715 GiB / 715 GiB avail 2024-02-13T14:20:35.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:34 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:20:35.241 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:34 smithi087 ceph-mon[27372]: osdmap e233: 8 total, 8 up, 8 in 2024-02-13T14:20:35.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:34 smithi087 ceph-mon[27372]: pgmap v302: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 310 MiB used, 715 GiB / 715 GiB avail 2024-02-13T14:20:35.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:34 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:20:35.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:34 smithi087 ceph-mon[39120]: osdmap e233: 8 total, 8 up, 8 in 2024-02-13T14:20:35.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:34 smithi087 ceph-mon[39120]: pgmap v302: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 310 MiB used, 715 GiB / 715 GiB avail 2024-02-13T14:20:35.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:34 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:20:36.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:35 smithi148 ceph-mon[31198]: osdmap e234: 8 total, 8 up, 8 in 2024-02-13T14:20:36.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:35 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/2319336254' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:20:36.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:35 smithi148 ceph-mon[31198]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:20:36.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:35 smithi148 ceph-mon[31198]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:20:36.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:35 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:20:36.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:35 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:20:36.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:35 smithi148 ceph-mon[31198]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:20:36.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:35 smithi148 ceph-mon[31198]: osdmap e235: 8 total, 8 up, 8 in 2024-02-13T14:20:36.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:35 smithi087 ceph-mon[27372]: osdmap e234: 8 total, 8 up, 8 in 2024-02-13T14:20:36.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:35 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/2319336254' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:20:36.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:35 smithi087 ceph-mon[27372]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:20:36.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:35 smithi087 ceph-mon[27372]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:20:36.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:35 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:20:36.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:35 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:20:36.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:35 smithi087 ceph-mon[27372]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:20:36.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:35 smithi087 ceph-mon[27372]: osdmap e235: 8 total, 8 up, 8 in 2024-02-13T14:20:36.243 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:35 smithi087 ceph-mon[39120]: osdmap e234: 8 total, 8 up, 8 in 2024-02-13T14:20:36.243 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:35 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/2319336254' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:20:36.243 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:35 smithi087 ceph-mon[39120]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:20:36.243 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:35 smithi087 ceph-mon[39120]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:20:36.243 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:35 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:20:36.243 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:35 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:20:36.243 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:35 smithi087 ceph-mon[39120]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:20:36.243 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:35 smithi087 ceph-mon[39120]: osdmap e235: 8 total, 8 up, 8 in 2024-02-13T14:20:36.823 INFO:tasks.workunit.client.0.smithi087.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_clear_omap PASSED [ 58%] 2024-02-13T14:20:37.118 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:36 smithi148 ceph-mon[31198]: pgmap v304: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 310 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:20:37.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:36 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:20:37.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:36 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:20:37.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:36 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:20:37.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:36 smithi148 ceph-mon[31198]: osdmap e236: 8 total, 8 up, 8 in 2024-02-13T14:20:37.221 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:36 smithi087 ceph-mon[39120]: pgmap v304: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 310 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:20:37.221 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:36 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:20:37.221 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:36 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:20:37.221 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:36 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:20:37.221 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:36 smithi087 ceph-mon[39120]: osdmap e236: 8 total, 8 up, 8 in 2024-02-13T14:20:37.222 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:36 smithi087 ceph-mon[27372]: pgmap v304: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 310 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:20:37.222 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:36 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:20:37.222 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:36 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:20:37.222 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:36 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:20:37.222 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:36 smithi087 ceph-mon[27372]: osdmap e236: 8 total, 8 up, 8 in 2024-02-13T14:20:37.492 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:20:37 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: ::ffff:172.21.15.148 - - [13/Feb/2024:14:20:37] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-13T14:20:38.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:37 smithi087 ceph-mon[27372]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T14:20:38.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:37 smithi087 ceph-mon[27372]: osdmap e237: 8 total, 8 up, 8 in 2024-02-13T14:20:38.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:37 smithi087 ceph-mon[39120]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T14:20:38.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:37 smithi087 ceph-mon[39120]: osdmap e237: 8 total, 8 up, 8 in 2024-02-13T14:20:38.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:37 smithi148 ceph-mon[31198]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T14:20:38.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:37 smithi148 ceph-mon[31198]: osdmap e237: 8 total, 8 up, 8 in 2024-02-13T14:20:39.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:38 smithi087 ceph-mon[27372]: pgmap v307: 164 pgs: 164 active+clean; 582 KiB data, 310 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:20:39.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:38 smithi087 ceph-mon[27372]: osdmap e238: 8 total, 8 up, 8 in 2024-02-13T14:20:39.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:38 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/3614915000' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:20:39.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:38 smithi087 ceph-mon[27372]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:20:39.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:38 smithi087 ceph-mon[39120]: pgmap v307: 164 pgs: 164 active+clean; 582 KiB data, 310 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:20:39.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:38 smithi087 ceph-mon[39120]: osdmap e238: 8 total, 8 up, 8 in 2024-02-13T14:20:39.243 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:38 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/3614915000' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:20:39.243 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:38 smithi087 ceph-mon[39120]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:20:39.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:38 smithi148 ceph-mon[31198]: pgmap v307: 164 pgs: 164 active+clean; 582 KiB data, 310 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:20:39.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:38 smithi148 ceph-mon[31198]: osdmap e238: 8 total, 8 up, 8 in 2024-02-13T14:20:39.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:38 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/3614915000' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:20:39.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:38 smithi148 ceph-mon[31198]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:20:40.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:39 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-13T14:20:40.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:39 smithi087 ceph-mon[27372]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:20:40.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:39 smithi087 ceph-mon[27372]: osdmap e239: 8 total, 8 up, 8 in 2024-02-13T14:20:40.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:39 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-13T14:20:40.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:39 smithi087 ceph-mon[39120]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:20:40.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:39 smithi087 ceph-mon[39120]: osdmap e239: 8 total, 8 up, 8 in 2024-02-13T14:20:40.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:39 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-13T14:20:40.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:39 smithi148 ceph-mon[31198]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:20:40.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:39 smithi148 ceph-mon[31198]: osdmap e239: 8 total, 8 up, 8 in 2024-02-13T14:20:40.848 INFO:tasks.workunit.client.0.smithi087.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_remove_omap_range2 PASSED [ 60%] 2024-02-13T14:20:41.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:41 smithi148 ceph-mon[31198]: pgmap v310: 196 pgs: 196 active+clean; 582 KiB data, 319 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-13T14:20:41.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:41 smithi148 ceph-mon[31198]: osdmap e240: 8 total, 8 up, 8 in 2024-02-13T14:20:41.491 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:41 smithi087 ceph-mon[27372]: pgmap v310: 196 pgs: 196 active+clean; 582 KiB data, 319 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-13T14:20:41.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:41 smithi087 ceph-mon[27372]: osdmap e240: 8 total, 8 up, 8 in 2024-02-13T14:20:41.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:41 smithi087 ceph-mon[39120]: pgmap v310: 196 pgs: 196 active+clean; 582 KiB data, 319 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-13T14:20:41.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:41 smithi087 ceph-mon[39120]: osdmap e240: 8 total, 8 up, 8 in 2024-02-13T14:20:43.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:42 smithi148 ceph-mon[31198]: pgmap v313: 164 pgs: 164 active+clean; 582 KiB data, 319 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:20:43.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:42 smithi148 ceph-mon[31198]: osdmap e241: 8 total, 8 up, 8 in 2024-02-13T14:20:43.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:42 smithi087 ceph-mon[27372]: pgmap v313: 164 pgs: 164 active+clean; 582 KiB data, 319 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:20:43.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:42 smithi087 ceph-mon[27372]: osdmap e241: 8 total, 8 up, 8 in 2024-02-13T14:20:43.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:42 smithi087 ceph-mon[39120]: pgmap v313: 164 pgs: 164 active+clean; 582 KiB data, 319 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:20:43.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:42 smithi087 ceph-mon[39120]: osdmap e241: 8 total, 8 up, 8 in 2024-02-13T14:20:44.118 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:43 smithi148 ceph-mon[31198]: osdmap e242: 8 total, 8 up, 8 in 2024-02-13T14:20:44.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:43 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/1163639234' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:20:44.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:43 smithi148 ceph-mon[31198]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:20:44.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:43 smithi087 ceph-mon[27372]: osdmap e242: 8 total, 8 up, 8 in 2024-02-13T14:20:44.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:43 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/1163639234' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:20:44.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:43 smithi087 ceph-mon[27372]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:20:44.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:43 smithi087 ceph-mon[39120]: osdmap e242: 8 total, 8 up, 8 in 2024-02-13T14:20:44.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:43 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/1163639234' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:20:44.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:43 smithi087 ceph-mon[39120]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:20:44.869 INFO:tasks.workunit.client.0.smithi087.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_omap_cmp PASSED [ 61%] 2024-02-13T14:20:45.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:44 smithi087 ceph-mon[27372]: pgmap v316: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 319 MiB used, 715 GiB / 715 GiB avail 2024-02-13T14:20:45.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:44 smithi087 ceph-mon[27372]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T14:20:45.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:44 smithi087 ceph-mon[27372]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:20:45.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:44 smithi087 ceph-mon[27372]: osdmap e243: 8 total, 8 up, 8 in 2024-02-13T14:20:45.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:44 smithi087 ceph-mon[39120]: pgmap v316: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 319 MiB used, 715 GiB / 715 GiB avail 2024-02-13T14:20:45.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:44 smithi087 ceph-mon[39120]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T14:20:45.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:44 smithi087 ceph-mon[39120]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:20:45.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:44 smithi087 ceph-mon[39120]: osdmap e243: 8 total, 8 up, 8 in 2024-02-13T14:20:45.368 INFO:journalctl@ceph.iscsi.iscsi.a.smithi148.stdout:Feb 13 14:20:44 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-iscsi-iscsi-a[59644]: debug there is no tcmu-runner data available 2024-02-13T14:20:45.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:44 smithi148 ceph-mon[31198]: pgmap v316: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 319 MiB used, 715 GiB / 715 GiB avail 2024-02-13T14:20:45.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:44 smithi148 ceph-mon[31198]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T14:20:45.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:44 smithi148 ceph-mon[31198]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:20:45.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:44 smithi148 ceph-mon[31198]: osdmap e243: 8 total, 8 up, 8 in 2024-02-13T14:20:46.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:45 smithi087 ceph-mon[27372]: osdmap e244: 8 total, 8 up, 8 in 2024-02-13T14:20:46.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:45 smithi087 ceph-mon[27372]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:20:46.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:45 smithi087 ceph-mon[39120]: osdmap e244: 8 total, 8 up, 8 in 2024-02-13T14:20:46.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:45 smithi087 ceph-mon[39120]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:20:46.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:45 smithi148 ceph-mon[31198]: osdmap e244: 8 total, 8 up, 8 in 2024-02-13T14:20:46.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:45 smithi148 ceph-mon[31198]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:20:47.221 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:46 smithi087 ceph-mon[39120]: pgmap v319: 164 pgs: 164 active+clean; 582 KiB data, 324 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:20:47.221 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:46 smithi087 ceph-mon[39120]: osdmap e245: 8 total, 8 up, 8 in 2024-02-13T14:20:47.221 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:46 smithi087 ceph-mon[27372]: pgmap v319: 164 pgs: 164 active+clean; 582 KiB data, 324 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:20:47.221 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:46 smithi087 ceph-mon[27372]: osdmap e245: 8 total, 8 up, 8 in 2024-02-13T14:20:47.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:46 smithi148 ceph-mon[31198]: pgmap v319: 164 pgs: 164 active+clean; 582 KiB data, 324 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:20:47.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:46 smithi148 ceph-mon[31198]: osdmap e245: 8 total, 8 up, 8 in 2024-02-13T14:20:47.491 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:20:47 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: ::ffff:172.21.15.148 - - [13/Feb/2024:14:20:47] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-13T14:20:48.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:47 smithi087 ceph-mon[39120]: osdmap e246: 8 total, 8 up, 8 in 2024-02-13T14:20:48.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:47 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/803641059' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:20:48.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:47 smithi087 ceph-mon[27372]: osdmap e246: 8 total, 8 up, 8 in 2024-02-13T14:20:48.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:47 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/803641059' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:20:48.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:47 smithi148 ceph-mon[31198]: osdmap e246: 8 total, 8 up, 8 in 2024-02-13T14:20:48.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:47 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/803641059' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:20:48.900 INFO:tasks.workunit.client.0.smithi087.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_cmpext_op PASSED [ 62%] 2024-02-13T14:20:49.241 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:48 smithi087 ceph-mon[39120]: pgmap v322: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 324 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:20:49.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:48 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/803641059' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:20:49.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:48 smithi087 ceph-mon[39120]: osdmap e247: 8 total, 8 up, 8 in 2024-02-13T14:20:49.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:48 smithi087 ceph-mon[27372]: pgmap v322: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 324 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:20:49.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:48 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/803641059' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:20:49.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:48 smithi087 ceph-mon[27372]: osdmap e247: 8 total, 8 up, 8 in 2024-02-13T14:20:49.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:48 smithi148 ceph-mon[31198]: pgmap v322: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 324 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:20:49.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:48 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/803641059' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:20:49.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:48 smithi148 ceph-mon[31198]: osdmap e247: 8 total, 8 up, 8 in 2024-02-13T14:20:50.241 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:49 smithi087 ceph-mon[27372]: osdmap e248: 8 total, 8 up, 8 in 2024-02-13T14:20:50.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:49 smithi087 ceph-mon[39120]: osdmap e248: 8 total, 8 up, 8 in 2024-02-13T14:20:50.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:49 smithi148 ceph-mon[31198]: osdmap e248: 8 total, 8 up, 8 in 2024-02-13T14:20:51.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:50 smithi087 ceph-mon[27372]: pgmap v325: 164 pgs: 164 active+clean; 582 KiB data, 324 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:20:51.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:50 smithi087 ceph-mon[27372]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T14:20:51.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:50 smithi087 ceph-mon[27372]: osdmap e249: 8 total, 8 up, 8 in 2024-02-13T14:20:51.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:50 smithi087 ceph-mon[39120]: pgmap v325: 164 pgs: 164 active+clean; 582 KiB data, 324 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:20:51.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:50 smithi087 ceph-mon[39120]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T14:20:51.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:50 smithi087 ceph-mon[39120]: osdmap e249: 8 total, 8 up, 8 in 2024-02-13T14:20:51.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:50 smithi148 ceph-mon[31198]: pgmap v325: 164 pgs: 164 active+clean; 582 KiB data, 324 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:20:51.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:50 smithi148 ceph-mon[31198]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T14:20:51.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:50 smithi148 ceph-mon[31198]: osdmap e249: 8 total, 8 up, 8 in 2024-02-13T14:20:52.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:51 smithi087 ceph-mon[27372]: osdmap e250: 8 total, 8 up, 8 in 2024-02-13T14:20:52.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:51 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/3251889661' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:20:52.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:51 smithi087 ceph-mon[27372]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:20:52.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:51 smithi087 ceph-mon[39120]: osdmap e250: 8 total, 8 up, 8 in 2024-02-13T14:20:52.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:51 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/3251889661' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:20:52.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:51 smithi087 ceph-mon[39120]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:20:52.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:51 smithi148 ceph-mon[31198]: osdmap e250: 8 total, 8 up, 8 in 2024-02-13T14:20:52.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:51 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/3251889661' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:20:52.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:51 smithi148 ceph-mon[31198]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:20:52.926 INFO:tasks.workunit.client.0.smithi087.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_xattrs_op PASSED [ 63%] 2024-02-13T14:20:53.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:52 smithi087 ceph-mon[39120]: pgmap v328: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 324 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:20:53.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:52 smithi087 ceph-mon[39120]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:20:53.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:52 smithi087 ceph-mon[39120]: osdmap e251: 8 total, 8 up, 8 in 2024-02-13T14:20:53.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:52 smithi087 ceph-mon[27372]: pgmap v328: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 324 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:20:53.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:52 smithi087 ceph-mon[27372]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:20:53.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:52 smithi087 ceph-mon[27372]: osdmap e251: 8 total, 8 up, 8 in 2024-02-13T14:20:53.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:52 smithi148 ceph-mon[31198]: pgmap v328: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 324 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:20:53.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:52 smithi148 ceph-mon[31198]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:20:53.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:52 smithi148 ceph-mon[31198]: osdmap e251: 8 total, 8 up, 8 in 2024-02-13T14:20:54.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:53 smithi087 ceph-mon[27372]: osdmap e252: 8 total, 8 up, 8 in 2024-02-13T14:20:54.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:53 smithi087 ceph-mon[39120]: osdmap e252: 8 total, 8 up, 8 in 2024-02-13T14:20:54.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:53 smithi148 ceph-mon[31198]: osdmap e252: 8 total, 8 up, 8 in 2024-02-13T14:20:55.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:54 smithi087 ceph-mon[27372]: pgmap v331: 164 pgs: 164 active+clean; 582 KiB data, 324 MiB used, 715 GiB / 715 GiB avail 2024-02-13T14:20:55.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:54 smithi087 ceph-mon[27372]: osdmap e253: 8 total, 8 up, 8 in 2024-02-13T14:20:55.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:54 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-13T14:20:55.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:54 smithi087 ceph-mon[39120]: pgmap v331: 164 pgs: 164 active+clean; 582 KiB data, 324 MiB used, 715 GiB / 715 GiB avail 2024-02-13T14:20:55.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:54 smithi087 ceph-mon[39120]: osdmap e253: 8 total, 8 up, 8 in 2024-02-13T14:20:55.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:54 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-13T14:20:55.369 INFO:journalctl@ceph.iscsi.iscsi.a.smithi148.stdout:Feb 13 14:20:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-iscsi-iscsi-a[59644]: debug there is no tcmu-runner data available 2024-02-13T14:20:55.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:54 smithi148 ceph-mon[31198]: pgmap v331: 164 pgs: 164 active+clean; 582 KiB data, 324 MiB used, 715 GiB / 715 GiB avail 2024-02-13T14:20:55.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:54 smithi148 ceph-mon[31198]: osdmap e253: 8 total, 8 up, 8 in 2024-02-13T14:20:55.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:54 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-13T14:20:56.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:55 smithi087 ceph-mon[39120]: osdmap e254: 8 total, 8 up, 8 in 2024-02-13T14:20:56.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:55 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/291759180' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:20:56.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:55 smithi087 ceph-mon[39120]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:20:56.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:55 smithi087 ceph-mon[27372]: osdmap e254: 8 total, 8 up, 8 in 2024-02-13T14:20:56.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:55 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/291759180' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:20:56.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:55 smithi087 ceph-mon[27372]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:20:56.270 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:55 smithi148 ceph-mon[31198]: osdmap e254: 8 total, 8 up, 8 in 2024-02-13T14:20:56.270 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:55 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/291759180' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:20:56.270 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:55 smithi148 ceph-mon[31198]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:20:56.960 INFO:tasks.workunit.client.0.smithi087.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_locator PASSED [ 64%] 2024-02-13T14:20:57.221 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:56 smithi087 ceph-mon[39120]: pgmap v334: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 329 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:20:57.221 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:56 smithi087 ceph-mon[39120]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T14:20:57.221 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:56 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/291759180' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:20:57.221 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:56 smithi087 ceph-mon[39120]: osdmap e255: 8 total, 8 up, 8 in 2024-02-13T14:20:57.221 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:56 smithi087 ceph-mon[27372]: pgmap v334: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 329 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:20:57.221 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:56 smithi087 ceph-mon[27372]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T14:20:57.221 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:56 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/291759180' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:20:57.221 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:56 smithi087 ceph-mon[27372]: osdmap e255: 8 total, 8 up, 8 in 2024-02-13T14:20:57.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:56 smithi148 ceph-mon[31198]: pgmap v334: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 329 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:20:57.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:56 smithi148 ceph-mon[31198]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T14:20:57.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:56 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/291759180' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:20:57.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:56 smithi148 ceph-mon[31198]: osdmap e255: 8 total, 8 up, 8 in 2024-02-13T14:20:57.491 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:20:57 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: ::ffff:172.21.15.148 - - [13/Feb/2024:14:20:57] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-13T14:20:58.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:57 smithi087 ceph-mon[27372]: osdmap e256: 8 total, 8 up, 8 in 2024-02-13T14:20:58.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:57 smithi087 ceph-mon[39120]: osdmap e256: 8 total, 8 up, 8 in 2024-02-13T14:20:58.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:57 smithi148 ceph-mon[31198]: osdmap e256: 8 total, 8 up, 8 in 2024-02-13T14:20:59.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:58 smithi087 ceph-mon[27372]: pgmap v337: 164 pgs: 164 active+clean; 582 KiB data, 329 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:20:59.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:58 smithi087 ceph-mon[27372]: osdmap e257: 8 total, 8 up, 8 in 2024-02-13T14:20:59.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:58 smithi087 ceph-mon[39120]: pgmap v337: 164 pgs: 164 active+clean; 582 KiB data, 329 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:20:59.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:58 smithi087 ceph-mon[39120]: osdmap e257: 8 total, 8 up, 8 in 2024-02-13T14:20:59.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:58 smithi148 ceph-mon[31198]: pgmap v337: 164 pgs: 164 active+clean; 582 KiB data, 329 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:20:59.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:58 smithi148 ceph-mon[31198]: osdmap e257: 8 total, 8 up, 8 in 2024-02-13T14:21:00.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:59 smithi087 ceph-mon[27372]: osdmap e258: 8 total, 8 up, 8 in 2024-02-13T14:21:00.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:59 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/3887120271' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:21:00.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:20:59 smithi087 ceph-mon[27372]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:21:00.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:59 smithi087 ceph-mon[39120]: osdmap e258: 8 total, 8 up, 8 in 2024-02-13T14:21:00.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:59 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/3887120271' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:21:00.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:20:59 smithi087 ceph-mon[39120]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:21:00.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:59 smithi148 ceph-mon[31198]: osdmap e258: 8 total, 8 up, 8 in 2024-02-13T14:21:00.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:59 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/3887120271' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:21:00.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:20:59 smithi148 ceph-mon[31198]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:21:00.988 INFO:tasks.workunit.client.0.smithi087.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_operate_aio_write_op PASSED [ 65%] 2024-02-13T14:21:01.241 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:00 smithi087 ceph-mon[27372]: pgmap v340: 196 pgs: 196 active+clean; 582 KiB data, 330 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-13T14:21:01.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:00 smithi087 ceph-mon[27372]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:21:01.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:00 smithi087 ceph-mon[27372]: osdmap e259: 8 total, 8 up, 8 in 2024-02-13T14:21:01.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:00 smithi087 ceph-mon[39120]: pgmap v340: 196 pgs: 196 active+clean; 582 KiB data, 330 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-13T14:21:01.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:00 smithi087 ceph-mon[39120]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:21:01.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:00 smithi087 ceph-mon[39120]: osdmap e259: 8 total, 8 up, 8 in 2024-02-13T14:21:01.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:00 smithi148 ceph-mon[31198]: pgmap v340: 196 pgs: 196 active+clean; 582 KiB data, 330 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-13T14:21:01.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:00 smithi148 ceph-mon[31198]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:21:01.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:00 smithi148 ceph-mon[31198]: osdmap e259: 8 total, 8 up, 8 in 2024-02-13T14:21:02.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:01 smithi148 ceph-mon[31198]: osdmap e260: 8 total, 8 up, 8 in 2024-02-13T14:21:02.491 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:01 smithi087 ceph-mon[27372]: osdmap e260: 8 total, 8 up, 8 in 2024-02-13T14:21:02.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:01 smithi087 ceph-mon[39120]: osdmap e260: 8 total, 8 up, 8 in 2024-02-13T14:21:03.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:03 smithi148 ceph-mon[31198]: pgmap v343: 164 pgs: 164 active+clean; 582 KiB data, 330 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:21:03.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:03 smithi148 ceph-mon[31198]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T14:21:03.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:03 smithi148 ceph-mon[31198]: osdmap e261: 8 total, 8 up, 8 in 2024-02-13T14:21:03.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:03 smithi087 ceph-mon[39120]: pgmap v343: 164 pgs: 164 active+clean; 582 KiB data, 330 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:21:03.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:03 smithi087 ceph-mon[39120]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T14:21:03.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:03 smithi087 ceph-mon[39120]: osdmap e261: 8 total, 8 up, 8 in 2024-02-13T14:21:03.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:03 smithi087 ceph-mon[27372]: pgmap v343: 164 pgs: 164 active+clean; 582 KiB data, 330 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:21:03.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:03 smithi087 ceph-mon[27372]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T14:21:03.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:03 smithi087 ceph-mon[27372]: osdmap e261: 8 total, 8 up, 8 in 2024-02-13T14:21:04.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:04 smithi148 ceph-mon[31198]: osdmap e262: 8 total, 8 up, 8 in 2024-02-13T14:21:04.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:04 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/2234904480' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:21:04.491 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:04 smithi087 ceph-mon[27372]: osdmap e262: 8 total, 8 up, 8 in 2024-02-13T14:21:04.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:04 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/2234904480' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:21:04.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:04 smithi087 ceph-mon[39120]: osdmap e262: 8 total, 8 up, 8 in 2024-02-13T14:21:04.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:04 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/2234904480' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:21:05.024 INFO:tasks.workunit.client.0.smithi087.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_aio_write PASSED [ 66%] 2024-02-13T14:21:05.369 INFO:journalctl@ceph.iscsi.iscsi.a.smithi148.stdout:Feb 13 14:21:05 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-iscsi-iscsi-a[59644]: debug there is no tcmu-runner data available 2024-02-13T14:21:05.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:05 smithi148 ceph-mon[31198]: pgmap v346: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 330 MiB used, 715 GiB / 715 GiB avail 2024-02-13T14:21:05.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:05 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/2234904480' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:21:05.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:05 smithi148 ceph-mon[31198]: osdmap e263: 8 total, 8 up, 8 in 2024-02-13T14:21:05.491 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:05 smithi087 ceph-mon[39120]: pgmap v346: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 330 MiB used, 715 GiB / 715 GiB avail 2024-02-13T14:21:05.491 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:05 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/2234904480' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:21:05.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:05 smithi087 ceph-mon[39120]: osdmap e263: 8 total, 8 up, 8 in 2024-02-13T14:21:05.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:05 smithi087 ceph-mon[27372]: pgmap v346: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 330 MiB used, 715 GiB / 715 GiB avail 2024-02-13T14:21:05.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:05 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/2234904480' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:21:05.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:05 smithi087 ceph-mon[27372]: osdmap e263: 8 total, 8 up, 8 in 2024-02-13T14:21:06.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:06 smithi148 ceph-mon[31198]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:21:06.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:06 smithi148 ceph-mon[31198]: osdmap e264: 8 total, 8 up, 8 in 2024-02-13T14:21:06.491 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:06 smithi087 ceph-mon[27372]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:21:06.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:06 smithi087 ceph-mon[27372]: osdmap e264: 8 total, 8 up, 8 in 2024-02-13T14:21:06.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:06 smithi087 ceph-mon[39120]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:21:06.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:06 smithi087 ceph-mon[39120]: osdmap e264: 8 total, 8 up, 8 in 2024-02-13T14:21:07.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:07 smithi148 ceph-mon[31198]: pgmap v349: 164 pgs: 164 active+clean; 582 KiB data, 330 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:21:07.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:07 smithi148 ceph-mon[31198]: osdmap e265: 8 total, 8 up, 8 in 2024-02-13T14:21:07.491 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:07 smithi087 ceph-mon[27372]: pgmap v349: 164 pgs: 164 active+clean; 582 KiB data, 330 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:21:07.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:07 smithi087 ceph-mon[27372]: osdmap e265: 8 total, 8 up, 8 in 2024-02-13T14:21:07.492 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:21:07 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: ::ffff:172.21.15.148 - - [13/Feb/2024:14:21:07] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-13T14:21:07.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:07 smithi087 ceph-mon[39120]: pgmap v349: 164 pgs: 164 active+clean; 582 KiB data, 330 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:21:07.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:07 smithi087 ceph-mon[39120]: osdmap e265: 8 total, 8 up, 8 in 2024-02-13T14:21:08.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:08 smithi148 ceph-mon[31198]: osdmap e266: 8 total, 8 up, 8 in 2024-02-13T14:21:08.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:08 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/2998693053' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:21:08.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:08 smithi148 ceph-mon[31198]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:21:08.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:08 smithi148 ceph-mon[31198]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:21:08.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:08 smithi148 ceph-mon[31198]: osdmap e267: 8 total, 8 up, 8 in 2024-02-13T14:21:08.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:08 smithi087 ceph-mon[39120]: osdmap e266: 8 total, 8 up, 8 in 2024-02-13T14:21:08.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:08 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/2998693053' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:21:08.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:08 smithi087 ceph-mon[39120]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:21:08.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:08 smithi087 ceph-mon[39120]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:21:08.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:08 smithi087 ceph-mon[39120]: osdmap e267: 8 total, 8 up, 8 in 2024-02-13T14:21:08.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:08 smithi087 ceph-mon[27372]: osdmap e266: 8 total, 8 up, 8 in 2024-02-13T14:21:08.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:08 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/2998693053' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:21:08.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:08 smithi087 ceph-mon[27372]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:21:08.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:08 smithi087 ceph-mon[27372]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:21:08.493 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:08 smithi087 ceph-mon[27372]: osdmap e267: 8 total, 8 up, 8 in 2024-02-13T14:21:09.047 INFO:tasks.workunit.client.0.smithi087.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_aio_cmpext PASSED [ 67%] 2024-02-13T14:21:09.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:09 smithi148 ceph-mon[31198]: pgmap v352: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 330 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:21:09.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:09 smithi148 ceph-mon[31198]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T14:21:09.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:09 smithi148 ceph-mon[31198]: osdmap e268: 8 total, 8 up, 8 in 2024-02-13T14:21:09.491 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:09 smithi087 ceph-mon[39120]: pgmap v352: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 330 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:21:09.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:09 smithi087 ceph-mon[39120]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T14:21:09.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:09 smithi087 ceph-mon[39120]: osdmap e268: 8 total, 8 up, 8 in 2024-02-13T14:21:09.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:09 smithi087 ceph-mon[27372]: pgmap v352: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 330 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:21:09.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:09 smithi087 ceph-mon[27372]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T14:21:09.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:09 smithi087 ceph-mon[27372]: osdmap e268: 8 total, 8 up, 8 in 2024-02-13T14:21:10.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:10 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-13T14:21:10.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:10 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-13T14:21:10.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:10 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-13T14:21:11.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:11 smithi148 ceph-mon[31198]: pgmap v355: 164 pgs: 164 active+clean; 582 KiB data, 331 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:21:11.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:11 smithi148 ceph-mon[31198]: osdmap e269: 8 total, 8 up, 8 in 2024-02-13T14:21:11.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:11 smithi087 ceph-mon[27372]: pgmap v355: 164 pgs: 164 active+clean; 582 KiB data, 331 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:21:11.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:11 smithi087 ceph-mon[27372]: osdmap e269: 8 total, 8 up, 8 in 2024-02-13T14:21:11.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:11 smithi087 ceph-mon[39120]: pgmap v355: 164 pgs: 164 active+clean; 582 KiB data, 331 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:21:11.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:11 smithi087 ceph-mon[39120]: osdmap e269: 8 total, 8 up, 8 in 2024-02-13T14:21:12.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:12 smithi148 ceph-mon[31198]: osdmap e270: 8 total, 8 up, 8 in 2024-02-13T14:21:12.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:12 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/2613206725' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:21:12.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:12 smithi148 ceph-mon[31198]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:21:12.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:12 smithi087 ceph-mon[27372]: osdmap e270: 8 total, 8 up, 8 in 2024-02-13T14:21:12.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:12 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/2613206725' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:21:12.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:12 smithi087 ceph-mon[27372]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:21:12.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:12 smithi087 ceph-mon[39120]: osdmap e270: 8 total, 8 up, 8 in 2024-02-13T14:21:12.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:12 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/2613206725' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:21:12.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:12 smithi087 ceph-mon[39120]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:21:13.086 INFO:tasks.workunit.client.0.smithi087.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_aio_rmxattr PASSED [ 68%] 2024-02-13T14:21:13.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:13 smithi148 ceph-mon[31198]: pgmap v358: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 331 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:21:13.370 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:13 smithi148 ceph-mon[31198]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:21:13.370 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:13 smithi148 ceph-mon[31198]: osdmap e271: 8 total, 8 up, 8 in 2024-02-13T14:21:13.491 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:13 smithi087 ceph-mon[39120]: pgmap v358: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 331 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:21:13.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:13 smithi087 ceph-mon[39120]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:21:13.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:13 smithi087 ceph-mon[39120]: osdmap e271: 8 total, 8 up, 8 in 2024-02-13T14:21:13.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:13 smithi087 ceph-mon[27372]: pgmap v358: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 331 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:21:13.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:13 smithi087 ceph-mon[27372]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:21:13.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:13 smithi087 ceph-mon[27372]: osdmap e271: 8 total, 8 up, 8 in 2024-02-13T14:21:14.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:14 smithi148 ceph-mon[31198]: osdmap e272: 8 total, 8 up, 8 in 2024-02-13T14:21:14.491 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:14 smithi087 ceph-mon[39120]: osdmap e272: 8 total, 8 up, 8 in 2024-02-13T14:21:14.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:14 smithi087 ceph-mon[27372]: osdmap e272: 8 total, 8 up, 8 in 2024-02-13T14:21:15.369 INFO:journalctl@ceph.iscsi.iscsi.a.smithi148.stdout:Feb 13 14:21:15 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-iscsi-iscsi-a[59644]: debug there is no tcmu-runner data available 2024-02-13T14:21:15.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:15 smithi148 ceph-mon[31198]: pgmap v361: 164 pgs: 164 active+clean; 582 KiB data, 331 MiB used, 715 GiB / 715 GiB avail 2024-02-13T14:21:15.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:15 smithi148 ceph-mon[31198]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T14:21:15.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:15 smithi148 ceph-mon[31198]: osdmap e273: 8 total, 8 up, 8 in 2024-02-13T14:21:15.491 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:15 smithi087 ceph-mon[27372]: pgmap v361: 164 pgs: 164 active+clean; 582 KiB data, 331 MiB used, 715 GiB / 715 GiB avail 2024-02-13T14:21:15.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:15 smithi087 ceph-mon[27372]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T14:21:15.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:15 smithi087 ceph-mon[27372]: osdmap e273: 8 total, 8 up, 8 in 2024-02-13T14:21:15.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:15 smithi087 ceph-mon[39120]: pgmap v361: 164 pgs: 164 active+clean; 582 KiB data, 331 MiB used, 715 GiB / 715 GiB avail 2024-02-13T14:21:15.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:15 smithi087 ceph-mon[39120]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T14:21:15.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:15 smithi087 ceph-mon[39120]: osdmap e273: 8 total, 8 up, 8 in 2024-02-13T14:21:16.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:16 smithi148 ceph-mon[31198]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:21:16.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:16 smithi148 ceph-mon[31198]: osdmap e274: 8 total, 8 up, 8 in 2024-02-13T14:21:16.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:16 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/317108493' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:21:16.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:16 smithi148 ceph-mon[31198]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:21:16.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:16 smithi087 ceph-mon[39120]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:21:16.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:16 smithi087 ceph-mon[39120]: osdmap e274: 8 total, 8 up, 8 in 2024-02-13T14:21:16.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:16 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/317108493' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:21:16.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:16 smithi087 ceph-mon[39120]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:21:16.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:16 smithi087 ceph-mon[27372]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:21:16.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:16 smithi087 ceph-mon[27372]: osdmap e274: 8 total, 8 up, 8 in 2024-02-13T14:21:16.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:16 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/317108493' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:21:16.493 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:16 smithi087 ceph-mon[27372]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:21:17.124 INFO:tasks.workunit.client.0.smithi087.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_aio_write_no_comp_ref PASSED [ 70%] 2024-02-13T14:21:17.492 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:21:17 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: ::ffff:172.21.15.148 - - [13/Feb/2024:14:21:17] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-13T14:21:17.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:17 smithi087 ceph-mon[27372]: pgmap v364: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 332 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:21:17.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:17 smithi087 ceph-mon[27372]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:21:17.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:17 smithi087 ceph-mon[27372]: osdmap e275: 8 total, 8 up, 8 in 2024-02-13T14:21:17.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:17 smithi087 ceph-mon[39120]: pgmap v364: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 332 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:21:17.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:17 smithi087 ceph-mon[39120]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:21:17.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:17 smithi087 ceph-mon[39120]: osdmap e275: 8 total, 8 up, 8 in 2024-02-13T14:21:17.618 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:17 smithi148 ceph-mon[31198]: pgmap v364: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 332 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:21:17.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:17 smithi148 ceph-mon[31198]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:21:17.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:17 smithi148 ceph-mon[31198]: osdmap e275: 8 total, 8 up, 8 in 2024-02-13T14:21:18.491 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:18 smithi087 ceph-mon[27372]: osdmap e276: 8 total, 8 up, 8 in 2024-02-13T14:21:18.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:18 smithi087 ceph-mon[39120]: osdmap e276: 8 total, 8 up, 8 in 2024-02-13T14:21:18.618 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:18 smithi148 ceph-mon[31198]: osdmap e276: 8 total, 8 up, 8 in 2024-02-13T14:21:19.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:19 smithi087 ceph-mon[27372]: pgmap v367: 164 pgs: 164 active+clean; 582 KiB data, 332 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:21:19.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:19 smithi087 ceph-mon[27372]: osdmap e277: 8 total, 8 up, 8 in 2024-02-13T14:21:19.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:19 smithi087 ceph-mon[39120]: pgmap v367: 164 pgs: 164 active+clean; 582 KiB data, 332 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:21:19.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:19 smithi087 ceph-mon[39120]: osdmap e277: 8 total, 8 up, 8 in 2024-02-13T14:21:19.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:19 smithi148 ceph-mon[31198]: pgmap v367: 164 pgs: 164 active+clean; 582 KiB data, 332 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:21:19.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:19 smithi148 ceph-mon[31198]: osdmap e277: 8 total, 8 up, 8 in 2024-02-13T14:21:20.491 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:20 smithi087 ceph-mon[39120]: osdmap e278: 8 total, 8 up, 8 in 2024-02-13T14:21:20.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:20 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/625458527' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:21:20.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:20 smithi087 ceph-mon[27372]: osdmap e278: 8 total, 8 up, 8 in 2024-02-13T14:21:20.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:20 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/625458527' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:21:20.618 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:20 smithi148 ceph-mon[31198]: osdmap e278: 8 total, 8 up, 8 in 2024-02-13T14:21:20.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:20 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/625458527' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:21:21.155 INFO:tasks.workunit.client.0.smithi087.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_aio_append PASSED [ 71%] 2024-02-13T14:21:21.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:21 smithi087 ceph-mon[39120]: pgmap v370: 196 pgs: 196 active+clean; 582 KiB data, 332 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-13T14:21:21.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:21 smithi087 ceph-mon[39120]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T14:21:21.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:21 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/625458527' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:21:21.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:21 smithi087 ceph-mon[39120]: osdmap e279: 8 total, 8 up, 8 in 2024-02-13T14:21:21.493 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:21 smithi087 ceph-mon[27372]: pgmap v370: 196 pgs: 196 active+clean; 582 KiB data, 332 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-13T14:21:21.493 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:21 smithi087 ceph-mon[27372]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T14:21:21.493 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:21 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/625458527' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:21:21.493 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:21 smithi087 ceph-mon[27372]: osdmap e279: 8 total, 8 up, 8 in 2024-02-13T14:21:21.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:21 smithi148 ceph-mon[31198]: pgmap v370: 196 pgs: 196 active+clean; 582 KiB data, 332 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-13T14:21:21.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:21 smithi148 ceph-mon[31198]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T14:21:21.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:21 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/625458527' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:21:21.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:21 smithi148 ceph-mon[31198]: osdmap e279: 8 total, 8 up, 8 in 2024-02-13T14:21:22.491 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:22 smithi087 ceph-mon[27372]: osdmap e280: 8 total, 8 up, 8 in 2024-02-13T14:21:22.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:22 smithi087 ceph-mon[39120]: osdmap e280: 8 total, 8 up, 8 in 2024-02-13T14:21:22.618 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:22 smithi148 ceph-mon[31198]: osdmap e280: 8 total, 8 up, 8 in 2024-02-13T14:21:23.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:23 smithi087 ceph-mon[27372]: pgmap v373: 164 pgs: 164 active+clean; 582 KiB data, 332 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:21:23.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:23 smithi087 ceph-mon[27372]: osdmap e281: 8 total, 8 up, 8 in 2024-02-13T14:21:23.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:23 smithi087 ceph-mon[39120]: pgmap v373: 164 pgs: 164 active+clean; 582 KiB data, 332 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:21:23.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:23 smithi087 ceph-mon[39120]: osdmap e281: 8 total, 8 up, 8 in 2024-02-13T14:21:23.618 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:23 smithi148 ceph-mon[31198]: pgmap v373: 164 pgs: 164 active+clean; 582 KiB data, 332 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:21:23.618 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:23 smithi148 ceph-mon[31198]: osdmap e281: 8 total, 8 up, 8 in 2024-02-13T14:21:24.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:24 smithi087 ceph-mon[39120]: osdmap e282: 8 total, 8 up, 8 in 2024-02-13T14:21:24.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:24 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/4014716681' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:21:24.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:24 smithi087 ceph-mon[27372]: osdmap e282: 8 total, 8 up, 8 in 2024-02-13T14:21:24.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:24 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/4014716681' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:21:24.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:24 smithi148 ceph-mon[31198]: osdmap e282: 8 total, 8 up, 8 in 2024-02-13T14:21:24.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:24 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/4014716681' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:21:25.188 INFO:tasks.workunit.client.0.smithi087.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_aio_write_full PASSED [ 72%] 2024-02-13T14:21:25.368 INFO:journalctl@ceph.iscsi.iscsi.a.smithi148.stdout:Feb 13 14:21:25 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-iscsi-iscsi-a[59644]: debug there is no tcmu-runner data available 2024-02-13T14:21:25.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:25 smithi148 ceph-mon[31198]: pgmap v376: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 332 MiB used, 715 GiB / 715 GiB avail 2024-02-13T14:21:25.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:25 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/4014716681' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:21:25.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:25 smithi148 ceph-mon[31198]: osdmap e283: 8 total, 8 up, 8 in 2024-02-13T14:21:25.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:25 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-13T14:21:25.491 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:25 smithi087 ceph-mon[27372]: pgmap v376: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 332 MiB used, 715 GiB / 715 GiB avail 2024-02-13T14:21:25.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:25 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/4014716681' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:21:25.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:25 smithi087 ceph-mon[27372]: osdmap e283: 8 total, 8 up, 8 in 2024-02-13T14:21:25.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:25 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-13T14:21:25.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:25 smithi087 ceph-mon[39120]: pgmap v376: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 332 MiB used, 715 GiB / 715 GiB avail 2024-02-13T14:21:25.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:25 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/4014716681' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:21:25.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:25 smithi087 ceph-mon[39120]: osdmap e283: 8 total, 8 up, 8 in 2024-02-13T14:21:25.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:25 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-13T14:21:26.491 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:26 smithi087 ceph-mon[39120]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:21:26.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:26 smithi087 ceph-mon[39120]: osdmap e284: 8 total, 8 up, 8 in 2024-02-13T14:21:26.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:26 smithi087 ceph-mon[27372]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:21:26.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:26 smithi087 ceph-mon[27372]: osdmap e284: 8 total, 8 up, 8 in 2024-02-13T14:21:26.618 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:26 smithi148 ceph-mon[31198]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:21:26.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:26 smithi148 ceph-mon[31198]: osdmap e284: 8 total, 8 up, 8 in 2024-02-13T14:21:27.492 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:21:27 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: ::ffff:172.21.15.148 - - [13/Feb/2024:14:21:27] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-13T14:21:27.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:27 smithi087 ceph-mon[39120]: pgmap v379: 164 pgs: 164 active+clean; 582 KiB data, 337 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:21:27.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:27 smithi087 ceph-mon[39120]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T14:21:27.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:27 smithi087 ceph-mon[39120]: osdmap e285: 8 total, 8 up, 8 in 2024-02-13T14:21:27.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:27 smithi087 ceph-mon[27372]: pgmap v379: 164 pgs: 164 active+clean; 582 KiB data, 337 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:21:27.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:27 smithi087 ceph-mon[27372]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T14:21:27.493 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:27 smithi087 ceph-mon[27372]: osdmap e285: 8 total, 8 up, 8 in 2024-02-13T14:21:27.618 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:27 smithi148 ceph-mon[31198]: pgmap v379: 164 pgs: 164 active+clean; 582 KiB data, 337 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:21:27.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:27 smithi148 ceph-mon[31198]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T14:21:27.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:27 smithi148 ceph-mon[31198]: osdmap e285: 8 total, 8 up, 8 in 2024-02-13T14:21:28.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:28 smithi087 ceph-mon[39120]: osdmap e286: 8 total, 8 up, 8 in 2024-02-13T14:21:28.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:28 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/2538457161' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:21:28.493 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:28 smithi087 ceph-mon[27372]: osdmap e286: 8 total, 8 up, 8 in 2024-02-13T14:21:28.493 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:28 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/2538457161' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:21:28.618 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:28 smithi148 ceph-mon[31198]: osdmap e286: 8 total, 8 up, 8 in 2024-02-13T14:21:28.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:28 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/2538457161' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:21:29.220 INFO:tasks.workunit.client.0.smithi087.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_aio_writesame PASSED [ 73%] 2024-02-13T14:21:29.491 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:29 smithi087 ceph-mon[27372]: pgmap v382: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 337 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:21:29.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:29 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/2538457161' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:21:29.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:29 smithi087 ceph-mon[27372]: osdmap e287: 8 total, 8 up, 8 in 2024-02-13T14:21:29.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:29 smithi087 ceph-mon[39120]: pgmap v382: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 337 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:21:29.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:29 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/2538457161' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:21:29.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:29 smithi087 ceph-mon[39120]: osdmap e287: 8 total, 8 up, 8 in 2024-02-13T14:21:29.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:29 smithi148 ceph-mon[31198]: pgmap v382: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 337 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:21:29.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:29 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/2538457161' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:21:29.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:29 smithi148 ceph-mon[31198]: osdmap e287: 8 total, 8 up, 8 in 2024-02-13T14:21:30.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:30 smithi087 ceph-mon[27372]: osdmap e288: 8 total, 8 up, 8 in 2024-02-13T14:21:30.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:30 smithi087 ceph-mon[39120]: osdmap e288: 8 total, 8 up, 8 in 2024-02-13T14:21:30.618 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:30 smithi148 ceph-mon[31198]: osdmap e288: 8 total, 8 up, 8 in 2024-02-13T14:21:31.491 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:31 smithi087 ceph-mon[27372]: pgmap v385: 164 pgs: 164 active+clean; 582 KiB data, 337 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:21:31.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:31 smithi087 ceph-mon[27372]: osdmap e289: 8 total, 8 up, 8 in 2024-02-13T14:21:31.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:31 smithi087 ceph-mon[39120]: pgmap v385: 164 pgs: 164 active+clean; 582 KiB data, 337 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:21:31.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:31 smithi087 ceph-mon[39120]: osdmap e289: 8 total, 8 up, 8 in 2024-02-13T14:21:31.618 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:31 smithi148 ceph-mon[31198]: pgmap v385: 164 pgs: 164 active+clean; 582 KiB data, 337 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:21:31.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:31 smithi148 ceph-mon[31198]: osdmap e289: 8 total, 8 up, 8 in 2024-02-13T14:21:32.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:32 smithi087 ceph-mon[27372]: osdmap e290: 8 total, 8 up, 8 in 2024-02-13T14:21:32.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:32 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/3043220237' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:21:32.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:32 smithi087 ceph-mon[27372]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:21:32.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:32 smithi087 ceph-mon[39120]: osdmap e290: 8 total, 8 up, 8 in 2024-02-13T14:21:32.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:32 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/3043220237' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:21:32.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:32 smithi087 ceph-mon[39120]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:21:32.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:32 smithi148 ceph-mon[31198]: osdmap e290: 8 total, 8 up, 8 in 2024-02-13T14:21:32.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:32 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/3043220237' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:21:32.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:32 smithi148 ceph-mon[31198]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:21:33.258 INFO:tasks.workunit.client.0.smithi087.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_aio_stat PASSED [ 74%] 2024-02-13T14:21:33.618 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:33 smithi148 ceph-mon[31198]: pgmap v388: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 337 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:21:33.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:33 smithi148 ceph-mon[31198]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:21:33.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:33 smithi148 ceph-mon[31198]: osdmap e291: 8 total, 8 up, 8 in 2024-02-13T14:21:33.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:33 smithi148 ceph-mon[31198]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T14:21:33.741 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:33 smithi087 ceph-mon[27372]: pgmap v388: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 337 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:21:33.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:33 smithi087 ceph-mon[27372]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:21:33.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:33 smithi087 ceph-mon[27372]: osdmap e291: 8 total, 8 up, 8 in 2024-02-13T14:21:33.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:33 smithi087 ceph-mon[27372]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T14:21:33.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:33 smithi087 ceph-mon[39120]: pgmap v388: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 337 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:21:33.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:33 smithi087 ceph-mon[39120]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:21:33.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:33 smithi087 ceph-mon[39120]: osdmap e291: 8 total, 8 up, 8 in 2024-02-13T14:21:33.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:33 smithi087 ceph-mon[39120]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T14:21:34.618 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:34 smithi148 ceph-mon[31198]: osdmap e292: 8 total, 8 up, 8 in 2024-02-13T14:21:34.741 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:34 smithi087 ceph-mon[27372]: osdmap e292: 8 total, 8 up, 8 in 2024-02-13T14:21:34.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:34 smithi087 ceph-mon[39120]: osdmap e292: 8 total, 8 up, 8 in 2024-02-13T14:21:35.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:35 smithi148 ceph-mon[31198]: pgmap v391: 164 pgs: 164 active+clean; 582 KiB data, 337 MiB used, 715 GiB / 715 GiB avail 2024-02-13T14:21:35.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:35 smithi148 ceph-mon[31198]: osdmap e293: 8 total, 8 up, 8 in 2024-02-13T14:21:35.369 INFO:journalctl@ceph.iscsi.iscsi.a.smithi148.stdout:Feb 13 14:21:35 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-iscsi-iscsi-a[59644]: debug there is no tcmu-runner data available 2024-02-13T14:21:35.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:35 smithi087 ceph-mon[27372]: pgmap v391: 164 pgs: 164 active+clean; 582 KiB data, 337 MiB used, 715 GiB / 715 GiB avail 2024-02-13T14:21:35.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:35 smithi087 ceph-mon[27372]: osdmap e293: 8 total, 8 up, 8 in 2024-02-13T14:21:35.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:35 smithi087 ceph-mon[39120]: pgmap v391: 164 pgs: 164 active+clean; 582 KiB data, 337 MiB used, 715 GiB / 715 GiB avail 2024-02-13T14:21:35.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:35 smithi087 ceph-mon[39120]: osdmap e293: 8 total, 8 up, 8 in 2024-02-13T14:21:36.618 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:36 smithi148 ceph-mon[31198]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:21:36.618 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:36 smithi148 ceph-mon[31198]: osdmap e294: 8 total, 8 up, 8 in 2024-02-13T14:21:36.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:36 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/3929290799' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:21:36.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:36 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:21:36.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:36 smithi087 ceph-mon[27372]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:21:36.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:36 smithi087 ceph-mon[27372]: osdmap e294: 8 total, 8 up, 8 in 2024-02-13T14:21:36.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:36 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/3929290799' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:21:36.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:36 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:21:36.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:36 smithi087 ceph-mon[39120]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:21:36.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:36 smithi087 ceph-mon[39120]: osdmap e294: 8 total, 8 up, 8 in 2024-02-13T14:21:36.743 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:36 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/3929290799' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:21:36.743 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:36 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:21:37.282 INFO:tasks.workunit.client.0.smithi087.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_aio_remove PASSED [ 75%] 2024-02-13T14:21:37.491 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:21:37 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: ::ffff:172.21.15.148 - - [13/Feb/2024:14:21:37] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-13T14:21:37.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:37 smithi087 ceph-mon[27372]: pgmap v394: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 338 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:21:37.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:37 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/3929290799' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:21:37.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:37 smithi087 ceph-mon[27372]: osdmap e295: 8 total, 8 up, 8 in 2024-02-13T14:21:37.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:37 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:21:37.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:37 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:21:37.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:37 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:21:37.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:37 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:21:37.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:37 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:21:37.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:37 smithi087 ceph-mon[39120]: pgmap v394: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 338 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:21:37.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:37 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/3929290799' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:21:37.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:37 smithi087 ceph-mon[39120]: osdmap e295: 8 total, 8 up, 8 in 2024-02-13T14:21:37.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:37 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:21:37.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:37 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:21:37.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:37 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:21:37.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:37 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:21:37.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:37 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:21:37.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:37 smithi148 ceph-mon[31198]: pgmap v394: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 338 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:21:37.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:37 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/3929290799' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:21:37.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:37 smithi148 ceph-mon[31198]: osdmap e295: 8 total, 8 up, 8 in 2024-02-13T14:21:37.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:37 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:21:37.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:37 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:21:37.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:37 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:21:37.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:37 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:21:37.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:37 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:21:38.618 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:38 smithi148 ceph-mon[31198]: osdmap e296: 8 total, 8 up, 8 in 2024-02-13T14:21:38.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:38 smithi087 ceph-mon[27372]: osdmap e296: 8 total, 8 up, 8 in 2024-02-13T14:21:38.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:38 smithi087 ceph-mon[39120]: osdmap e296: 8 total, 8 up, 8 in 2024-02-13T14:21:39.618 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:39 smithi148 ceph-mon[31198]: pgmap v397: 164 pgs: 164 active+clean; 582 KiB data, 338 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:21:39.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:39 smithi148 ceph-mon[31198]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T14:21:39.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:39 smithi148 ceph-mon[31198]: osdmap e297: 8 total, 8 up, 8 in 2024-02-13T14:21:39.741 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:39 smithi087 ceph-mon[27372]: pgmap v397: 164 pgs: 164 active+clean; 582 KiB data, 338 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:21:39.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:39 smithi087 ceph-mon[27372]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T14:21:39.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:39 smithi087 ceph-mon[27372]: osdmap e297: 8 total, 8 up, 8 in 2024-02-13T14:21:39.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:39 smithi087 ceph-mon[39120]: pgmap v397: 164 pgs: 164 active+clean; 582 KiB data, 338 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:21:39.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:39 smithi087 ceph-mon[39120]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T14:21:39.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:39 smithi087 ceph-mon[39120]: osdmap e297: 8 total, 8 up, 8 in 2024-02-13T14:21:40.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:40 smithi148 ceph-mon[31198]: osdmap e298: 8 total, 8 up, 8 in 2024-02-13T14:21:40.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:40 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-13T14:21:40.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:40 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/1423304268' entity='client.admin' cmd=[{"prefix": "osd map", "pool": "test_pool", "object": "foo", "format": "json"}]: dispatch 2024-02-13T14:21:40.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:40 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/1423304268' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noup"}]: dispatch 2024-02-13T14:21:40.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:40 smithi148 ceph-mon[31198]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noup"}]: dispatch 2024-02-13T14:21:40.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:40 smithi087 ceph-mon[39120]: osdmap e298: 8 total, 8 up, 8 in 2024-02-13T14:21:40.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:40 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-13T14:21:40.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:40 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/1423304268' entity='client.admin' cmd=[{"prefix": "osd map", "pool": "test_pool", "object": "foo", "format": "json"}]: dispatch 2024-02-13T14:21:40.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:40 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/1423304268' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noup"}]: dispatch 2024-02-13T14:21:40.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:40 smithi087 ceph-mon[39120]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noup"}]: dispatch 2024-02-13T14:21:40.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:40 smithi087 ceph-mon[27372]: osdmap e298: 8 total, 8 up, 8 in 2024-02-13T14:21:40.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:40 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-13T14:21:40.743 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:40 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/1423304268' entity='client.admin' cmd=[{"prefix": "osd map", "pool": "test_pool", "object": "foo", "format": "json"}]: dispatch 2024-02-13T14:21:40.743 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:40 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/1423304268' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noup"}]: dispatch 2024-02-13T14:21:40.743 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:40 smithi087 ceph-mon[27372]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noup"}]: dispatch 2024-02-13T14:21:40.743 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:40 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mon-a[27349]: 2024-02-13T14:21:40.306+0000 7f6feb25f700 -1 mon.a@0(leader).osd e299 definitely_dead 0 2024-02-13T14:21:41.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:41 smithi148 ceph-mon[31198]: pgmap v400: 196 pgs: 196 active+clean; 582 KiB data, 339 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-13T14:21:41.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:41 smithi148 ceph-mon[31198]: Health check failed: noup flag(s) set (OSDMAP_FLAGS) 2024-02-13T14:21:41.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:41 smithi148 ceph-mon[31198]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noup"}]': finished 2024-02-13T14:21:41.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:41 smithi148 ceph-mon[31198]: osdmap e299: 8 total, 8 up, 8 in 2024-02-13T14:21:41.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:41 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/1423304268' entity='client.admin' cmd=[{"prefix": "osd down", "ids": ["5", "2", "4"]}]: dispatch 2024-02-13T14:21:41.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:41 smithi148 ceph-mon[31198]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd down", "ids": ["5", "2", "4"]}]: dispatch 2024-02-13T14:21:41.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:41 smithi087 ceph-mon[27372]: pgmap v400: 196 pgs: 196 active+clean; 582 KiB data, 339 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-13T14:21:41.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:41 smithi087 ceph-mon[27372]: Health check failed: noup flag(s) set (OSDMAP_FLAGS) 2024-02-13T14:21:41.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:41 smithi087 ceph-mon[27372]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noup"}]': finished 2024-02-13T14:21:41.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:41 smithi087 ceph-mon[27372]: osdmap e299: 8 total, 8 up, 8 in 2024-02-13T14:21:41.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:41 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/1423304268' entity='client.admin' cmd=[{"prefix": "osd down", "ids": ["5", "2", "4"]}]: dispatch 2024-02-13T14:21:41.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:41 smithi087 ceph-mon[27372]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd down", "ids": ["5", "2", "4"]}]: dispatch 2024-02-13T14:21:41.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:41 smithi087 ceph-mon[39120]: pgmap v400: 196 pgs: 196 active+clean; 582 KiB data, 339 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-13T14:21:41.743 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:41 smithi087 ceph-mon[39120]: Health check failed: noup flag(s) set (OSDMAP_FLAGS) 2024-02-13T14:21:41.743 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:41 smithi087 ceph-mon[39120]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noup"}]': finished 2024-02-13T14:21:41.743 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:41 smithi087 ceph-mon[39120]: osdmap e299: 8 total, 8 up, 8 in 2024-02-13T14:21:41.743 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:41 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/1423304268' entity='client.admin' cmd=[{"prefix": "osd down", "ids": ["5", "2", "4"]}]: dispatch 2024-02-13T14:21:41.743 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:41 smithi087 ceph-mon[39120]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd down", "ids": ["5", "2", "4"]}]: dispatch 2024-02-13T14:21:42.618 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:42 smithi148 ceph-mon[31198]: Health check failed: 3 osds down (OSD_DOWN) 2024-02-13T14:21:42.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:42 smithi148 ceph-mon[31198]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd down", "ids": ["5", "2", "4"]}]': finished 2024-02-13T14:21:42.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:42 smithi148 ceph-mon[31198]: osdmap e300: 8 total, 5 up, 8 in 2024-02-13T14:21:42.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:42 smithi087 ceph-mon[39120]: Health check failed: 3 osds down (OSD_DOWN) 2024-02-13T14:21:42.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:42 smithi087 ceph-mon[39120]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd down", "ids": ["5", "2", "4"]}]': finished 2024-02-13T14:21:42.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:42 smithi087 ceph-mon[39120]: osdmap e300: 8 total, 5 up, 8 in 2024-02-13T14:21:42.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:42 smithi087 ceph-mon[27372]: Health check failed: 3 osds down (OSD_DOWN) 2024-02-13T14:21:42.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:42 smithi087 ceph-mon[27372]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd down", "ids": ["5", "2", "4"]}]': finished 2024-02-13T14:21:42.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:42 smithi087 ceph-mon[27372]: osdmap e300: 8 total, 5 up, 8 in 2024-02-13T14:21:43.619 INFO:journalctl@ceph.osd.5.smithi148.stdout:Feb 13 14:21:43 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-5[42659]: 2024-02-13T14:21:43.374+0000 7f973d1d1700 -1 osd.5 302 osdmap NOUP flag is set, waiting for it to clear 2024-02-13T14:21:43.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:43 smithi148 ceph-mon[31198]: pgmap v403: 196 pgs: 68 stale+active+clean, 128 active+clean; 582 KiB data, 339 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-13T14:21:43.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:43 smithi148 ceph-mon[31198]: osdmap e301: 8 total, 5 up, 8 in 2024-02-13T14:21:43.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:43 smithi148 ceph-mon[31198]: Monitor daemon marked osd.5 down, but it is still running 2024-02-13T14:21:43.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:43 smithi148 ceph-mon[31198]: map e301 wrongly marked me down at e300 2024-02-13T14:21:43.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:43 smithi148 ceph-mon[31198]: osd.5 marked itself dead as of e301 2024-02-13T14:21:43.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:43 smithi148 ceph-mon[31198]: Monitor daemon marked osd.4 down, but it is still running 2024-02-13T14:21:43.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:43 smithi148 ceph-mon[31198]: map e301 wrongly marked me down at e300 2024-02-13T14:21:43.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:43 smithi148 ceph-mon[31198]: osd.4 marked itself dead as of e301 2024-02-13T14:21:43.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:43 smithi087 ceph-mon[39120]: pgmap v403: 196 pgs: 68 stale+active+clean, 128 active+clean; 582 KiB data, 339 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-13T14:21:43.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:43 smithi087 ceph-mon[39120]: osdmap e301: 8 total, 5 up, 8 in 2024-02-13T14:21:43.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:43 smithi087 ceph-mon[39120]: Monitor daemon marked osd.5 down, but it is still running 2024-02-13T14:21:43.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:43 smithi087 ceph-mon[39120]: map e301 wrongly marked me down at e300 2024-02-13T14:21:43.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:43 smithi087 ceph-mon[39120]: osd.5 marked itself dead as of e301 2024-02-13T14:21:43.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:43 smithi087 ceph-mon[39120]: Monitor daemon marked osd.4 down, but it is still running 2024-02-13T14:21:43.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:43 smithi087 ceph-mon[39120]: map e301 wrongly marked me down at e300 2024-02-13T14:21:43.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:43 smithi087 ceph-mon[39120]: osd.4 marked itself dead as of e301 2024-02-13T14:21:43.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:43 smithi087 ceph-mon[27372]: pgmap v403: 196 pgs: 68 stale+active+clean, 128 active+clean; 582 KiB data, 339 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-13T14:21:43.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:43 smithi087 ceph-mon[27372]: osdmap e301: 8 total, 5 up, 8 in 2024-02-13T14:21:43.743 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:43 smithi087 ceph-mon[27372]: Monitor daemon marked osd.5 down, but it is still running 2024-02-13T14:21:43.743 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:43 smithi087 ceph-mon[27372]: map e301 wrongly marked me down at e300 2024-02-13T14:21:43.743 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:43 smithi087 ceph-mon[27372]: osd.5 marked itself dead as of e301 2024-02-13T14:21:43.743 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:43 smithi087 ceph-mon[27372]: Monitor daemon marked osd.4 down, but it is still running 2024-02-13T14:21:43.743 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:43 smithi087 ceph-mon[27372]: map e301 wrongly marked me down at e300 2024-02-13T14:21:43.743 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:43 smithi087 ceph-mon[27372]: osd.4 marked itself dead as of e301 2024-02-13T14:21:44.333 INFO:journalctl@ceph.osd.4.smithi148.stdout:Feb 13 14:21:43 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-4[36733]: 2024-02-13T14:21:43.881+0000 7f4aeff9a700 -1 osd.4 302 osdmap NOUP flag is set, waiting for it to clear 2024-02-13T14:21:44.618 INFO:journalctl@ceph.osd.4.smithi148.stdout:Feb 13 14:21:44 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-4[36733]: 2024-02-13T14:21:44.340+0000 7f4ae7584700 -1 osd.4 303 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-13T14:21:44.619 INFO:journalctl@ceph.osd.5.smithi148.stdout:Feb 13 14:21:44 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-5[42659]: 2024-02-13T14:21:44.340+0000 7f9734fbc700 -1 osd.5 303 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-13T14:21:44.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:44 smithi148 ceph-mon[31198]: osdmap e302: 8 total, 5 up, 8 in 2024-02-13T14:21:44.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:44 smithi148 ceph-mon[31198]: osd.2 marked itself dead as of e302 2024-02-13T14:21:44.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:44 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/1423304268' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:21:44.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:44 smithi148 ceph-mon[31198]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:21:44.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:44 smithi087 ceph-mon[27372]: osdmap e302: 8 total, 5 up, 8 in 2024-02-13T14:21:44.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:44 smithi087 ceph-mon[27372]: osd.2 marked itself dead as of e302 2024-02-13T14:21:44.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:44 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/1423304268' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:21:44.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:44 smithi087 ceph-mon[27372]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:21:44.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:44 smithi087 ceph-mon[39120]: osdmap e302: 8 total, 5 up, 8 in 2024-02-13T14:21:44.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:44 smithi087 ceph-mon[39120]: osd.2 marked itself dead as of e302 2024-02-13T14:21:44.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:44 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/1423304268' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:21:44.743 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:44 smithi087 ceph-mon[39120]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:21:45.242 INFO:journalctl@ceph.osd.2.smithi087.stdout:Feb 13 14:21:44 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-2[55599]: 2024-02-13T14:21:44.811+0000 7f77be68a700 -1 osd.2 303 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-13T14:21:45.345 INFO:journalctl@ceph.iscsi.iscsi.a.smithi148.stdout:Feb 13 14:21:45 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-iscsi-iscsi-a[59644]: debug there is no tcmu-runner data available 2024-02-13T14:21:45.618 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:45 smithi148 ceph-mon[31198]: pgmap v406: 196 pgs: 68 stale+active+clean, 128 active+clean; 582 KiB data, 339 MiB used, 715 GiB / 715 GiB avail 2024-02-13T14:21:45.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:45 smithi148 ceph-mon[31198]: Monitor daemon marked osd.2 down, but it is still running 2024-02-13T14:21:45.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:45 smithi148 ceph-mon[31198]: map e302 wrongly marked me down at e300 2024-02-13T14:21:45.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:45 smithi148 ceph-mon[31198]: Health check cleared: OSDMAP_FLAGS (was: noup flag(s) set) 2024-02-13T14:21:45.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:45 smithi148 ceph-mon[31198]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:21:45.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:45 smithi148 ceph-mon[31198]: osdmap e303: 8 total, 5 up, 8 in 2024-02-13T14:21:45.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:45 smithi087 ceph-mon[27372]: pgmap v406: 196 pgs: 68 stale+active+clean, 128 active+clean; 582 KiB data, 339 MiB used, 715 GiB / 715 GiB avail 2024-02-13T14:21:45.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:45 smithi087 ceph-mon[27372]: Monitor daemon marked osd.2 down, but it is still running 2024-02-13T14:21:45.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:45 smithi087 ceph-mon[27372]: map e302 wrongly marked me down at e300 2024-02-13T14:21:45.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:45 smithi087 ceph-mon[27372]: Health check cleared: OSDMAP_FLAGS (was: noup flag(s) set) 2024-02-13T14:21:45.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:45 smithi087 ceph-mon[27372]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:21:45.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:45 smithi087 ceph-mon[27372]: osdmap e303: 8 total, 5 up, 8 in 2024-02-13T14:21:45.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:45 smithi087 ceph-mon[39120]: pgmap v406: 196 pgs: 68 stale+active+clean, 128 active+clean; 582 KiB data, 339 MiB used, 715 GiB / 715 GiB avail 2024-02-13T14:21:45.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:45 smithi087 ceph-mon[39120]: Monitor daemon marked osd.2 down, but it is still running 2024-02-13T14:21:45.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:45 smithi087 ceph-mon[39120]: map e302 wrongly marked me down at e300 2024-02-13T14:21:45.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:45 smithi087 ceph-mon[39120]: Health check cleared: OSDMAP_FLAGS (was: noup flag(s) set) 2024-02-13T14:21:45.743 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:45 smithi087 ceph-mon[39120]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:21:45.743 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:45 smithi087 ceph-mon[39120]: osdmap e303: 8 total, 5 up, 8 in 2024-02-13T14:21:46.618 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:46 smithi148 ceph-mon[31198]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:21:46.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:46 smithi148 ceph-mon[31198]: Health check cleared: OSD_DOWN (was: 3 osds down) 2024-02-13T14:21:46.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:46 smithi148 ceph-mon[31198]: osd.5 [v2:172.21.15.148:6808/4089539002,v1:172.21.15.148:6809/4089539002] boot 2024-02-13T14:21:46.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:46 smithi148 ceph-mon[31198]: osd.4 [v2:172.21.15.148:6800/1403587816,v1:172.21.15.148:6801/1403587816] boot 2024-02-13T14:21:46.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:46 smithi148 ceph-mon[31198]: osd.2 [v2:172.21.15.87:6818/1764493594,v1:172.21.15.87:6819/1764493594] boot 2024-02-13T14:21:46.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:46 smithi148 ceph-mon[31198]: osdmap e304: 8 total, 8 up, 8 in 2024-02-13T14:21:46.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:46 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-13T14:21:46.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:46 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-13T14:21:46.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:46 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-13T14:21:46.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:46 smithi148 ceph-mon[31198]: osdmap e305: 8 total, 8 up, 8 in 2024-02-13T14:21:46.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:46 smithi087 ceph-mon[27372]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:21:46.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:46 smithi087 ceph-mon[27372]: Health check cleared: OSD_DOWN (was: 3 osds down) 2024-02-13T14:21:46.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:46 smithi087 ceph-mon[27372]: osd.5 [v2:172.21.15.148:6808/4089539002,v1:172.21.15.148:6809/4089539002] boot 2024-02-13T14:21:46.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:46 smithi087 ceph-mon[27372]: osd.4 [v2:172.21.15.148:6800/1403587816,v1:172.21.15.148:6801/1403587816] boot 2024-02-13T14:21:46.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:46 smithi087 ceph-mon[27372]: osd.2 [v2:172.21.15.87:6818/1764493594,v1:172.21.15.87:6819/1764493594] boot 2024-02-13T14:21:46.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:46 smithi087 ceph-mon[27372]: osdmap e304: 8 total, 8 up, 8 in 2024-02-13T14:21:46.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:46 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-13T14:21:46.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:46 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-13T14:21:46.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:46 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-13T14:21:46.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:46 smithi087 ceph-mon[27372]: osdmap e305: 8 total, 8 up, 8 in 2024-02-13T14:21:46.743 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:46 smithi087 ceph-mon[39120]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:21:46.743 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:46 smithi087 ceph-mon[39120]: Health check cleared: OSD_DOWN (was: 3 osds down) 2024-02-13T14:21:46.743 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:46 smithi087 ceph-mon[39120]: osd.5 [v2:172.21.15.148:6808/4089539002,v1:172.21.15.148:6809/4089539002] boot 2024-02-13T14:21:46.743 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:46 smithi087 ceph-mon[39120]: osd.4 [v2:172.21.15.148:6800/1403587816,v1:172.21.15.148:6801/1403587816] boot 2024-02-13T14:21:46.743 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:46 smithi087 ceph-mon[39120]: osd.2 [v2:172.21.15.87:6818/1764493594,v1:172.21.15.87:6819/1764493594] boot 2024-02-13T14:21:46.743 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:46 smithi087 ceph-mon[39120]: osdmap e304: 8 total, 8 up, 8 in 2024-02-13T14:21:46.743 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:46 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-13T14:21:46.743 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:46 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-13T14:21:46.743 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:46 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-13T14:21:46.743 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:46 smithi087 ceph-mon[39120]: osdmap e305: 8 total, 8 up, 8 in 2024-02-13T14:21:47.492 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:21:47 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: ::ffff:172.21.15.148 - - [13/Feb/2024:14:21:47] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-13T14:21:47.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:47 smithi087 ceph-mon[27372]: pgmap v409: 196 pgs: 2 undersized+degraded+peered+wait, 4 active+undersized+degraded+wait, 34 active+clean, 32 undersized+peered, 70 active+undersized, 3 stale+active+clean, 5 undersized+degraded+peered, 28 active+undersized+degraded, 13 active+undersized+wait, 5 undersized+peered+wait; 582 KiB data, 343 MiB used, 715 GiB / 715 GiB avail; 171/597 objects degraded (28.643%) 2024-02-13T14:21:47.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:47 smithi087 ceph-mon[27372]: Health check failed: Reduced data availability: 19 pgs inactive (PG_AVAILABILITY) 2024-02-13T14:21:47.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:47 smithi087 ceph-mon[27372]: Health check failed: Degraded data redundancy: 171/597 objects degraded (28.643%), 39 pgs degraded (PG_DEGRADED) 2024-02-13T14:21:47.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:47 smithi087 ceph-mon[27372]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T14:21:47.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:47 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/1423304268' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:21:47.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:47 smithi087 ceph-mon[27372]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:21:47.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:47 smithi087 ceph-mon[39120]: pgmap v409: 196 pgs: 2 undersized+degraded+peered+wait, 4 active+undersized+degraded+wait, 34 active+clean, 32 undersized+peered, 70 active+undersized, 3 stale+active+clean, 5 undersized+degraded+peered, 28 active+undersized+degraded, 13 active+undersized+wait, 5 undersized+peered+wait; 582 KiB data, 343 MiB used, 715 GiB / 715 GiB avail; 171/597 objects degraded (28.643%) 2024-02-13T14:21:47.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:47 smithi087 ceph-mon[39120]: Health check failed: Reduced data availability: 19 pgs inactive (PG_AVAILABILITY) 2024-02-13T14:21:47.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:47 smithi087 ceph-mon[39120]: Health check failed: Degraded data redundancy: 171/597 objects degraded (28.643%), 39 pgs degraded (PG_DEGRADED) 2024-02-13T14:21:47.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:47 smithi087 ceph-mon[39120]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T14:21:47.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:47 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/1423304268' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:21:47.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:47 smithi087 ceph-mon[39120]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:21:47.618 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:47 smithi148 ceph-mon[31198]: pgmap v409: 196 pgs: 2 undersized+degraded+peered+wait, 4 active+undersized+degraded+wait, 34 active+clean, 32 undersized+peered, 70 active+undersized, 3 stale+active+clean, 5 undersized+degraded+peered, 28 active+undersized+degraded, 13 active+undersized+wait, 5 undersized+peered+wait; 582 KiB data, 343 MiB used, 715 GiB / 715 GiB avail; 171/597 objects degraded (28.643%) 2024-02-13T14:21:47.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:47 smithi148 ceph-mon[31198]: Health check failed: Reduced data availability: 19 pgs inactive (PG_AVAILABILITY) 2024-02-13T14:21:47.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:47 smithi148 ceph-mon[31198]: Health check failed: Degraded data redundancy: 171/597 objects degraded (28.643%), 39 pgs degraded (PG_DEGRADED) 2024-02-13T14:21:47.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:47 smithi148 ceph-mon[31198]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T14:21:47.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:47 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/1423304268' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:21:47.619 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:47 smithi148 ceph-mon[31198]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:21:48.371 INFO:tasks.workunit.client.0.smithi087.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_aio_read_wait_for_complete PASSED [ 76%] 2024-02-13T14:21:48.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:48 smithi087 ceph-mon[27372]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:21:48.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:48 smithi087 ceph-mon[27372]: osdmap e306: 8 total, 8 up, 8 in 2024-02-13T14:21:48.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:48 smithi087 ceph-mon[27372]: osdmap e307: 8 total, 8 up, 8 in 2024-02-13T14:21:48.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:48 smithi087 ceph-mon[39120]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:21:48.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:48 smithi087 ceph-mon[39120]: osdmap e306: 8 total, 8 up, 8 in 2024-02-13T14:21:48.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:48 smithi087 ceph-mon[39120]: osdmap e307: 8 total, 8 up, 8 in 2024-02-13T14:21:48.868 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:48 smithi148 ceph-mon[31198]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:21:48.868 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:48 smithi148 ceph-mon[31198]: osdmap e306: 8 total, 8 up, 8 in 2024-02-13T14:21:48.868 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:48 smithi148 ceph-mon[31198]: osdmap e307: 8 total, 8 up, 8 in 2024-02-13T14:21:49.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:49 smithi087 ceph-mon[27372]: pgmap v412: 196 pgs: 2 undersized+degraded+peered+wait, 4 active+undersized+degraded+wait, 34 active+clean, 32 undersized+peered, 70 active+undersized, 3 stale+active+clean, 5 undersized+degraded+peered, 28 active+undersized+degraded, 13 active+undersized+wait, 5 undersized+peered+wait; 582 KiB data, 343 MiB used, 715 GiB / 715 GiB avail; 171/597 objects degraded (28.643%) 2024-02-13T14:21:49.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:49 smithi087 ceph-mon[39120]: pgmap v412: 196 pgs: 2 undersized+degraded+peered+wait, 4 active+undersized+degraded+wait, 34 active+clean, 32 undersized+peered, 70 active+undersized, 3 stale+active+clean, 5 undersized+degraded+peered, 28 active+undersized+degraded, 13 active+undersized+wait, 5 undersized+peered+wait; 582 KiB data, 343 MiB used, 715 GiB / 715 GiB avail; 171/597 objects degraded (28.643%) 2024-02-13T14:21:49.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:49 smithi148 ceph-mon[31198]: pgmap v412: 196 pgs: 2 undersized+degraded+peered+wait, 4 active+undersized+degraded+wait, 34 active+clean, 32 undersized+peered, 70 active+undersized, 3 stale+active+clean, 5 undersized+degraded+peered, 28 active+undersized+degraded, 13 active+undersized+wait, 5 undersized+peered+wait; 582 KiB data, 343 MiB used, 715 GiB / 715 GiB avail; 171/597 objects degraded (28.643%) 2024-02-13T14:21:50.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:50 smithi087 ceph-mon[27372]: osdmap e308: 8 total, 8 up, 8 in 2024-02-13T14:21:50.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:50 smithi087 ceph-mon[39120]: osdmap e308: 8 total, 8 up, 8 in 2024-02-13T14:21:50.868 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:50 smithi148 ceph-mon[31198]: osdmap e308: 8 total, 8 up, 8 in 2024-02-13T14:21:51.643 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:51 smithi087 ceph-mon[39120]: pgmap v415: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 352 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s; 82 B/s, 1 objects/s recovering 2024-02-13T14:21:51.643 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:51 smithi087 ceph-mon[39120]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 19 pgs inactive) 2024-02-13T14:21:51.643 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:51 smithi087 ceph-mon[39120]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 171/597 objects degraded (28.643%), 39 pgs degraded) 2024-02-13T14:21:51.643 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:51 smithi087 ceph-mon[39120]: osdmap e309: 8 total, 8 up, 8 in 2024-02-13T14:21:51.643 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:51 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/3340540793' entity='client.admin' cmd=[{"prefix": "osd map", "pool": "test_pool", "object": "foo", "format": "json"}]: dispatch 2024-02-13T14:21:51.643 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:51 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/3340540793' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noup"}]: dispatch 2024-02-13T14:21:51.644 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:51 smithi087 ceph-mon[39120]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noup"}]: dispatch 2024-02-13T14:21:51.644 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:51 smithi087 ceph-mon[27372]: pgmap v415: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 352 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s; 82 B/s, 1 objects/s recovering 2024-02-13T14:21:51.644 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:51 smithi087 ceph-mon[27372]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 19 pgs inactive) 2024-02-13T14:21:51.644 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:51 smithi087 ceph-mon[27372]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 171/597 objects degraded (28.643%), 39 pgs degraded) 2024-02-13T14:21:51.644 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:51 smithi087 ceph-mon[27372]: osdmap e309: 8 total, 8 up, 8 in 2024-02-13T14:21:51.644 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:51 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/3340540793' entity='client.admin' cmd=[{"prefix": "osd map", "pool": "test_pool", "object": "foo", "format": "json"}]: dispatch 2024-02-13T14:21:51.644 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:51 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/3340540793' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noup"}]: dispatch 2024-02-13T14:21:51.644 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:51 smithi087 ceph-mon[27372]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noup"}]: dispatch 2024-02-13T14:21:51.644 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:51 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mon-a[27349]: 2024-02-13T14:21:51.397+0000 7f6feb25f700 -1 mon.a@0(leader).osd e310 definitely_dead 0 2024-02-13T14:21:51.868 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:51 smithi148 ceph-mon[31198]: pgmap v415: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 352 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s; 82 B/s, 1 objects/s recovering 2024-02-13T14:21:51.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:51 smithi148 ceph-mon[31198]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 19 pgs inactive) 2024-02-13T14:21:51.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:51 smithi148 ceph-mon[31198]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 171/597 objects degraded (28.643%), 39 pgs degraded) 2024-02-13T14:21:51.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:51 smithi148 ceph-mon[31198]: osdmap e309: 8 total, 8 up, 8 in 2024-02-13T14:21:51.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:51 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/3340540793' entity='client.admin' cmd=[{"prefix": "osd map", "pool": "test_pool", "object": "foo", "format": "json"}]: dispatch 2024-02-13T14:21:51.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:51 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/3340540793' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noup"}]: dispatch 2024-02-13T14:21:51.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:51 smithi148 ceph-mon[31198]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noup"}]: dispatch 2024-02-13T14:21:52.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:52 smithi087 ceph-mon[39120]: Health check failed: noup flag(s) set (OSDMAP_FLAGS) 2024-02-13T14:21:52.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:52 smithi087 ceph-mon[39120]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noup"}]': finished 2024-02-13T14:21:52.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:52 smithi087 ceph-mon[39120]: osdmap e310: 8 total, 8 up, 8 in 2024-02-13T14:21:52.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:52 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/3340540793' entity='client.admin' cmd=[{"prefix": "osd down", "ids": ["4", "0", "7"]}]: dispatch 2024-02-13T14:21:52.743 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:52 smithi087 ceph-mon[39120]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd down", "ids": ["4", "0", "7"]}]: dispatch 2024-02-13T14:21:52.743 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:52 smithi087 ceph-mon[27372]: Health check failed: noup flag(s) set (OSDMAP_FLAGS) 2024-02-13T14:21:52.743 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:52 smithi087 ceph-mon[27372]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noup"}]': finished 2024-02-13T14:21:52.743 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:52 smithi087 ceph-mon[27372]: osdmap e310: 8 total, 8 up, 8 in 2024-02-13T14:21:52.743 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:52 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/3340540793' entity='client.admin' cmd=[{"prefix": "osd down", "ids": ["4", "0", "7"]}]: dispatch 2024-02-13T14:21:52.744 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:52 smithi087 ceph-mon[27372]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd down", "ids": ["4", "0", "7"]}]: dispatch 2024-02-13T14:21:52.868 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:52 smithi148 ceph-mon[31198]: Health check failed: noup flag(s) set (OSDMAP_FLAGS) 2024-02-13T14:21:52.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:52 smithi148 ceph-mon[31198]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noup"}]': finished 2024-02-13T14:21:52.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:52 smithi148 ceph-mon[31198]: osdmap e310: 8 total, 8 up, 8 in 2024-02-13T14:21:52.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:52 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/3340540793' entity='client.admin' cmd=[{"prefix": "osd down", "ids": ["4", "0", "7"]}]: dispatch 2024-02-13T14:21:52.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:52 smithi148 ceph-mon[31198]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd down", "ids": ["4", "0", "7"]}]: dispatch 2024-02-13T14:21:53.368 INFO:journalctl@ceph.osd.7.smithi148.stdout:Feb 13 14:21:53 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-7[54686]: 2024-02-13T14:21:53.073+0000 7f402c82c700 -1 osd.7 311 osdmap NOUP flag is set, waiting for it to clear 2024-02-13T14:21:53.742 INFO:journalctl@ceph.osd.0.smithi087.stdout:Feb 13 14:21:53 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-0[43500]: 2024-02-13T14:21:53.328+0000 7f0a968f1700 -1 osd.0 311 osdmap NOUP flag is set, waiting for it to clear 2024-02-13T14:21:53.742 INFO:journalctl@ceph.osd.0.smithi087.stdout:Feb 13 14:21:53 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-0[43500]: 2024-02-13T14:21:53.405+0000 7f0a85ecb700 -1 osd.0 312 osdmap NOUP flag is set, waiting for it to clear 2024-02-13T14:21:53.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:53 smithi087 ceph-mon[27372]: pgmap v418: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 352 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s; 82 B/s, 1 objects/s recovering 2024-02-13T14:21:53.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:53 smithi087 ceph-mon[27372]: Health check failed: 3 osds down (OSD_DOWN) 2024-02-13T14:21:53.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:53 smithi087 ceph-mon[27372]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd down", "ids": ["4", "0", "7"]}]': finished 2024-02-13T14:21:53.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:53 smithi087 ceph-mon[27372]: osdmap e311: 8 total, 5 up, 8 in 2024-02-13T14:21:53.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:53 smithi087 ceph-mon[27372]: Monitor daemon marked osd.7 down, but it is still running 2024-02-13T14:21:53.743 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:53 smithi087 ceph-mon[27372]: map e311 wrongly marked me down at e311 2024-02-13T14:21:53.743 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:53 smithi087 ceph-mon[27372]: osd.7 marked itself dead as of e311 2024-02-13T14:21:53.743 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:53 smithi087 ceph-mon[27372]: osd.0 marked itself dead as of e311 2024-02-13T14:21:53.743 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:53 smithi087 ceph-mon[39120]: pgmap v418: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 352 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s; 82 B/s, 1 objects/s recovering 2024-02-13T14:21:53.743 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:53 smithi087 ceph-mon[39120]: Health check failed: 3 osds down (OSD_DOWN) 2024-02-13T14:21:53.743 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:53 smithi087 ceph-mon[39120]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd down", "ids": ["4", "0", "7"]}]': finished 2024-02-13T14:21:53.743 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:53 smithi087 ceph-mon[39120]: osdmap e311: 8 total, 5 up, 8 in 2024-02-13T14:21:53.743 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:53 smithi087 ceph-mon[39120]: Monitor daemon marked osd.7 down, but it is still running 2024-02-13T14:21:53.743 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:53 smithi087 ceph-mon[39120]: map e311 wrongly marked me down at e311 2024-02-13T14:21:53.743 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:53 smithi087 ceph-mon[39120]: osd.7 marked itself dead as of e311 2024-02-13T14:21:53.743 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:53 smithi087 ceph-mon[39120]: osd.0 marked itself dead as of e311 2024-02-13T14:21:53.868 INFO:journalctl@ceph.osd.7.smithi148.stdout:Feb 13 14:21:53 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-7[54686]: 2024-02-13T14:21:53.404+0000 7f401be06700 -1 osd.7 312 osdmap NOUP flag is set, waiting for it to clear 2024-02-13T14:21:53.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:53 smithi148 ceph-mon[31198]: pgmap v418: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 352 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s; 82 B/s, 1 objects/s recovering 2024-02-13T14:21:53.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:53 smithi148 ceph-mon[31198]: Health check failed: 3 osds down (OSD_DOWN) 2024-02-13T14:21:53.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:53 smithi148 ceph-mon[31198]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd down", "ids": ["4", "0", "7"]}]': finished 2024-02-13T14:21:53.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:53 smithi148 ceph-mon[31198]: osdmap e311: 8 total, 5 up, 8 in 2024-02-13T14:21:53.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:53 smithi148 ceph-mon[31198]: Monitor daemon marked osd.7 down, but it is still running 2024-02-13T14:21:53.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:53 smithi148 ceph-mon[31198]: map e311 wrongly marked me down at e311 2024-02-13T14:21:53.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:53 smithi148 ceph-mon[31198]: osd.7 marked itself dead as of e311 2024-02-13T14:21:53.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:53 smithi148 ceph-mon[31198]: osd.0 marked itself dead as of e311 2024-02-13T14:21:54.742 INFO:journalctl@ceph.osd.0.smithi087.stdout:Feb 13 14:21:54 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-0[43500]: 2024-02-13T14:21:54.407+0000 7f0a85ecb700 -1 osd.0 313 osdmap NOUP flag is set, waiting for it to clear 2024-02-13T14:21:54.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:54 smithi087 ceph-mon[39120]: Monitor daemon marked osd.0 down, but it is still running 2024-02-13T14:21:54.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:54 smithi087 ceph-mon[39120]: map e311 wrongly marked me down at e311 2024-02-13T14:21:54.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:54 smithi087 ceph-mon[39120]: osdmap e312: 8 total, 5 up, 8 in 2024-02-13T14:21:54.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:54 smithi087 ceph-mon[39120]: Monitor daemon marked osd.4 down, but it is still running 2024-02-13T14:21:54.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:54 smithi087 ceph-mon[39120]: map e312 wrongly marked me down at e311 2024-02-13T14:21:54.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:54 smithi087 ceph-mon[39120]: osd.4 marked itself dead as of e312 2024-02-13T14:21:54.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:54 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:21:54.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:54 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-13T14:21:54.743 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:54 smithi087 ceph-mon[27372]: Monitor daemon marked osd.0 down, but it is still running 2024-02-13T14:21:54.743 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:54 smithi087 ceph-mon[27372]: map e311 wrongly marked me down at e311 2024-02-13T14:21:54.743 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:54 smithi087 ceph-mon[27372]: osdmap e312: 8 total, 5 up, 8 in 2024-02-13T14:21:54.743 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:54 smithi087 ceph-mon[27372]: Monitor daemon marked osd.4 down, but it is still running 2024-02-13T14:21:54.743 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:54 smithi087 ceph-mon[27372]: map e312 wrongly marked me down at e311 2024-02-13T14:21:54.743 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:54 smithi087 ceph-mon[27372]: osd.4 marked itself dead as of e312 2024-02-13T14:21:54.743 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:54 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:21:54.743 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:54 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-13T14:21:54.869 INFO:journalctl@ceph.osd.7.smithi148.stdout:Feb 13 14:21:54 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-7[54686]: 2024-02-13T14:21:54.410+0000 7f401be06700 -1 osd.7 313 osdmap NOUP flag is set, waiting for it to clear 2024-02-13T14:21:54.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:54 smithi148 ceph-mon[31198]: Monitor daemon marked osd.0 down, but it is still running 2024-02-13T14:21:54.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:54 smithi148 ceph-mon[31198]: map e311 wrongly marked me down at e311 2024-02-13T14:21:54.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:54 smithi148 ceph-mon[31198]: osdmap e312: 8 total, 5 up, 8 in 2024-02-13T14:21:54.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:54 smithi148 ceph-mon[31198]: Monitor daemon marked osd.4 down, but it is still running 2024-02-13T14:21:54.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:54 smithi148 ceph-mon[31198]: map e312 wrongly marked me down at e311 2024-02-13T14:21:54.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:54 smithi148 ceph-mon[31198]: osd.4 marked itself dead as of e312 2024-02-13T14:21:54.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:54 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:21:54.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:54 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-13T14:21:55.368 INFO:journalctl@ceph.osd.4.smithi148.stdout:Feb 13 14:21:54 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-4[36733]: 2024-02-13T14:21:54.943+0000 7f4aef799700 -1 osd.4 313 osdmap NOUP flag is set, waiting for it to clear 2024-02-13T14:21:55.369 INFO:journalctl@ceph.iscsi.iscsi.a.smithi148.stdout:Feb 13 14:21:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-iscsi-iscsi-a[59644]: debug there is no tcmu-runner data available 2024-02-13T14:21:55.741 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:55 smithi087 ceph-mon[27372]: pgmap v421: 196 pgs: 11 stale+creating+peering, 73 stale+active+clean, 21 creating+peering, 91 active+clean; 582 KiB data, 352 MiB used, 715 GiB / 715 GiB avail 2024-02-13T14:21:55.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:55 smithi087 ceph-mon[27372]: osdmap e313: 8 total, 5 up, 8 in 2024-02-13T14:21:55.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:55 smithi087 ceph-mon[39120]: pgmap v421: 196 pgs: 11 stale+creating+peering, 73 stale+active+clean, 21 creating+peering, 91 active+clean; 582 KiB data, 352 MiB used, 715 GiB / 715 GiB avail 2024-02-13T14:21:55.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:55 smithi087 ceph-mon[39120]: osdmap e313: 8 total, 5 up, 8 in 2024-02-13T14:21:55.868 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:55 smithi148 ceph-mon[31198]: pgmap v421: 196 pgs: 11 stale+creating+peering, 73 stale+active+clean, 21 creating+peering, 91 active+clean; 582 KiB data, 352 MiB used, 715 GiB / 715 GiB avail 2024-02-13T14:21:55.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:55 smithi148 ceph-mon[31198]: osdmap e313: 8 total, 5 up, 8 in 2024-02-13T14:21:56.742 INFO:journalctl@ceph.osd.0.smithi087.stdout:Feb 13 14:21:56 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-0[43500]: 2024-02-13T14:21:56.416+0000 7f0a8dedb700 -1 osd.0 314 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-13T14:21:56.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:56 smithi087 ceph-mon[27372]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:21:56.743 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:56 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/3340540793' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:21:56.743 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:56 smithi087 ceph-mon[27372]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:21:56.743 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:56 smithi087 ceph-mon[39120]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:21:56.743 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:56 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/3340540793' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:21:56.743 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:56 smithi087 ceph-mon[39120]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:21:56.770 INFO:journalctl@ceph.osd.4.smithi148.stdout:Feb 13 14:21:56 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-4[36733]: 2024-02-13T14:21:56.415+0000 7f4ae7584700 -1 osd.4 314 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-13T14:21:56.770 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:56 smithi148 ceph-mon[31198]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:21:56.770 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:56 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/3340540793' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:21:56.770 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:56 smithi148 ceph-mon[31198]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:21:56.771 INFO:journalctl@ceph.osd.7.smithi148.stdout:Feb 13 14:21:56 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-7[54686]: 2024-02-13T14:21:56.414+0000 7f4023e16700 -1 osd.7 314 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-13T14:21:57.491 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:21:57 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: ::ffff:172.21.15.148 - - [13/Feb/2024:14:21:57] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-13T14:21:57.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:57 smithi087 ceph-mon[27372]: pgmap v423: 196 pgs: 36 undersized+peered, 68 active+undersized, 25 active+undersized+degraded, 6 active+undersized+wait, 6 undersized+peered+wait, 12 undersized+degraded+peered, 3 active+undersized+degraded+wait, 3 undersized+degraded+peered+wait, 1 stale+creating+peering, 3 stale+active+clean, 33 active+clean; 582 KiB data, 337 MiB used, 715 GiB / 715 GiB avail; 217/594 objects degraded (36.532%) 2024-02-13T14:21:57.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:57 smithi087 ceph-mon[27372]: Health check cleared: OSDMAP_FLAGS (was: noup flag(s) set) 2024-02-13T14:21:57.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:57 smithi087 ceph-mon[27372]: Health check failed: Reduced data availability: 6 pgs inactive (PG_AVAILABILITY) 2024-02-13T14:21:57.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:57 smithi087 ceph-mon[27372]: Health check failed: Degraded data redundancy: 217/594 objects degraded (36.532%), 43 pgs degraded (PG_DEGRADED) 2024-02-13T14:21:57.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:57 smithi087 ceph-mon[27372]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:21:57.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:57 smithi087 ceph-mon[27372]: osdmap e314: 8 total, 5 up, 8 in 2024-02-13T14:21:57.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:57 smithi087 ceph-mon[27372]: Health check cleared: OSD_DOWN (was: 3 osds down) 2024-02-13T14:21:57.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:57 smithi087 ceph-mon[39120]: pgmap v423: 196 pgs: 36 undersized+peered, 68 active+undersized, 25 active+undersized+degraded, 6 active+undersized+wait, 6 undersized+peered+wait, 12 undersized+degraded+peered, 3 active+undersized+degraded+wait, 3 undersized+degraded+peered+wait, 1 stale+creating+peering, 3 stale+active+clean, 33 active+clean; 582 KiB data, 337 MiB used, 715 GiB / 715 GiB avail; 217/594 objects degraded (36.532%) 2024-02-13T14:21:57.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:57 smithi087 ceph-mon[39120]: Health check cleared: OSDMAP_FLAGS (was: noup flag(s) set) 2024-02-13T14:21:57.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:57 smithi087 ceph-mon[39120]: Health check failed: Reduced data availability: 6 pgs inactive (PG_AVAILABILITY) 2024-02-13T14:21:57.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:57 smithi087 ceph-mon[39120]: Health check failed: Degraded data redundancy: 217/594 objects degraded (36.532%), 43 pgs degraded (PG_DEGRADED) 2024-02-13T14:21:57.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:57 smithi087 ceph-mon[39120]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:21:57.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:57 smithi087 ceph-mon[39120]: osdmap e314: 8 total, 5 up, 8 in 2024-02-13T14:21:57.493 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:57 smithi087 ceph-mon[39120]: Health check cleared: OSD_DOWN (was: 3 osds down) 2024-02-13T14:21:57.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:57 smithi148 ceph-mon[31198]: pgmap v423: 196 pgs: 36 undersized+peered, 68 active+undersized, 25 active+undersized+degraded, 6 active+undersized+wait, 6 undersized+peered+wait, 12 undersized+degraded+peered, 3 active+undersized+degraded+wait, 3 undersized+degraded+peered+wait, 1 stale+creating+peering, 3 stale+active+clean, 33 active+clean; 582 KiB data, 337 MiB used, 715 GiB / 715 GiB avail; 217/594 objects degraded (36.532%) 2024-02-13T14:21:57.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:57 smithi148 ceph-mon[31198]: Health check cleared: OSDMAP_FLAGS (was: noup flag(s) set) 2024-02-13T14:21:57.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:57 smithi148 ceph-mon[31198]: Health check failed: Reduced data availability: 6 pgs inactive (PG_AVAILABILITY) 2024-02-13T14:21:57.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:57 smithi148 ceph-mon[31198]: Health check failed: Degraded data redundancy: 217/594 objects degraded (36.532%), 43 pgs degraded (PG_DEGRADED) 2024-02-13T14:21:57.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:57 smithi148 ceph-mon[31198]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:21:57.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:57 smithi148 ceph-mon[31198]: osdmap e314: 8 total, 5 up, 8 in 2024-02-13T14:21:57.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:57 smithi148 ceph-mon[31198]: Health check cleared: OSD_DOWN (was: 3 osds down) 2024-02-13T14:21:58.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:58 smithi087 ceph-mon[27372]: osd.4 [v2:172.21.15.148:6800/1403587816,v1:172.21.15.148:6801/1403587816] boot 2024-02-13T14:21:58.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:58 smithi087 ceph-mon[27372]: osd.7 [v2:172.21.15.148:6824/2269337484,v1:172.21.15.148:6825/2269337484] boot 2024-02-13T14:21:58.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:58 smithi087 ceph-mon[27372]: osd.0 [v2:172.21.15.87:6802/2357055010,v1:172.21.15.87:6803/2357055010] boot 2024-02-13T14:21:58.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:58 smithi087 ceph-mon[27372]: osdmap e315: 8 total, 8 up, 8 in 2024-02-13T14:21:58.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:58 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-13T14:21:58.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:58 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-13T14:21:58.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:58 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-13T14:21:58.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:58 smithi087 ceph-mon[27372]: pgmap v426: 196 pgs: 36 undersized+peered, 68 active+undersized, 25 active+undersized+degraded, 6 active+undersized+wait, 6 undersized+peered+wait, 12 undersized+degraded+peered, 3 active+undersized+degraded+wait, 3 undersized+degraded+peered+wait, 1 stale+creating+peering, 3 stale+active+clean, 33 active+clean; 582 KiB data, 337 MiB used, 715 GiB / 715 GiB avail; 217/594 objects degraded (36.532%) 2024-02-13T14:21:58.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:58 smithi087 ceph-mon[27372]: osdmap e316: 8 total, 8 up, 8 in 2024-02-13T14:21:58.743 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:58 smithi087 ceph-mon[39120]: osd.4 [v2:172.21.15.148:6800/1403587816,v1:172.21.15.148:6801/1403587816] boot 2024-02-13T14:21:58.743 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:58 smithi087 ceph-mon[39120]: osd.7 [v2:172.21.15.148:6824/2269337484,v1:172.21.15.148:6825/2269337484] boot 2024-02-13T14:21:58.743 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:58 smithi087 ceph-mon[39120]: osd.0 [v2:172.21.15.87:6802/2357055010,v1:172.21.15.87:6803/2357055010] boot 2024-02-13T14:21:58.743 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:58 smithi087 ceph-mon[39120]: osdmap e315: 8 total, 8 up, 8 in 2024-02-13T14:21:58.743 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:58 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-13T14:21:58.743 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:58 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-13T14:21:58.743 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:58 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-13T14:21:58.743 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:58 smithi087 ceph-mon[39120]: pgmap v426: 196 pgs: 36 undersized+peered, 68 active+undersized, 25 active+undersized+degraded, 6 active+undersized+wait, 6 undersized+peered+wait, 12 undersized+degraded+peered, 3 active+undersized+degraded+wait, 3 undersized+degraded+peered+wait, 1 stale+creating+peering, 3 stale+active+clean, 33 active+clean; 582 KiB data, 337 MiB used, 715 GiB / 715 GiB avail; 217/594 objects degraded (36.532%) 2024-02-13T14:21:58.743 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:58 smithi087 ceph-mon[39120]: osdmap e316: 8 total, 8 up, 8 in 2024-02-13T14:21:58.868 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:58 smithi148 ceph-mon[31198]: osd.4 [v2:172.21.15.148:6800/1403587816,v1:172.21.15.148:6801/1403587816] boot 2024-02-13T14:21:58.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:58 smithi148 ceph-mon[31198]: osd.7 [v2:172.21.15.148:6824/2269337484,v1:172.21.15.148:6825/2269337484] boot 2024-02-13T14:21:58.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:58 smithi148 ceph-mon[31198]: osd.0 [v2:172.21.15.87:6802/2357055010,v1:172.21.15.87:6803/2357055010] boot 2024-02-13T14:21:58.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:58 smithi148 ceph-mon[31198]: osdmap e315: 8 total, 8 up, 8 in 2024-02-13T14:21:58.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:58 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-13T14:21:58.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:58 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-13T14:21:58.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:58 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-13T14:21:58.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:58 smithi148 ceph-mon[31198]: pgmap v426: 196 pgs: 36 undersized+peered, 68 active+undersized, 25 active+undersized+degraded, 6 active+undersized+wait, 6 undersized+peered+wait, 12 undersized+degraded+peered, 3 active+undersized+degraded+wait, 3 undersized+degraded+peered+wait, 1 stale+creating+peering, 3 stale+active+clean, 33 active+clean; 582 KiB data, 337 MiB used, 715 GiB / 715 GiB avail; 217/594 objects degraded (36.532%) 2024-02-13T14:21:58.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:58 smithi148 ceph-mon[31198]: osdmap e316: 8 total, 8 up, 8 in 2024-02-13T14:21:59.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:59 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/3340540793' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:21:59.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:21:59 smithi087 ceph-mon[27372]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:21:59.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:59 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/3340540793' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:21:59.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:21:59 smithi087 ceph-mon[39120]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:21:59.868 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:59 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/3340540793' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:21:59.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:21:59 smithi148 ceph-mon[31198]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:22:00.472 INFO:tasks.workunit.client.0.smithi087.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_aio_read_wait_for_complete_and_cb PASSED [ 77%] 2024-02-13T14:22:00.741 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:00 smithi087 ceph-mon[27372]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:22:00.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:00 smithi087 ceph-mon[27372]: osdmap e317: 8 total, 8 up, 8 in 2024-02-13T14:22:00.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:00 smithi087 ceph-mon[27372]: pgmap v429: 196 pgs: 196 active+clean; 582 KiB data, 325 MiB used, 715 GiB / 715 GiB avail; 105 KiB/s rd, 255 B/s wr, 105 op/s 2024-02-13T14:22:00.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:00 smithi087 ceph-mon[39120]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:22:00.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:00 smithi087 ceph-mon[39120]: osdmap e317: 8 total, 8 up, 8 in 2024-02-13T14:22:00.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:00 smithi087 ceph-mon[39120]: pgmap v429: 196 pgs: 196 active+clean; 582 KiB data, 325 MiB used, 715 GiB / 715 GiB avail; 105 KiB/s rd, 255 B/s wr, 105 op/s 2024-02-13T14:22:00.868 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:00 smithi148 ceph-mon[31198]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:22:00.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:00 smithi148 ceph-mon[31198]: osdmap e317: 8 total, 8 up, 8 in 2024-02-13T14:22:00.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:00 smithi148 ceph-mon[31198]: pgmap v429: 196 pgs: 196 active+clean; 582 KiB data, 325 MiB used, 715 GiB / 715 GiB avail; 105 KiB/s rd, 255 B/s wr, 105 op/s 2024-02-13T14:22:01.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:01 smithi087 ceph-mon[27372]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 6 pgs inactive) 2024-02-13T14:22:01.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:01 smithi087 ceph-mon[27372]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 217/594 objects degraded (36.532%), 43 pgs degraded) 2024-02-13T14:22:01.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:01 smithi087 ceph-mon[27372]: osdmap e318: 8 total, 8 up, 8 in 2024-02-13T14:22:01.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:01 smithi087 ceph-mon[39120]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 6 pgs inactive) 2024-02-13T14:22:01.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:01 smithi087 ceph-mon[39120]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 217/594 objects degraded (36.532%), 43 pgs degraded) 2024-02-13T14:22:01.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:01 smithi087 ceph-mon[39120]: osdmap e318: 8 total, 8 up, 8 in 2024-02-13T14:22:01.868 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:01 smithi148 ceph-mon[31198]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 6 pgs inactive) 2024-02-13T14:22:01.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:01 smithi148 ceph-mon[31198]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 217/594 objects degraded (36.532%), 43 pgs degraded) 2024-02-13T14:22:01.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:01 smithi148 ceph-mon[31198]: osdmap e318: 8 total, 8 up, 8 in 2024-02-13T14:22:02.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:02 smithi087 ceph-mon[39120]: osdmap e319: 8 total, 8 up, 8 in 2024-02-13T14:22:02.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:02 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/855470270' entity='client.admin' cmd=[{"prefix": "osd map", "pool": "test_pool", "object": "bar", "format": "json"}]: dispatch 2024-02-13T14:22:02.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:02 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/855470270' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noup"}]: dispatch 2024-02-13T14:22:02.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:02 smithi087 ceph-mon[39120]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noup"}]: dispatch 2024-02-13T14:22:02.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:02 smithi087 ceph-mon[39120]: pgmap v432: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 325 MiB used, 715 GiB / 715 GiB avail; 104 KiB/s rd, 104 op/s 2024-02-13T14:22:02.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:02 smithi087 ceph-mon[39120]: Health check failed: noup flag(s) set (OSDMAP_FLAGS) 2024-02-13T14:22:02.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:02 smithi087 ceph-mon[39120]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noup"}]': finished 2024-02-13T14:22:02.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:02 smithi087 ceph-mon[39120]: osdmap e320: 8 total, 8 up, 8 in 2024-02-13T14:22:02.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:02 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/855470270' entity='client.admin' cmd=[{"prefix": "osd down", "ids": ["7", "1", "0"]}]: dispatch 2024-02-13T14:22:02.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:02 smithi087 ceph-mon[39120]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd down", "ids": ["7", "1", "0"]}]: dispatch 2024-02-13T14:22:02.743 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:02 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mon-a[27349]: 2024-02-13T14:22:02.470+0000 7f6feb25f700 -1 mon.a@0(leader).osd e320 definitely_dead 0 2024-02-13T14:22:02.743 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:02 smithi087 ceph-mon[27372]: osdmap e319: 8 total, 8 up, 8 in 2024-02-13T14:22:02.743 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:02 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/855470270' entity='client.admin' cmd=[{"prefix": "osd map", "pool": "test_pool", "object": "bar", "format": "json"}]: dispatch 2024-02-13T14:22:02.743 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:02 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/855470270' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noup"}]: dispatch 2024-02-13T14:22:02.743 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:02 smithi087 ceph-mon[27372]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noup"}]: dispatch 2024-02-13T14:22:02.743 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:02 smithi087 ceph-mon[27372]: pgmap v432: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 325 MiB used, 715 GiB / 715 GiB avail; 104 KiB/s rd, 104 op/s 2024-02-13T14:22:02.743 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:02 smithi087 ceph-mon[27372]: Health check failed: noup flag(s) set (OSDMAP_FLAGS) 2024-02-13T14:22:02.743 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:02 smithi087 ceph-mon[27372]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noup"}]': finished 2024-02-13T14:22:02.743 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:02 smithi087 ceph-mon[27372]: osdmap e320: 8 total, 8 up, 8 in 2024-02-13T14:22:02.743 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:02 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/855470270' entity='client.admin' cmd=[{"prefix": "osd down", "ids": ["7", "1", "0"]}]: dispatch 2024-02-13T14:22:02.743 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:02 smithi087 ceph-mon[27372]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd down", "ids": ["7", "1", "0"]}]: dispatch 2024-02-13T14:22:02.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:02 smithi148 ceph-mon[31198]: osdmap e319: 8 total, 8 up, 8 in 2024-02-13T14:22:02.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:02 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/855470270' entity='client.admin' cmd=[{"prefix": "osd map", "pool": "test_pool", "object": "bar", "format": "json"}]: dispatch 2024-02-13T14:22:02.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:02 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/855470270' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noup"}]: dispatch 2024-02-13T14:22:02.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:02 smithi148 ceph-mon[31198]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noup"}]: dispatch 2024-02-13T14:22:02.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:02 smithi148 ceph-mon[31198]: pgmap v432: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 325 MiB used, 715 GiB / 715 GiB avail; 104 KiB/s rd, 104 op/s 2024-02-13T14:22:02.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:02 smithi148 ceph-mon[31198]: Health check failed: noup flag(s) set (OSDMAP_FLAGS) 2024-02-13T14:22:02.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:02 smithi148 ceph-mon[31198]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noup"}]': finished 2024-02-13T14:22:02.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:02 smithi148 ceph-mon[31198]: osdmap e320: 8 total, 8 up, 8 in 2024-02-13T14:22:02.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:02 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/855470270' entity='client.admin' cmd=[{"prefix": "osd down", "ids": ["7", "1", "0"]}]: dispatch 2024-02-13T14:22:02.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:02 smithi148 ceph-mon[31198]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd down", "ids": ["7", "1", "0"]}]: dispatch 2024-02-13T14:22:03.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:03 smithi087 ceph-mon[27372]: Health check failed: 3 osds down (OSD_DOWN) 2024-02-13T14:22:03.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:03 smithi087 ceph-mon[27372]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd down", "ids": ["7", "1", "0"]}]': finished 2024-02-13T14:22:03.742 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:03 smithi087 ceph-mon[27372]: osdmap e321: 8 total, 5 up, 8 in 2024-02-13T14:22:03.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:03 smithi087 ceph-mon[39120]: Health check failed: 3 osds down (OSD_DOWN) 2024-02-13T14:22:03.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:03 smithi087 ceph-mon[39120]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd down", "ids": ["7", "1", "0"]}]': finished 2024-02-13T14:22:03.742 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:03 smithi087 ceph-mon[39120]: osdmap e321: 8 total, 5 up, 8 in 2024-02-13T14:22:03.868 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:03 smithi148 ceph-mon[31198]: Health check failed: 3 osds down (OSD_DOWN) 2024-02-13T14:22:03.868 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:03 smithi148 ceph-mon[31198]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd down", "ids": ["7", "1", "0"]}]': finished 2024-02-13T14:22:03.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:03 smithi148 ceph-mon[31198]: osdmap e321: 8 total, 5 up, 8 in 2024-02-13T14:22:04.868 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:04 smithi148 ceph-mon[31198]: pgmap v435: 196 pgs: 65 stale+active+clean, 32 unknown, 99 active+clean; 582 KiB data, 325 MiB used, 715 GiB / 715 GiB avail 2024-02-13T14:22:04.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:04 smithi148 ceph-mon[31198]: osdmap e322: 8 total, 5 up, 8 in 2024-02-13T14:22:04.991 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:04 smithi087 ceph-mon[27372]: pgmap v435: 196 pgs: 65 stale+active+clean, 32 unknown, 99 active+clean; 582 KiB data, 325 MiB used, 715 GiB / 715 GiB avail 2024-02-13T14:22:04.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:04 smithi087 ceph-mon[27372]: osdmap e322: 8 total, 5 up, 8 in 2024-02-13T14:22:04.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:04 smithi087 ceph-mon[39120]: pgmap v435: 196 pgs: 65 stale+active+clean, 32 unknown, 99 active+clean; 582 KiB data, 325 MiB used, 715 GiB / 715 GiB avail 2024-02-13T14:22:04.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:04 smithi087 ceph-mon[39120]: osdmap e322: 8 total, 5 up, 8 in 2024-02-13T14:22:05.371 INFO:journalctl@ceph.iscsi.iscsi.a.smithi148.stdout:Feb 13 14:22:05 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-iscsi-iscsi-a[59644]: debug there is no tcmu-runner data available 2024-02-13T14:22:05.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:05 smithi148 ceph-mon[31198]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:22:05.991 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:05 smithi087 ceph-mon[39120]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:22:05.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:05 smithi087 ceph-mon[27372]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:22:05.992 INFO:journalctl@ceph.osd.1.smithi087.stdout:Feb 13 14:22:05 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-1[49621]: 2024-02-13T14:22:05.669+0000 7f213c6fe700 -1 osd.1 322 osdmap NOUP flag is set, waiting for it to clear 2024-02-13T14:22:06.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:06 smithi148 ceph-mon[31198]: osd.1 marked itself dead as of e322 2024-02-13T14:22:06.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:06 smithi148 ceph-mon[31198]: pgmap v437: 196 pgs: 12 undersized+degraded+peered+wait, 30 active+undersized+degraded+wait, 35 undersized+peered+wait, 64 active+undersized+wait, 55 active+clean; 582 KiB data, 334 MiB used, 715 GiB / 715 GiB avail; 205/594 objects degraded (34.512%) 2024-02-13T14:22:06.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:06 smithi148 ceph-mon[31198]: osd.0 marked itself dead as of e322 2024-02-13T14:22:06.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:06 smithi148 ceph-mon[31198]: osd.7 marked itself dead as of e322 2024-02-13T14:22:06.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:06 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/855470270' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:22:06.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:06 smithi148 ceph-mon[31198]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:22:06.992 INFO:journalctl@ceph.osd.1.smithi087.stdout:Feb 13 14:22:06 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-1[49621]: 2024-02-13T14:22:06.517+0000 7f2137cf0700 -1 osd.1 323 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-13T14:22:06.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:06 smithi087 ceph-mon[27372]: osd.1 marked itself dead as of e322 2024-02-13T14:22:06.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:06 smithi087 ceph-mon[27372]: pgmap v437: 196 pgs: 12 undersized+degraded+peered+wait, 30 active+undersized+degraded+wait, 35 undersized+peered+wait, 64 active+undersized+wait, 55 active+clean; 582 KiB data, 334 MiB used, 715 GiB / 715 GiB avail; 205/594 objects degraded (34.512%) 2024-02-13T14:22:06.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:06 smithi087 ceph-mon[27372]: osd.0 marked itself dead as of e322 2024-02-13T14:22:06.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:06 smithi087 ceph-mon[27372]: osd.7 marked itself dead as of e322 2024-02-13T14:22:06.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:06 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/855470270' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:22:06.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:06 smithi087 ceph-mon[27372]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:22:06.993 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:06 smithi087 ceph-mon[39120]: osd.1 marked itself dead as of e322 2024-02-13T14:22:06.993 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:06 smithi087 ceph-mon[39120]: pgmap v437: 196 pgs: 12 undersized+degraded+peered+wait, 30 active+undersized+degraded+wait, 35 undersized+peered+wait, 64 active+undersized+wait, 55 active+clean; 582 KiB data, 334 MiB used, 715 GiB / 715 GiB avail; 205/594 objects degraded (34.512%) 2024-02-13T14:22:06.993 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:06 smithi087 ceph-mon[39120]: osd.0 marked itself dead as of e322 2024-02-13T14:22:06.993 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:06 smithi087 ceph-mon[39120]: osd.7 marked itself dead as of e322 2024-02-13T14:22:06.993 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:06 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/855470270' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:22:06.993 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:06 smithi087 ceph-mon[39120]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:22:07.492 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:22:07 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: ::ffff:172.21.15.148 - - [13/Feb/2024:14:22:07] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-13T14:22:07.492 INFO:journalctl@ceph.osd.0.smithi087.stdout:Feb 13 14:22:07 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-0[43500]: 2024-02-13T14:22:07.414+0000 7f0a8dedb700 -1 osd.0 323 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-13T14:22:07.504 INFO:journalctl@ceph.osd.7.smithi148.stdout:Feb 13 14:22:07 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-7[54686]: 2024-02-13T14:22:07.150+0000 7f4023e16700 -1 osd.7 323 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-13T14:22:07.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:07 smithi148 ceph-mon[31198]: Monitor daemon marked osd.1 down, but it is still running 2024-02-13T14:22:07.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:07 smithi148 ceph-mon[31198]: map e322 wrongly marked me down at e321 2024-02-13T14:22:07.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:07 smithi148 ceph-mon[31198]: Monitor daemon marked osd.0 down, but it is still running 2024-02-13T14:22:07.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:07 smithi148 ceph-mon[31198]: map e322 wrongly marked me down at e321 2024-02-13T14:22:07.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:07 smithi148 ceph-mon[31198]: Monitor daemon marked osd.7 down, but it is still running 2024-02-13T14:22:07.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:07 smithi148 ceph-mon[31198]: map e322 wrongly marked me down at e321 2024-02-13T14:22:07.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:07 smithi148 ceph-mon[31198]: Health check failed: Degraded data redundancy: 205/594 objects degraded (34.512%), 42 pgs degraded (PG_DEGRADED) 2024-02-13T14:22:07.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:07 smithi148 ceph-mon[31198]: Health check cleared: OSDMAP_FLAGS (was: noup flag(s) set) 2024-02-13T14:22:07.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:07 smithi148 ceph-mon[31198]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:22:07.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:07 smithi148 ceph-mon[31198]: osdmap e323: 8 total, 5 up, 8 in 2024-02-13T14:22:07.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:07 smithi087 ceph-mon[27372]: Monitor daemon marked osd.1 down, but it is still running 2024-02-13T14:22:07.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:07 smithi087 ceph-mon[27372]: map e322 wrongly marked me down at e321 2024-02-13T14:22:07.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:07 smithi087 ceph-mon[27372]: Monitor daemon marked osd.0 down, but it is still running 2024-02-13T14:22:07.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:07 smithi087 ceph-mon[27372]: map e322 wrongly marked me down at e321 2024-02-13T14:22:07.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:07 smithi087 ceph-mon[27372]: Monitor daemon marked osd.7 down, but it is still running 2024-02-13T14:22:07.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:07 smithi087 ceph-mon[27372]: map e322 wrongly marked me down at e321 2024-02-13T14:22:07.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:07 smithi087 ceph-mon[27372]: Health check failed: Degraded data redundancy: 205/594 objects degraded (34.512%), 42 pgs degraded (PG_DEGRADED) 2024-02-13T14:22:07.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:07 smithi087 ceph-mon[27372]: Health check cleared: OSDMAP_FLAGS (was: noup flag(s) set) 2024-02-13T14:22:07.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:07 smithi087 ceph-mon[27372]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:22:07.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:07 smithi087 ceph-mon[27372]: osdmap e323: 8 total, 5 up, 8 in 2024-02-13T14:22:07.993 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:07 smithi087 ceph-mon[39120]: Monitor daemon marked osd.1 down, but it is still running 2024-02-13T14:22:07.993 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:07 smithi087 ceph-mon[39120]: map e322 wrongly marked me down at e321 2024-02-13T14:22:07.993 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:07 smithi087 ceph-mon[39120]: Monitor daemon marked osd.0 down, but it is still running 2024-02-13T14:22:07.993 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:07 smithi087 ceph-mon[39120]: map e322 wrongly marked me down at e321 2024-02-13T14:22:07.993 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:07 smithi087 ceph-mon[39120]: Monitor daemon marked osd.7 down, but it is still running 2024-02-13T14:22:07.993 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:07 smithi087 ceph-mon[39120]: map e322 wrongly marked me down at e321 2024-02-13T14:22:07.993 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:07 smithi087 ceph-mon[39120]: Health check failed: Degraded data redundancy: 205/594 objects degraded (34.512%), 42 pgs degraded (PG_DEGRADED) 2024-02-13T14:22:07.993 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:07 smithi087 ceph-mon[39120]: Health check cleared: OSDMAP_FLAGS (was: noup flag(s) set) 2024-02-13T14:22:07.993 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:07 smithi087 ceph-mon[39120]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:22:07.993 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:07 smithi087 ceph-mon[39120]: osdmap e323: 8 total, 5 up, 8 in 2024-02-13T14:22:08.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:08 smithi148 ceph-mon[31198]: Health check cleared: OSD_DOWN (was: 3 osds down) 2024-02-13T14:22:08.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:08 smithi148 ceph-mon[31198]: osd.1 [v2:172.21.15.87:6810/1433765078,v1:172.21.15.87:6811/1433765078] boot 2024-02-13T14:22:08.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:08 smithi148 ceph-mon[31198]: osd.7 [v2:172.21.15.148:6824/2269337484,v1:172.21.15.148:6825/2269337484] boot 2024-02-13T14:22:08.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:08 smithi148 ceph-mon[31198]: osd.0 [v2:172.21.15.87:6802/2357055010,v1:172.21.15.87:6803/2357055010] boot 2024-02-13T14:22:08.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:08 smithi148 ceph-mon[31198]: osdmap e324: 8 total, 8 up, 8 in 2024-02-13T14:22:08.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:08 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-13T14:22:08.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:08 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-13T14:22:08.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:08 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-13T14:22:08.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:08 smithi148 ceph-mon[31198]: pgmap v440: 196 pgs: 29 stale+active+clean, 12 undersized+degraded+peered+wait, 30 active+undersized+degraded+wait, 35 undersized+peered+wait, 64 active+undersized+wait, 26 active+clean; 582 KiB data, 334 MiB used, 715 GiB / 715 GiB avail; 205/594 objects degraded (34.512%) 2024-02-13T14:22:08.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:08 smithi087 ceph-mon[27372]: Health check cleared: OSD_DOWN (was: 3 osds down) 2024-02-13T14:22:08.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:08 smithi087 ceph-mon[27372]: osd.1 [v2:172.21.15.87:6810/1433765078,v1:172.21.15.87:6811/1433765078] boot 2024-02-13T14:22:08.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:08 smithi087 ceph-mon[27372]: osd.7 [v2:172.21.15.148:6824/2269337484,v1:172.21.15.148:6825/2269337484] boot 2024-02-13T14:22:08.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:08 smithi087 ceph-mon[27372]: osd.0 [v2:172.21.15.87:6802/2357055010,v1:172.21.15.87:6803/2357055010] boot 2024-02-13T14:22:08.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:08 smithi087 ceph-mon[27372]: osdmap e324: 8 total, 8 up, 8 in 2024-02-13T14:22:08.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:08 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-13T14:22:08.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:08 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-13T14:22:08.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:08 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-13T14:22:08.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:08 smithi087 ceph-mon[27372]: pgmap v440: 196 pgs: 29 stale+active+clean, 12 undersized+degraded+peered+wait, 30 active+undersized+degraded+wait, 35 undersized+peered+wait, 64 active+undersized+wait, 26 active+clean; 582 KiB data, 334 MiB used, 715 GiB / 715 GiB avail; 205/594 objects degraded (34.512%) 2024-02-13T14:22:08.993 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:08 smithi087 ceph-mon[39120]: Health check cleared: OSD_DOWN (was: 3 osds down) 2024-02-13T14:22:08.993 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:08 smithi087 ceph-mon[39120]: osd.1 [v2:172.21.15.87:6810/1433765078,v1:172.21.15.87:6811/1433765078] boot 2024-02-13T14:22:08.993 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:08 smithi087 ceph-mon[39120]: osd.7 [v2:172.21.15.148:6824/2269337484,v1:172.21.15.148:6825/2269337484] boot 2024-02-13T14:22:08.993 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:08 smithi087 ceph-mon[39120]: osd.0 [v2:172.21.15.87:6802/2357055010,v1:172.21.15.87:6803/2357055010] boot 2024-02-13T14:22:08.993 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:08 smithi087 ceph-mon[39120]: osdmap e324: 8 total, 8 up, 8 in 2024-02-13T14:22:08.993 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:08 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-13T14:22:08.993 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:08 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-13T14:22:08.993 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:08 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-13T14:22:08.993 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:08 smithi087 ceph-mon[39120]: pgmap v440: 196 pgs: 29 stale+active+clean, 12 undersized+degraded+peered+wait, 30 active+undersized+degraded+wait, 35 undersized+peered+wait, 64 active+undersized+wait, 26 active+clean; 582 KiB data, 334 MiB used, 715 GiB / 715 GiB avail; 205/594 objects degraded (34.512%) 2024-02-13T14:22:09.868 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:09 smithi148 ceph-mon[31198]: osdmap e325: 8 total, 8 up, 8 in 2024-02-13T14:22:09.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:09 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:22:09.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:09 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-13T14:22:09.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:09 smithi087 ceph-mon[27372]: osdmap e325: 8 total, 8 up, 8 in 2024-02-13T14:22:09.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:09 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:22:09.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:09 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-13T14:22:09.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:09 smithi087 ceph-mon[39120]: osdmap e325: 8 total, 8 up, 8 in 2024-02-13T14:22:09.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:09 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:22:09.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:09 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-13T14:22:10.870 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:10 smithi148 ceph-mon[31198]: pgmap v442: 196 pgs: 196 active+clean; 582 KiB data, 302 MiB used, 715 GiB / 715 GiB avail; 953 B/s rd, 0 op/s 2024-02-13T14:22:10.870 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:10 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/855470270' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:22:10.870 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:10 smithi148 ceph-mon[31198]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:22:10.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:10 smithi087 ceph-mon[27372]: pgmap v442: 196 pgs: 196 active+clean; 582 KiB data, 302 MiB used, 715 GiB / 715 GiB avail; 953 B/s rd, 0 op/s 2024-02-13T14:22:10.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:10 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/855470270' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:22:10.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:10 smithi087 ceph-mon[27372]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:22:10.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:10 smithi087 ceph-mon[39120]: pgmap v442: 196 pgs: 196 active+clean; 582 KiB data, 302 MiB used, 715 GiB / 715 GiB avail; 953 B/s rd, 0 op/s 2024-02-13T14:22:10.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:10 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/855470270' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:22:10.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:10 smithi087 ceph-mon[39120]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:22:11.544 INFO:tasks.workunit.client.0.smithi087.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_aio_read_wait_for_complete_and_cb_error PASSED [ 78%] 2024-02-13T14:22:11.868 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:11 smithi148 ceph-mon[31198]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 205/594 objects degraded (34.512%), 42 pgs degraded) 2024-02-13T14:22:11.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:11 smithi148 ceph-mon[31198]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:22:11.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:11 smithi148 ceph-mon[31198]: osdmap e326: 8 total, 8 up, 8 in 2024-02-13T14:22:11.991 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:11 smithi087 ceph-mon[27372]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 205/594 objects degraded (34.512%), 42 pgs degraded) 2024-02-13T14:22:11.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:11 smithi087 ceph-mon[27372]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:22:11.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:11 smithi087 ceph-mon[27372]: osdmap e326: 8 total, 8 up, 8 in 2024-02-13T14:22:11.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:11 smithi087 ceph-mon[39120]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 205/594 objects degraded (34.512%), 42 pgs degraded) 2024-02-13T14:22:11.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:11 smithi087 ceph-mon[39120]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:22:11.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:11 smithi087 ceph-mon[39120]: osdmap e326: 8 total, 8 up, 8 in 2024-02-13T14:22:12.868 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:12 smithi148 ceph-mon[31198]: osdmap e327: 8 total, 8 up, 8 in 2024-02-13T14:22:12.868 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:12 smithi148 ceph-mon[31198]: pgmap v445: 164 pgs: 164 active+clean; 582 KiB data, 302 MiB used, 715 GiB / 715 GiB avail; 7.2 KiB/s rd, 7 op/s 2024-02-13T14:22:12.991 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:12 smithi087 ceph-mon[27372]: osdmap e327: 8 total, 8 up, 8 in 2024-02-13T14:22:12.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:12 smithi087 ceph-mon[27372]: pgmap v445: 164 pgs: 164 active+clean; 582 KiB data, 302 MiB used, 715 GiB / 715 GiB avail; 7.2 KiB/s rd, 7 op/s 2024-02-13T14:22:12.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:12 smithi087 ceph-mon[39120]: osdmap e327: 8 total, 8 up, 8 in 2024-02-13T14:22:12.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:12 smithi087 ceph-mon[39120]: pgmap v445: 164 pgs: 164 active+clean; 582 KiB data, 302 MiB used, 715 GiB / 715 GiB avail; 7.2 KiB/s rd, 7 op/s 2024-02-13T14:22:13.868 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:13 smithi148 ceph-mon[31198]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T14:22:13.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:13 smithi148 ceph-mon[31198]: osdmap e328: 8 total, 8 up, 8 in 2024-02-13T14:22:13.991 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:13 smithi087 ceph-mon[27372]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T14:22:13.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:13 smithi087 ceph-mon[27372]: osdmap e328: 8 total, 8 up, 8 in 2024-02-13T14:22:13.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:13 smithi087 ceph-mon[39120]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T14:22:13.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:13 smithi087 ceph-mon[39120]: osdmap e328: 8 total, 8 up, 8 in 2024-02-13T14:22:14.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:14 smithi148 ceph-mon[31198]: osdmap e329: 8 total, 8 up, 8 in 2024-02-13T14:22:14.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:14 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/3885127328' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:22:14.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:14 smithi148 ceph-mon[31198]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:22:14.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:14 smithi148 ceph-mon[31198]: pgmap v448: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 302 MiB used, 715 GiB / 715 GiB avail 2024-02-13T14:22:14.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:14 smithi087 ceph-mon[27372]: osdmap e329: 8 total, 8 up, 8 in 2024-02-13T14:22:14.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:14 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/3885127328' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:22:14.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:14 smithi087 ceph-mon[27372]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:22:14.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:14 smithi087 ceph-mon[27372]: pgmap v448: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 302 MiB used, 715 GiB / 715 GiB avail 2024-02-13T14:22:14.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:14 smithi087 ceph-mon[39120]: osdmap e329: 8 total, 8 up, 8 in 2024-02-13T14:22:14.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:14 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/3885127328' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:22:14.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:14 smithi087 ceph-mon[39120]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:22:14.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:14 smithi087 ceph-mon[39120]: pgmap v448: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 302 MiB used, 715 GiB / 715 GiB avail 2024-02-13T14:22:15.368 INFO:journalctl@ceph.iscsi.iscsi.a.smithi148.stdout:Feb 13 14:22:15 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-iscsi-iscsi-a[59644]: debug there is no tcmu-runner data available 2024-02-13T14:22:15.571 INFO:tasks.workunit.client.0.smithi087.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_lock PASSED [ 80%] 2024-02-13T14:22:15.868 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:15 smithi148 ceph-mon[31198]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:22:15.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:15 smithi148 ceph-mon[31198]: osdmap e330: 8 total, 8 up, 8 in 2024-02-13T14:22:15.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:15 smithi148 ceph-mon[31198]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:22:15.991 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:15 smithi087 ceph-mon[27372]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:22:15.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:15 smithi087 ceph-mon[27372]: osdmap e330: 8 total, 8 up, 8 in 2024-02-13T14:22:15.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:15 smithi087 ceph-mon[27372]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:22:15.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:15 smithi087 ceph-mon[39120]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:22:15.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:15 smithi087 ceph-mon[39120]: osdmap e330: 8 total, 8 up, 8 in 2024-02-13T14:22:15.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:15 smithi087 ceph-mon[39120]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:22:16.868 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:16 smithi148 ceph-mon[31198]: osdmap e331: 8 total, 8 up, 8 in 2024-02-13T14:22:16.868 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:16 smithi148 ceph-mon[31198]: pgmap v451: 164 pgs: 164 active+clean; 582 KiB data, 302 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:22:16.991 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:16 smithi087 ceph-mon[39120]: osdmap e331: 8 total, 8 up, 8 in 2024-02-13T14:22:16.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:16 smithi087 ceph-mon[39120]: pgmap v451: 164 pgs: 164 active+clean; 582 KiB data, 302 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:22:16.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:16 smithi087 ceph-mon[27372]: osdmap e331: 8 total, 8 up, 8 in 2024-02-13T14:22:16.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:16 smithi087 ceph-mon[27372]: pgmap v451: 164 pgs: 164 active+clean; 582 KiB data, 302 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:22:17.491 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:22:17 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: ::ffff:172.21.15.148 - - [13/Feb/2024:14:22:17] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-13T14:22:17.868 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:17 smithi148 ceph-mon[31198]: osdmap e332: 8 total, 8 up, 8 in 2024-02-13T14:22:17.991 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:17 smithi087 ceph-mon[27372]: osdmap e332: 8 total, 8 up, 8 in 2024-02-13T14:22:17.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:17 smithi087 ceph-mon[39120]: osdmap e332: 8 total, 8 up, 8 in 2024-02-13T14:22:18.868 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:18 smithi148 ceph-mon[31198]: osdmap e333: 8 total, 8 up, 8 in 2024-02-13T14:22:18.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:18 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/1413587702' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:22:18.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:18 smithi148 ceph-mon[31198]: pgmap v454: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 302 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:22:18.991 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:18 smithi087 ceph-mon[39120]: osdmap e333: 8 total, 8 up, 8 in 2024-02-13T14:22:18.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:18 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/1413587702' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:22:18.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:18 smithi087 ceph-mon[39120]: pgmap v454: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 302 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:22:18.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:18 smithi087 ceph-mon[27372]: osdmap e333: 8 total, 8 up, 8 in 2024-02-13T14:22:18.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:18 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/1413587702' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:22:18.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:18 smithi087 ceph-mon[27372]: pgmap v454: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 302 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:22:19.601 INFO:tasks.workunit.client.0.smithi087.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_execute PASSED [ 81%] 2024-02-13T14:22:19.868 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:19 smithi148 ceph-mon[31198]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T14:22:19.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:19 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/1413587702' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:22:19.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:19 smithi148 ceph-mon[31198]: osdmap e334: 8 total, 8 up, 8 in 2024-02-13T14:22:19.991 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:19 smithi087 ceph-mon[27372]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T14:22:19.991 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:19 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/1413587702' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:22:19.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:19 smithi087 ceph-mon[27372]: osdmap e334: 8 total, 8 up, 8 in 2024-02-13T14:22:19.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:19 smithi087 ceph-mon[39120]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T14:22:19.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:19 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/1413587702' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:22:19.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:19 smithi087 ceph-mon[39120]: osdmap e334: 8 total, 8 up, 8 in 2024-02-13T14:22:20.871 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:20 smithi148 ceph-mon[31198]: osdmap e335: 8 total, 8 up, 8 in 2024-02-13T14:22:20.871 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:20 smithi148 ceph-mon[31198]: pgmap v457: 164 pgs: 164 active+clean; 582 KiB data, 290 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:22:20.991 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:20 smithi087 ceph-mon[27372]: osdmap e335: 8 total, 8 up, 8 in 2024-02-13T14:22:20.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:20 smithi087 ceph-mon[27372]: pgmap v457: 164 pgs: 164 active+clean; 582 KiB data, 290 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:22:20.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:20 smithi087 ceph-mon[39120]: osdmap e335: 8 total, 8 up, 8 in 2024-02-13T14:22:20.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:20 smithi087 ceph-mon[39120]: pgmap v457: 164 pgs: 164 active+clean; 582 KiB data, 290 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:22:21.893 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:21 smithi087 ceph-mon[27372]: osdmap e336: 8 total, 8 up, 8 in 2024-02-13T14:22:21.893 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:21 smithi087 ceph-mon[39120]: osdmap e336: 8 total, 8 up, 8 in 2024-02-13T14:22:22.118 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:21 smithi148 ceph-mon[31198]: osdmap e336: 8 total, 8 up, 8 in 2024-02-13T14:22:22.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:22 smithi087 ceph-mon[27372]: osdmap e337: 8 total, 8 up, 8 in 2024-02-13T14:22:22.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:22 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/1811881760' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:22:22.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:22 smithi087 ceph-mon[27372]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:22:22.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:22 smithi087 ceph-mon[27372]: pgmap v460: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 290 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:22:22.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:22 smithi087 ceph-mon[27372]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:22:22.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:22 smithi087 ceph-mon[27372]: osdmap e338: 8 total, 8 up, 8 in 2024-02-13T14:22:22.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:22 smithi087 ceph-mon[39120]: osdmap e337: 8 total, 8 up, 8 in 2024-02-13T14:22:22.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:22 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/1811881760' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:22:22.993 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:22 smithi087 ceph-mon[39120]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:22:22.993 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:22 smithi087 ceph-mon[39120]: pgmap v460: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 290 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:22:22.993 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:22 smithi087 ceph-mon[39120]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:22:22.993 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:22 smithi087 ceph-mon[39120]: osdmap e338: 8 total, 8 up, 8 in 2024-02-13T14:22:23.118 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:22 smithi148 ceph-mon[31198]: osdmap e337: 8 total, 8 up, 8 in 2024-02-13T14:22:23.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:22 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/1811881760' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:22:23.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:22 smithi148 ceph-mon[31198]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:22:23.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:22 smithi148 ceph-mon[31198]: pgmap v460: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 290 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:22:23.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:22 smithi148 ceph-mon[31198]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:22:23.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:22 smithi148 ceph-mon[31198]: osdmap e338: 8 total, 8 up, 8 in 2024-02-13T14:22:23.624 INFO:tasks.workunit.client.0.smithi087.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_aio_execute PASSED [ 82%] 2024-02-13T14:22:24.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:24 smithi087 ceph-mon[27372]: osdmap e339: 8 total, 8 up, 8 in 2024-02-13T14:22:24.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:24 smithi087 ceph-mon[27372]: pgmap v463: 164 pgs: 164 active+clean; 582 KiB data, 290 MiB used, 715 GiB / 715 GiB avail 2024-02-13T14:22:24.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:24 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:22:24.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:24 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-13T14:22:24.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:24 smithi087 ceph-mon[39120]: osdmap e339: 8 total, 8 up, 8 in 2024-02-13T14:22:24.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:24 smithi087 ceph-mon[39120]: pgmap v463: 164 pgs: 164 active+clean; 582 KiB data, 290 MiB used, 715 GiB / 715 GiB avail 2024-02-13T14:22:24.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:24 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:22:24.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:24 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-13T14:22:25.056 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:24 smithi148 ceph-mon[31198]: osdmap e339: 8 total, 8 up, 8 in 2024-02-13T14:22:25.056 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:24 smithi148 ceph-mon[31198]: pgmap v463: 164 pgs: 164 active+clean; 582 KiB data, 290 MiB used, 715 GiB / 715 GiB avail 2024-02-13T14:22:25.056 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:24 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:22:25.056 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:24 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-13T14:22:25.368 INFO:journalctl@ceph.iscsi.iscsi.a.smithi148.stdout:Feb 13 14:22:25 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-iscsi-iscsi-a[59644]: debug there is no tcmu-runner data available 2024-02-13T14:22:25.991 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:25 smithi087 ceph-mon[39120]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T14:22:25.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:25 smithi087 ceph-mon[39120]: osdmap e340: 8 total, 8 up, 8 in 2024-02-13T14:22:25.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:25 smithi087 ceph-mon[39120]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:22:25.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:25 smithi087 ceph-mon[27372]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T14:22:25.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:25 smithi087 ceph-mon[27372]: osdmap e340: 8 total, 8 up, 8 in 2024-02-13T14:22:25.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:25 smithi087 ceph-mon[27372]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:22:26.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:25 smithi148 ceph-mon[31198]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T14:22:26.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:25 smithi148 ceph-mon[31198]: osdmap e340: 8 total, 8 up, 8 in 2024-02-13T14:22:26.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:25 smithi148 ceph-mon[31198]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:22:26.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:26 smithi087 ceph-mon[39120]: osdmap e341: 8 total, 8 up, 8 in 2024-02-13T14:22:26.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:26 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/1993043347' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:22:26.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:26 smithi087 ceph-mon[39120]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:22:26.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:26 smithi087 ceph-mon[39120]: pgmap v466: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 291 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:22:26.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:26 smithi087 ceph-mon[27372]: osdmap e341: 8 total, 8 up, 8 in 2024-02-13T14:22:26.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:26 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/1993043347' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:22:26.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:26 smithi087 ceph-mon[27372]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:22:26.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:26 smithi087 ceph-mon[27372]: pgmap v466: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 291 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:22:27.118 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:26 smithi148 ceph-mon[31198]: osdmap e341: 8 total, 8 up, 8 in 2024-02-13T14:22:27.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:26 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/1993043347' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:22:27.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:26 smithi148 ceph-mon[31198]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:22:27.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:26 smithi148 ceph-mon[31198]: pgmap v466: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 291 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:22:27.491 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:22:27 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: ::ffff:172.21.15.148 - - [13/Feb/2024:14:22:27] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-13T14:22:27.644 INFO:tasks.workunit.client.0.smithi087.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_aio_setxattr PASSED [ 83%] 2024-02-13T14:22:27.991 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:27 smithi087 ceph-mon[39120]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:22:27.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:27 smithi087 ceph-mon[39120]: osdmap e342: 8 total, 8 up, 8 in 2024-02-13T14:22:27.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:27 smithi087 ceph-mon[27372]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:22:27.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:27 smithi087 ceph-mon[27372]: osdmap e342: 8 total, 8 up, 8 in 2024-02-13T14:22:28.118 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:27 smithi148 ceph-mon[31198]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:22:28.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:27 smithi148 ceph-mon[31198]: osdmap e342: 8 total, 8 up, 8 in 2024-02-13T14:22:28.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:28 smithi087 ceph-mon[39120]: osdmap e343: 8 total, 8 up, 8 in 2024-02-13T14:22:28.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:28 smithi087 ceph-mon[39120]: pgmap v469: 164 pgs: 164 active+clean; 582 KiB data, 291 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:22:28.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:28 smithi087 ceph-mon[27372]: osdmap e343: 8 total, 8 up, 8 in 2024-02-13T14:22:28.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:28 smithi087 ceph-mon[27372]: pgmap v469: 164 pgs: 164 active+clean; 582 KiB data, 291 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:22:29.118 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:28 smithi148 ceph-mon[31198]: osdmap e343: 8 total, 8 up, 8 in 2024-02-13T14:22:29.118 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:28 smithi148 ceph-mon[31198]: pgmap v469: 164 pgs: 164 active+clean; 582 KiB data, 291 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:22:29.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:29 smithi087 ceph-mon[27372]: osdmap e344: 8 total, 8 up, 8 in 2024-02-13T14:22:29.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:29 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/2994136514' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-13T14:22:29.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:29 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/2994136514' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test_pool","app": "app1"}]: dispatch 2024-02-13T14:22:29.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:29 smithi087 ceph-mon[27372]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test_pool","app": "app1"}]: dispatch 2024-02-13T14:22:29.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:29 smithi087 ceph-mon[39120]: osdmap e344: 8 total, 8 up, 8 in 2024-02-13T14:22:29.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:29 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/2994136514' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-13T14:22:29.993 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:29 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/2994136514' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test_pool","app": "app1"}]: dispatch 2024-02-13T14:22:29.993 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:29 smithi087 ceph-mon[39120]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test_pool","app": "app1"}]: dispatch 2024-02-13T14:22:30.118 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:29 smithi148 ceph-mon[31198]: osdmap e344: 8 total, 8 up, 8 in 2024-02-13T14:22:30.118 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:29 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/2994136514' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-13T14:22:30.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:29 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/2994136514' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test_pool","app": "app1"}]: dispatch 2024-02-13T14:22:30.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:29 smithi148 ceph-mon[31198]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test_pool","app": "app1"}]: dispatch 2024-02-13T14:22:30.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:30 smithi087 ceph-mon[27372]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test_pool","app": "app1"}]': finished 2024-02-13T14:22:30.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:30 smithi087 ceph-mon[27372]: osdmap e345: 8 total, 8 up, 8 in 2024-02-13T14:22:30.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:30 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/2994136514' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test_pool","app": "app2"}]: dispatch 2024-02-13T14:22:30.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:30 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/2994136514' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test_pool","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-02-13T14:22:30.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:30 smithi087 ceph-mon[27372]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test_pool","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-02-13T14:22:30.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:30 smithi087 ceph-mon[27372]: pgmap v472: 196 pgs: 196 active+clean; 582 KiB data, 292 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:22:30.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:30 smithi087 ceph-mon[39120]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test_pool","app": "app1"}]': finished 2024-02-13T14:22:30.993 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:30 smithi087 ceph-mon[39120]: osdmap e345: 8 total, 8 up, 8 in 2024-02-13T14:22:30.993 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:30 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/2994136514' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test_pool","app": "app2"}]: dispatch 2024-02-13T14:22:30.993 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:30 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/2994136514' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test_pool","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-02-13T14:22:30.993 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:30 smithi087 ceph-mon[39120]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test_pool","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-02-13T14:22:30.993 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:30 smithi087 ceph-mon[39120]: pgmap v472: 196 pgs: 196 active+clean; 582 KiB data, 292 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:22:31.118 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:30 smithi148 ceph-mon[31198]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test_pool","app": "app1"}]': finished 2024-02-13T14:22:31.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:30 smithi148 ceph-mon[31198]: osdmap e345: 8 total, 8 up, 8 in 2024-02-13T14:22:31.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:30 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/2994136514' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test_pool","app": "app2"}]: dispatch 2024-02-13T14:22:31.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:30 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/2994136514' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test_pool","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-02-13T14:22:31.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:30 smithi148 ceph-mon[31198]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test_pool","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-02-13T14:22:31.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:30 smithi148 ceph-mon[31198]: pgmap v472: 196 pgs: 196 active+clean; 582 KiB data, 292 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:22:31.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:31 smithi087 ceph-mon[27372]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test_pool","app": "app2","yes_i_really_mean_it": true}]': finished 2024-02-13T14:22:31.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:31 smithi087 ceph-mon[27372]: osdmap e346: 8 total, 8 up, 8 in 2024-02-13T14:22:31.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:31 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/2994136514' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"test_pool","app":"dne","key":"key","value":"key"}]: dispatch 2024-02-13T14:22:31.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:31 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/2994136514' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"test_pool","app":"app1","key":"key1","value":"val1"}]: dispatch 2024-02-13T14:22:31.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:31 smithi087 ceph-mon[27372]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"test_pool","app":"app1","key":"key1","value":"val1"}]: dispatch 2024-02-13T14:22:31.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:31 smithi087 ceph-mon[39120]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test_pool","app": "app2","yes_i_really_mean_it": true}]': finished 2024-02-13T14:22:31.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:31 smithi087 ceph-mon[39120]: osdmap e346: 8 total, 8 up, 8 in 2024-02-13T14:22:31.993 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:31 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/2994136514' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"test_pool","app":"dne","key":"key","value":"key"}]: dispatch 2024-02-13T14:22:31.993 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:31 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/2994136514' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"test_pool","app":"app1","key":"key1","value":"val1"}]: dispatch 2024-02-13T14:22:31.993 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:31 smithi087 ceph-mon[39120]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"test_pool","app":"app1","key":"key1","value":"val1"}]: dispatch 2024-02-13T14:22:32.118 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:31 smithi148 ceph-mon[31198]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test_pool","app": "app2","yes_i_really_mean_it": true}]': finished 2024-02-13T14:22:32.118 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:31 smithi148 ceph-mon[31198]: osdmap e346: 8 total, 8 up, 8 in 2024-02-13T14:22:32.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:31 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/2994136514' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"test_pool","app":"dne","key":"key","value":"key"}]: dispatch 2024-02-13T14:22:32.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:31 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/2994136514' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"test_pool","app":"app1","key":"key1","value":"val1"}]: dispatch 2024-02-13T14:22:32.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:31 smithi148 ceph-mon[31198]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"test_pool","app":"app1","key":"key1","value":"val1"}]: dispatch 2024-02-13T14:22:32.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:32 smithi087 ceph-mon[39120]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"test_pool","app":"app1","key":"key1","value":"val1"}]': finished 2024-02-13T14:22:32.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:32 smithi087 ceph-mon[39120]: osdmap e347: 8 total, 8 up, 8 in 2024-02-13T14:22:32.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:32 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/2994136514' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"test_pool","app":"app1","key":"key2","value":"val2"}]: dispatch 2024-02-13T14:22:32.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:32 smithi087 ceph-mon[39120]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"test_pool","app":"app1","key":"key2","value":"val2"}]: dispatch 2024-02-13T14:22:32.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:32 smithi087 ceph-mon[39120]: pgmap v475: 196 pgs: 196 active+clean; 582 KiB data, 292 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:22:32.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:32 smithi087 ceph-mon[39120]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"test_pool","app":"app1","key":"key2","value":"val2"}]': finished 2024-02-13T14:22:32.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:32 smithi087 ceph-mon[39120]: osdmap e348: 8 total, 8 up, 8 in 2024-02-13T14:22:32.993 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:32 smithi087 ceph-mon[27372]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"test_pool","app":"app1","key":"key1","value":"val1"}]': finished 2024-02-13T14:22:32.993 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:32 smithi087 ceph-mon[27372]: osdmap e347: 8 total, 8 up, 8 in 2024-02-13T14:22:32.993 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:32 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/2994136514' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"test_pool","app":"app1","key":"key2","value":"val2"}]: dispatch 2024-02-13T14:22:32.993 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:32 smithi087 ceph-mon[27372]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"test_pool","app":"app1","key":"key2","value":"val2"}]: dispatch 2024-02-13T14:22:32.993 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:32 smithi087 ceph-mon[27372]: pgmap v475: 196 pgs: 196 active+clean; 582 KiB data, 292 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:22:32.993 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:32 smithi087 ceph-mon[27372]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"test_pool","app":"app1","key":"key2","value":"val2"}]': finished 2024-02-13T14:22:32.993 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:32 smithi087 ceph-mon[27372]: osdmap e348: 8 total, 8 up, 8 in 2024-02-13T14:22:33.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:32 smithi148 ceph-mon[31198]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"test_pool","app":"app1","key":"key1","value":"val1"}]': finished 2024-02-13T14:22:33.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:32 smithi148 ceph-mon[31198]: osdmap e347: 8 total, 8 up, 8 in 2024-02-13T14:22:33.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:32 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/2994136514' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"test_pool","app":"app1","key":"key2","value":"val2"}]: dispatch 2024-02-13T14:22:33.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:32 smithi148 ceph-mon[31198]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"test_pool","app":"app1","key":"key2","value":"val2"}]: dispatch 2024-02-13T14:22:33.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:32 smithi148 ceph-mon[31198]: pgmap v475: 196 pgs: 196 active+clean; 582 KiB data, 292 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:22:33.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:32 smithi148 ceph-mon[31198]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"test_pool","app":"app1","key":"key2","value":"val2"}]': finished 2024-02-13T14:22:33.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:32 smithi148 ceph-mon[31198]: osdmap e348: 8 total, 8 up, 8 in 2024-02-13T14:22:33.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:33 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/2994136514' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"test_pool","app":"app2","key":"key1","value":"val1"}]: dispatch 2024-02-13T14:22:33.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:33 smithi087 ceph-mon[27372]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"test_pool","app":"app2","key":"key1","value":"val1"}]: dispatch 2024-02-13T14:22:33.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:33 smithi087 ceph-mon[27372]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"test_pool","app":"app2","key":"key1","value":"val1"}]': finished 2024-02-13T14:22:33.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:33 smithi087 ceph-mon[27372]: osdmap e349: 8 total, 8 up, 8 in 2024-02-13T14:22:33.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:33 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/2994136514' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"test_pool","app":"app1","key":"key1"}]: dispatch 2024-02-13T14:22:33.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:33 smithi087 ceph-mon[27372]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"test_pool","app":"app1","key":"key1"}]: dispatch 2024-02-13T14:22:33.993 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:33 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/2994136514' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"test_pool","app":"app2","key":"key1","value":"val1"}]: dispatch 2024-02-13T14:22:33.993 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:33 smithi087 ceph-mon[39120]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"test_pool","app":"app2","key":"key1","value":"val1"}]: dispatch 2024-02-13T14:22:33.993 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:33 smithi087 ceph-mon[39120]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"test_pool","app":"app2","key":"key1","value":"val1"}]': finished 2024-02-13T14:22:33.993 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:33 smithi087 ceph-mon[39120]: osdmap e349: 8 total, 8 up, 8 in 2024-02-13T14:22:33.993 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:33 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/2994136514' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"test_pool","app":"app1","key":"key1"}]: dispatch 2024-02-13T14:22:33.993 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:33 smithi087 ceph-mon[39120]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"test_pool","app":"app1","key":"key1"}]: dispatch 2024-02-13T14:22:34.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:33 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/2994136514' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"test_pool","app":"app2","key":"key1","value":"val1"}]: dispatch 2024-02-13T14:22:34.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:33 smithi148 ceph-mon[31198]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"test_pool","app":"app2","key":"key1","value":"val1"}]: dispatch 2024-02-13T14:22:34.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:33 smithi148 ceph-mon[31198]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"test_pool","app":"app2","key":"key1","value":"val1"}]': finished 2024-02-13T14:22:34.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:33 smithi148 ceph-mon[31198]: osdmap e349: 8 total, 8 up, 8 in 2024-02-13T14:22:34.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:33 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/2994136514' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"test_pool","app":"app1","key":"key1"}]: dispatch 2024-02-13T14:22:34.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:33 smithi148 ceph-mon[31198]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"test_pool","app":"app1","key":"key1"}]: dispatch 2024-02-13T14:22:34.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:34 smithi087 ceph-mon[39120]: pgmap v478: 196 pgs: 196 active+clean; 582 KiB data, 292 MiB used, 715 GiB / 715 GiB avail 2024-02-13T14:22:34.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:34 smithi087 ceph-mon[39120]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"test_pool","app":"app1","key":"key1"}]': finished 2024-02-13T14:22:34.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:34 smithi087 ceph-mon[39120]: osdmap e350: 8 total, 8 up, 8 in 2024-02-13T14:22:34.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:34 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/2994136514' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:22:34.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:34 smithi087 ceph-mon[39120]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:22:34.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:34 smithi087 ceph-mon[27372]: pgmap v478: 196 pgs: 196 active+clean; 582 KiB data, 292 MiB used, 715 GiB / 715 GiB avail 2024-02-13T14:22:34.993 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:34 smithi087 ceph-mon[27372]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"test_pool","app":"app1","key":"key1"}]': finished 2024-02-13T14:22:34.993 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:34 smithi087 ceph-mon[27372]: osdmap e350: 8 total, 8 up, 8 in 2024-02-13T14:22:34.993 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:34 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/2994136514' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:22:34.993 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:34 smithi087 ceph-mon[27372]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:22:35.065 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:34 smithi148 ceph-mon[31198]: pgmap v478: 196 pgs: 196 active+clean; 582 KiB data, 292 MiB used, 715 GiB / 715 GiB avail 2024-02-13T14:22:35.066 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:34 smithi148 ceph-mon[31198]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"test_pool","app":"app1","key":"key1"}]': finished 2024-02-13T14:22:35.066 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:34 smithi148 ceph-mon[31198]: osdmap e350: 8 total, 8 up, 8 in 2024-02-13T14:22:35.066 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:34 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/2994136514' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:22:35.066 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:34 smithi148 ceph-mon[31198]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:22:35.368 INFO:journalctl@ceph.iscsi.iscsi.a.smithi148.stdout:Feb 13 14:22:35 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-iscsi-iscsi-a[59644]: debug there is no tcmu-runner data available 2024-02-13T14:22:35.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:35 smithi087 ceph-mon[39120]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:22:35.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:35 smithi087 ceph-mon[39120]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:22:35.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:35 smithi087 ceph-mon[39120]: osdmap e351: 8 total, 8 up, 8 in 2024-02-13T14:22:35.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:35 smithi087 ceph-mon[27372]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:22:35.993 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:35 smithi087 ceph-mon[27372]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:22:35.993 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:35 smithi087 ceph-mon[27372]: osdmap e351: 8 total, 8 up, 8 in 2024-02-13T14:22:36.118 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:35 smithi148 ceph-mon[31198]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:22:36.118 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:35 smithi148 ceph-mon[31198]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:22:36.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:35 smithi148 ceph-mon[31198]: osdmap e351: 8 total, 8 up, 8 in 2024-02-13T14:22:36.675 INFO:tasks.workunit.client.0.smithi087.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_applications PASSED [ 84%] 2024-02-13T14:22:36.991 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:36 smithi087 ceph-mon[27372]: pgmap v481: 196 pgs: 196 active+clean; 582 KiB data, 292 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:22:36.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:36 smithi087 ceph-mon[27372]: osdmap e352: 8 total, 8 up, 8 in 2024-02-13T14:22:36.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:36 smithi087 ceph-mon[39120]: pgmap v481: 196 pgs: 196 active+clean; 582 KiB data, 292 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:22:36.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:36 smithi087 ceph-mon[39120]: osdmap e352: 8 total, 8 up, 8 in 2024-02-13T14:22:37.118 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:36 smithi148 ceph-mon[31198]: pgmap v481: 196 pgs: 196 active+clean; 582 KiB data, 292 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:22:37.118 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:36 smithi148 ceph-mon[31198]: osdmap e352: 8 total, 8 up, 8 in 2024-02-13T14:22:37.491 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:22:37 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: ::ffff:172.21.15.148 - - [13/Feb/2024:14:22:37] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-13T14:22:37.991 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:37 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:22:37.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:37 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:22:38.118 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:37 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:22:38.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:38 smithi087 ceph-mon[27372]: osdmap e353: 8 total, 8 up, 8 in 2024-02-13T14:22:38.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:38 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/1441118370' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:22:38.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:38 smithi087 ceph-mon[27372]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:22:38.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:38 smithi087 ceph-mon[27372]: pgmap v484: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 292 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:22:38.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:38 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:22:38.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:38 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:22:38.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:38 smithi087 ceph-mon[27372]: pgmap v485: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 292 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-13T14:22:38.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:38 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:22:38.993 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:38 smithi087 ceph-mon[39120]: osdmap e353: 8 total, 8 up, 8 in 2024-02-13T14:22:38.993 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:38 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/1441118370' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:22:38.993 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:38 smithi087 ceph-mon[39120]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:22:38.993 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:38 smithi087 ceph-mon[39120]: pgmap v484: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 292 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:22:38.993 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:38 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:22:38.993 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:38 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:22:38.993 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:38 smithi087 ceph-mon[39120]: pgmap v485: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 292 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-13T14:22:38.993 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:38 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:22:39.118 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:38 smithi148 ceph-mon[31198]: osdmap e353: 8 total, 8 up, 8 in 2024-02-13T14:22:39.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:38 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/1441118370' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:22:39.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:38 smithi148 ceph-mon[31198]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:22:39.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:38 smithi148 ceph-mon[31198]: pgmap v484: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 292 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:22:39.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:38 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:22:39.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:38 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:22:39.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:38 smithi148 ceph-mon[31198]: pgmap v485: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 292 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-13T14:22:39.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:38 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:22:39.716 INFO:tasks.workunit.client.0.smithi087.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_service_daemon PASSED [ 85%] 2024-02-13T14:22:39.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:39 smithi087 ceph-mon[27372]: Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON) 2024-02-13T14:22:39.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:39 smithi087 ceph-mon[27372]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T14:22:39.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:39 smithi087 ceph-mon[27372]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:22:39.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:39 smithi087 ceph-mon[27372]: osdmap e354: 8 total, 8 up, 8 in 2024-02-13T14:22:39.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:39 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:22:39.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:39 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-13T14:22:39.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:39 smithi087 ceph-mon[39120]: Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON) 2024-02-13T14:22:39.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:39 smithi087 ceph-mon[39120]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T14:22:39.993 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:39 smithi087 ceph-mon[39120]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:22:39.993 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:39 smithi087 ceph-mon[39120]: osdmap e354: 8 total, 8 up, 8 in 2024-02-13T14:22:39.993 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:39 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:22:39.993 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:39 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-13T14:22:40.118 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:39 smithi148 ceph-mon[31198]: Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON) 2024-02-13T14:22:40.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:39 smithi148 ceph-mon[31198]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T14:22:40.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:39 smithi148 ceph-mon[31198]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:22:40.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:39 smithi148 ceph-mon[31198]: osdmap e354: 8 total, 8 up, 8 in 2024-02-13T14:22:40.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:39 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' 2024-02-13T14:22:40.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:39 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-13T14:22:40.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:40 smithi087 ceph-mon[27372]: osdmap e355: 8 total, 8 up, 8 in 2024-02-13T14:22:40.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:40 smithi087 ceph-mon[27372]: pgmap v488: 164 pgs: 164 active+clean; 582 KiB data, 277 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-13T14:22:40.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:40 smithi087 ceph-mon[39120]: osdmap e355: 8 total, 8 up, 8 in 2024-02-13T14:22:40.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:40 smithi087 ceph-mon[39120]: pgmap v488: 164 pgs: 164 active+clean; 582 KiB data, 277 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-13T14:22:41.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:40 smithi148 ceph-mon[31198]: osdmap e355: 8 total, 8 up, 8 in 2024-02-13T14:22:41.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:40 smithi148 ceph-mon[31198]: pgmap v488: 164 pgs: 164 active+clean; 582 KiB data, 277 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-13T14:22:41.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:41 smithi087 ceph-mon[27372]: osdmap e356: 8 total, 8 up, 8 in 2024-02-13T14:22:41.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:41 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/2470883984' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:22:41.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:41 smithi087 ceph-mon[39120]: osdmap e356: 8 total, 8 up, 8 in 2024-02-13T14:22:41.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:41 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/2470883984' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:22:42.118 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:41 smithi148 ceph-mon[31198]: osdmap e356: 8 total, 8 up, 8 in 2024-02-13T14:22:42.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:41 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/2470883984' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:22:42.739 INFO:tasks.workunit.client.0.smithi087.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx::test_alignment PASSED [ 86%] 2024-02-13T14:22:42.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:42 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/2470883984' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:22:42.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:42 smithi087 ceph-mon[27372]: osdmap e357: 8 total, 8 up, 8 in 2024-02-13T14:22:42.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:42 smithi087 ceph-mon[27372]: pgmap v491: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 277 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:22:42.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:42 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/2470883984' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:22:42.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:42 smithi087 ceph-mon[39120]: osdmap e357: 8 total, 8 up, 8 in 2024-02-13T14:22:42.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:42 smithi087 ceph-mon[39120]: pgmap v491: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 277 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:22:43.118 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:42 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/2470883984' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:22:43.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:42 smithi148 ceph-mon[31198]: osdmap e357: 8 total, 8 up, 8 in 2024-02-13T14:22:43.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:42 smithi148 ceph-mon[31198]: pgmap v491: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 277 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:22:43.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:43 smithi087 ceph-mon[27372]: osdmap e358: 8 total, 8 up, 8 in 2024-02-13T14:22:43.992 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:43 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/2927798502' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-ec", "profile": ["k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T14:22:43.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:43 smithi087 ceph-mon[39120]: osdmap e358: 8 total, 8 up, 8 in 2024-02-13T14:22:43.992 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:43 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/2927798502' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-ec", "profile": ["k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T14:22:44.118 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:43 smithi148 ceph-mon[31198]: osdmap e358: 8 total, 8 up, 8 in 2024-02-13T14:22:44.118 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:43 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/2927798502' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-ec", "profile": ["k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-13T14:22:45.076 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:44 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/2927798502' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-ec", "profile": ["k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T14:22:45.076 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:44 smithi148 ceph-mon[31198]: osdmap e359: 8 total, 8 up, 8 in 2024-02-13T14:22:45.076 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:44 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/2927798502' entity='client.admin' cmd=[{"prefix": "osd pool create", "pg_num": 8, "pgp_num": 8, "pool": "test-ec", "pool_type": "erasure", "erasure_code_profile": "testprofile-test-ec"}]: dispatch 2024-02-13T14:22:45.076 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:44 smithi148 ceph-mon[31198]: pgmap v494: 164 pgs: 164 active+clean; 582 KiB data, 277 MiB used, 715 GiB / 715 GiB avail 2024-02-13T14:22:45.241 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:44 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/2927798502' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-ec", "profile": ["k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T14:22:45.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:44 smithi087 ceph-mon[27372]: osdmap e359: 8 total, 8 up, 8 in 2024-02-13T14:22:45.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:44 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/2927798502' entity='client.admin' cmd=[{"prefix": "osd pool create", "pg_num": 8, "pgp_num": 8, "pool": "test-ec", "pool_type": "erasure", "erasure_code_profile": "testprofile-test-ec"}]: dispatch 2024-02-13T14:22:45.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:44 smithi087 ceph-mon[27372]: pgmap v494: 164 pgs: 164 active+clean; 582 KiB data, 277 MiB used, 715 GiB / 715 GiB avail 2024-02-13T14:22:45.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:44 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/2927798502' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-ec", "profile": ["k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-13T14:22:45.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:44 smithi087 ceph-mon[39120]: osdmap e359: 8 total, 8 up, 8 in 2024-02-13T14:22:45.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:44 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/2927798502' entity='client.admin' cmd=[{"prefix": "osd pool create", "pg_num": 8, "pgp_num": 8, "pool": "test-ec", "pool_type": "erasure", "erasure_code_profile": "testprofile-test-ec"}]: dispatch 2024-02-13T14:22:45.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:44 smithi087 ceph-mon[39120]: pgmap v494: 164 pgs: 164 active+clean; 582 KiB data, 277 MiB used, 715 GiB / 715 GiB avail 2024-02-13T14:22:45.368 INFO:journalctl@ceph.iscsi.iscsi.a.smithi148.stdout:Feb 13 14:22:45 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-iscsi-iscsi-a[59644]: debug there is no tcmu-runner data available 2024-02-13T14:22:46.118 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:45 smithi148 ceph-mon[31198]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T14:22:46.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:45 smithi148 ceph-mon[31198]: osdmap e360: 8 total, 8 up, 8 in 2024-02-13T14:22:46.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:45 smithi148 ceph-mon[31198]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:22:46.241 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:45 smithi087 ceph-mon[27372]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T14:22:46.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:45 smithi087 ceph-mon[27372]: osdmap e360: 8 total, 8 up, 8 in 2024-02-13T14:22:46.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:45 smithi087 ceph-mon[27372]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:22:46.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:45 smithi087 ceph-mon[39120]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T14:22:46.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:45 smithi087 ceph-mon[39120]: osdmap e360: 8 total, 8 up, 8 in 2024-02-13T14:22:46.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:45 smithi087 ceph-mon[39120]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:22:47.118 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:46 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/2927798502' entity='client.admin' cmd='[{"prefix": "osd pool create", "pg_num": 8, "pgp_num": 8, "pool": "test-ec", "pool_type": "erasure", "erasure_code_profile": "testprofile-test-ec"}]': finished 2024-02-13T14:22:47.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:46 smithi148 ceph-mon[31198]: osdmap e361: 8 total, 8 up, 8 in 2024-02-13T14:22:47.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:46 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/2927798502' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:22:47.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:46 smithi148 ceph-mon[31198]: pgmap v497: 172 pgs: 8 unknown, 164 active+clean; 582 KiB data, 277 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:22:47.221 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:46 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/2927798502' entity='client.admin' cmd='[{"prefix": "osd pool create", "pg_num": 8, "pgp_num": 8, "pool": "test-ec", "pool_type": "erasure", "erasure_code_profile": "testprofile-test-ec"}]': finished 2024-02-13T14:22:47.222 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:46 smithi087 ceph-mon[27372]: osdmap e361: 8 total, 8 up, 8 in 2024-02-13T14:22:47.222 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:46 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/2927798502' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:22:47.222 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:46 smithi087 ceph-mon[27372]: pgmap v497: 172 pgs: 8 unknown, 164 active+clean; 582 KiB data, 277 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:22:47.222 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:46 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/2927798502' entity='client.admin' cmd='[{"prefix": "osd pool create", "pg_num": 8, "pgp_num": 8, "pool": "test-ec", "pool_type": "erasure", "erasure_code_profile": "testprofile-test-ec"}]': finished 2024-02-13T14:22:47.222 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:46 smithi087 ceph-mon[39120]: osdmap e361: 8 total, 8 up, 8 in 2024-02-13T14:22:47.222 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:46 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/2927798502' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:22:47.222 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:46 smithi087 ceph-mon[39120]: pgmap v497: 172 pgs: 8 unknown, 164 active+clean; 582 KiB data, 277 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:22:47.491 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:22:47 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: ::ffff:172.21.15.148 - - [13/Feb/2024:14:22:47] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-13T14:22:47.773 INFO:tasks.workunit.client.0.smithi087.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctxEc::test_alignment PASSED [ 87%] 2024-02-13T14:22:48.118 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:47 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/2927798502' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:22:48.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:47 smithi148 ceph-mon[31198]: osdmap e362: 8 total, 8 up, 8 in 2024-02-13T14:22:48.241 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:47 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/2927798502' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:22:48.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:47 smithi087 ceph-mon[27372]: osdmap e362: 8 total, 8 up, 8 in 2024-02-13T14:22:48.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:47 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/2927798502' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:22:48.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:47 smithi087 ceph-mon[39120]: osdmap e362: 8 total, 8 up, 8 in 2024-02-13T14:22:49.118 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:48 smithi148 ceph-mon[31198]: osdmap e363: 8 total, 8 up, 8 in 2024-02-13T14:22:49.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:48 smithi148 ceph-mon[31198]: pgmap v500: 164 pgs: 164 active+clean; 582 KiB data, 277 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:22:49.241 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:48 smithi087 ceph-mon[27372]: osdmap e363: 8 total, 8 up, 8 in 2024-02-13T14:22:49.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:48 smithi087 ceph-mon[27372]: pgmap v500: 164 pgs: 164 active+clean; 582 KiB data, 277 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:22:49.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:48 smithi087 ceph-mon[39120]: osdmap e363: 8 total, 8 up, 8 in 2024-02-13T14:22:49.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:48 smithi087 ceph-mon[39120]: pgmap v500: 164 pgs: 164 active+clean; 582 KiB data, 277 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:22:50.118 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:49 smithi148 ceph-mon[31198]: osdmap e364: 8 total, 8 up, 8 in 2024-02-13T14:22:50.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:49 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/2102060749' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:22:50.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:49 smithi087 ceph-mon[27372]: osdmap e364: 8 total, 8 up, 8 in 2024-02-13T14:22:50.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:49 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/2102060749' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:22:50.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:49 smithi087 ceph-mon[39120]: osdmap e364: 8 total, 8 up, 8 in 2024-02-13T14:22:50.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:49 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/2102060749' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:22:50.796 INFO:tasks.workunit.client.0.smithi087.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx2::test_get_last_version PASSED [ 88%] 2024-02-13T14:22:51.118 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:50 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/2102060749' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:22:51.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:50 smithi148 ceph-mon[31198]: osdmap e365: 8 total, 8 up, 8 in 2024-02-13T14:22:51.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:50 smithi148 ceph-mon[31198]: pgmap v503: 196 pgs: 21 creating+activating, 11 creating+peering, 164 active+clean; 582 KiB data, 278 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:22:51.143 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:50 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/2102060749' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:22:51.143 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:50 smithi087 ceph-mon[27372]: osdmap e365: 8 total, 8 up, 8 in 2024-02-13T14:22:51.143 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:50 smithi087 ceph-mon[27372]: pgmap v503: 196 pgs: 21 creating+activating, 11 creating+peering, 164 active+clean; 582 KiB data, 278 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:22:51.143 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:50 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/2102060749' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:22:51.143 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:50 smithi087 ceph-mon[39120]: osdmap e365: 8 total, 8 up, 8 in 2024-02-13T14:22:51.143 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:50 smithi087 ceph-mon[39120]: pgmap v503: 196 pgs: 21 creating+activating, 11 creating+peering, 164 active+clean; 582 KiB data, 278 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:22:52.118 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:51 smithi148 ceph-mon[31198]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T14:22:52.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:51 smithi148 ceph-mon[31198]: osdmap e366: 8 total, 8 up, 8 in 2024-02-13T14:22:52.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:51 smithi087 ceph-mon[27372]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T14:22:52.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:51 smithi087 ceph-mon[27372]: osdmap e366: 8 total, 8 up, 8 in 2024-02-13T14:22:52.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:51 smithi087 ceph-mon[39120]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T14:22:52.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:51 smithi087 ceph-mon[39120]: osdmap e366: 8 total, 8 up, 8 in 2024-02-13T14:22:53.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:52 smithi148 ceph-mon[31198]: osdmap e367: 8 total, 8 up, 8 in 2024-02-13T14:22:53.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:52 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/683324571' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:22:53.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:52 smithi148 ceph-mon[31198]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:22:53.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:52 smithi148 ceph-mon[31198]: pgmap v506: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 278 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:22:53.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:52 smithi087 ceph-mon[27372]: osdmap e367: 8 total, 8 up, 8 in 2024-02-13T14:22:53.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:52 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/683324571' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:22:53.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:52 smithi087 ceph-mon[27372]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:22:53.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:52 smithi087 ceph-mon[27372]: pgmap v506: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 278 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:22:53.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:52 smithi087 ceph-mon[39120]: osdmap e367: 8 total, 8 up, 8 in 2024-02-13T14:22:53.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:52 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/683324571' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:22:53.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:52 smithi087 ceph-mon[39120]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:22:53.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:52 smithi087 ceph-mon[39120]: pgmap v506: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 278 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:22:53.818 INFO:tasks.workunit.client.0.smithi087.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoctx2::test_get_stats PASSED [ 90%] 2024-02-13T14:22:54.118 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:53 smithi148 ceph-mon[31198]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:22:54.118 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:53 smithi148 ceph-mon[31198]: osdmap e368: 8 total, 8 up, 8 in 2024-02-13T14:22:54.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:53 smithi087 ceph-mon[39120]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:22:54.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:53 smithi087 ceph-mon[39120]: osdmap e368: 8 total, 8 up, 8 in 2024-02-13T14:22:54.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:53 smithi087 ceph-mon[27372]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:22:54.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:53 smithi087 ceph-mon[27372]: osdmap e368: 8 total, 8 up, 8 in 2024-02-13T14:22:55.084 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:54 smithi148 ceph-mon[31198]: osdmap e369: 8 total, 8 up, 8 in 2024-02-13T14:22:55.084 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:54 smithi148 ceph-mon[31198]: pgmap v509: 164 pgs: 164 active+clean; 582 KiB data, 278 MiB used, 715 GiB / 715 GiB avail 2024-02-13T14:22:55.085 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:54 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-13T14:22:55.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:54 smithi087 ceph-mon[27372]: osdmap e369: 8 total, 8 up, 8 in 2024-02-13T14:22:55.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:54 smithi087 ceph-mon[27372]: pgmap v509: 164 pgs: 164 active+clean; 582 KiB data, 278 MiB used, 715 GiB / 715 GiB avail 2024-02-13T14:22:55.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:54 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-13T14:22:55.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:54 smithi087 ceph-mon[39120]: osdmap e369: 8 total, 8 up, 8 in 2024-02-13T14:22:55.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:54 smithi087 ceph-mon[39120]: pgmap v509: 164 pgs: 164 active+clean; 582 KiB data, 278 MiB used, 715 GiB / 715 GiB avail 2024-02-13T14:22:55.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:54 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-13T14:22:55.368 INFO:journalctl@ceph.iscsi.iscsi.a.smithi148.stdout:Feb 13 14:22:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-iscsi-iscsi-a[59644]: debug there is no tcmu-runner data available 2024-02-13T14:22:56.118 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:55 smithi148 ceph-mon[31198]: osdmap e370: 8 total, 8 up, 8 in 2024-02-13T14:22:56.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:55 smithi148 ceph-mon[31198]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:22:56.241 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:55 smithi087 ceph-mon[27372]: osdmap e370: 8 total, 8 up, 8 in 2024-02-13T14:22:56.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:55 smithi087 ceph-mon[27372]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:22:56.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:55 smithi087 ceph-mon[39120]: osdmap e370: 8 total, 8 up, 8 in 2024-02-13T14:22:56.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:55 smithi087 ceph-mon[39120]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:22:56.845 INFO:tasks.workunit.client.0.smithi087.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestObject::test_read PASSED [ 91%] 2024-02-13T14:22:57.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:56 smithi148 ceph-mon[31198]: osdmap e371: 8 total, 8 up, 8 in 2024-02-13T14:22:57.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:56 smithi148 ceph-mon[31198]: pgmap v512: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 279 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:22:57.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:56 smithi148 ceph-mon[31198]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T14:22:57.221 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:56 smithi087 ceph-mon[27372]: osdmap e371: 8 total, 8 up, 8 in 2024-02-13T14:22:57.221 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:56 smithi087 ceph-mon[27372]: pgmap v512: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 279 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:22:57.221 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:56 smithi087 ceph-mon[27372]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T14:22:57.221 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:56 smithi087 ceph-mon[39120]: osdmap e371: 8 total, 8 up, 8 in 2024-02-13T14:22:57.221 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:56 smithi087 ceph-mon[39120]: pgmap v512: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 279 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:22:57.221 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:56 smithi087 ceph-mon[39120]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T14:22:57.491 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:22:57 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: ::ffff:172.21.15.148 - - [13/Feb/2024:14:22:57] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-13T14:22:58.118 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:57 smithi148 ceph-mon[31198]: osdmap e372: 8 total, 8 up, 8 in 2024-02-13T14:22:58.241 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:57 smithi087 ceph-mon[27372]: osdmap e372: 8 total, 8 up, 8 in 2024-02-13T14:22:58.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:57 smithi087 ceph-mon[39120]: osdmap e372: 8 total, 8 up, 8 in 2024-02-13T14:22:59.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:58 smithi148 ceph-mon[31198]: osdmap e373: 8 total, 8 up, 8 in 2024-02-13T14:22:59.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:58 smithi148 ceph-mon[31198]: pgmap v515: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 279 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:22:59.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:22:58 smithi148 ceph-mon[31198]: osdmap e374: 8 total, 8 up, 8 in 2024-02-13T14:22:59.241 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:58 smithi087 ceph-mon[27372]: osdmap e373: 8 total, 8 up, 8 in 2024-02-13T14:22:59.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:58 smithi087 ceph-mon[27372]: pgmap v515: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 279 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:22:59.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:22:58 smithi087 ceph-mon[27372]: osdmap e374: 8 total, 8 up, 8 in 2024-02-13T14:22:59.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:58 smithi087 ceph-mon[39120]: osdmap e373: 8 total, 8 up, 8 in 2024-02-13T14:22:59.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:58 smithi087 ceph-mon[39120]: pgmap v515: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 279 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:22:59.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:22:58 smithi087 ceph-mon[39120]: osdmap e374: 8 total, 8 up, 8 in 2024-02-13T14:22:59.860 INFO:tasks.workunit.client.0.smithi087.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestObject::test_seek PASSED [ 92%] 2024-02-13T14:23:01.118 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:23:00 smithi148 ceph-mon[31198]: osdmap e375: 8 total, 8 up, 8 in 2024-02-13T14:23:01.119 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:23:00 smithi148 ceph-mon[31198]: pgmap v518: 164 pgs: 164 active+clean; 582 KiB data, 287 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:23:01.241 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:23:00 smithi087 ceph-mon[27372]: osdmap e375: 8 total, 8 up, 8 in 2024-02-13T14:23:01.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:23:00 smithi087 ceph-mon[27372]: pgmap v518: 164 pgs: 164 active+clean; 582 KiB data, 287 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:23:01.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:23:00 smithi087 ceph-mon[39120]: osdmap e375: 8 total, 8 up, 8 in 2024-02-13T14:23:01.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:23:00 smithi087 ceph-mon[39120]: pgmap v518: 164 pgs: 164 active+clean; 582 KiB data, 287 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:23:02.118 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:23:01 smithi148 ceph-mon[31198]: osdmap e376: 8 total, 8 up, 8 in 2024-02-13T14:23:02.241 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:23:01 smithi087 ceph-mon[27372]: osdmap e376: 8 total, 8 up, 8 in 2024-02-13T14:23:02.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:23:01 smithi087 ceph-mon[39120]: osdmap e376: 8 total, 8 up, 8 in 2024-02-13T14:23:02.882 INFO:tasks.workunit.client.0.smithi087.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestObject::test_write PASSED [ 93%] 2024-02-13T14:23:03.241 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:23:02 smithi087 ceph-mon[27372]: osdmap e377: 8 total, 8 up, 8 in 2024-02-13T14:23:03.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:23:02 smithi087 ceph-mon[27372]: pgmap v521: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 287 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:23:03.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:23:02 smithi087 ceph-mon[39120]: osdmap e377: 8 total, 8 up, 8 in 2024-02-13T14:23:03.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:23:02 smithi087 ceph-mon[39120]: pgmap v521: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 287 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:23:03.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:23:02 smithi148 ceph-mon[31198]: osdmap e377: 8 total, 8 up, 8 in 2024-02-13T14:23:03.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:23:02 smithi148 ceph-mon[31198]: pgmap v521: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 287 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:23:04.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:23:03 smithi087 ceph-mon[27372]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T14:23:04.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:23:03 smithi087 ceph-mon[27372]: osdmap e378: 8 total, 8 up, 8 in 2024-02-13T14:23:04.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:23:03 smithi087 ceph-mon[39120]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T14:23:04.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:23:03 smithi087 ceph-mon[39120]: osdmap e378: 8 total, 8 up, 8 in 2024-02-13T14:23:04.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:23:03 smithi148 ceph-mon[31198]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T14:23:04.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:23:03 smithi148 ceph-mon[31198]: osdmap e378: 8 total, 8 up, 8 in 2024-02-13T14:23:05.241 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:23:04 smithi087 ceph-mon[27372]: osdmap e379: 8 total, 8 up, 8 in 2024-02-13T14:23:05.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:23:04 smithi087 ceph-mon[27372]: pgmap v524: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 287 MiB used, 715 GiB / 715 GiB avail 2024-02-13T14:23:05.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:23:04 smithi087 ceph-mon[39120]: osdmap e379: 8 total, 8 up, 8 in 2024-02-13T14:23:05.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:23:04 smithi087 ceph-mon[39120]: pgmap v524: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 287 MiB used, 715 GiB / 715 GiB avail 2024-02-13T14:23:05.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:23:04 smithi148 ceph-mon[31198]: osdmap e379: 8 total, 8 up, 8 in 2024-02-13T14:23:05.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:23:04 smithi148 ceph-mon[31198]: pgmap v524: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 287 MiB used, 715 GiB / 715 GiB avail 2024-02-13T14:23:05.369 INFO:journalctl@ceph.iscsi.iscsi.a.smithi148.stdout:Feb 13 14:23:05 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-iscsi-iscsi-a[59644]: debug there is no tcmu-runner data available 2024-02-13T14:23:06.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:23:05 smithi087 ceph-mon[27372]: osdmap e380: 8 total, 8 up, 8 in 2024-02-13T14:23:06.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:23:05 smithi087 ceph-mon[27372]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:23:06.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:23:05 smithi087 ceph-mon[39120]: osdmap e380: 8 total, 8 up, 8 in 2024-02-13T14:23:06.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:23:05 smithi087 ceph-mon[39120]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:23:06.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:23:05 smithi148 ceph-mon[31198]: osdmap e380: 8 total, 8 up, 8 in 2024-02-13T14:23:06.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:23:05 smithi148 ceph-mon[31198]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:23:07.221 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:23:06 smithi087 ceph-mon[27372]: osdmap e381: 8 total, 8 up, 8 in 2024-02-13T14:23:07.222 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:23:06 smithi087 ceph-mon[27372]: pgmap v527: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 288 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:23:07.222 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:23:06 smithi087 ceph-mon[39120]: osdmap e381: 8 total, 8 up, 8 in 2024-02-13T14:23:07.222 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:23:06 smithi087 ceph-mon[39120]: pgmap v527: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 288 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:23:07.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:23:06 smithi148 ceph-mon[31198]: osdmap e381: 8 total, 8 up, 8 in 2024-02-13T14:23:07.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:23:06 smithi148 ceph-mon[31198]: pgmap v527: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 288 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:23:07.491 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:23:07 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: ::ffff:172.21.15.148 - - [13/Feb/2024:14:23:07] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-13T14:23:08.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:23:07 smithi087 ceph-mon[27372]: osdmap e382: 8 total, 8 up, 8 in 2024-02-13T14:23:08.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:23:07 smithi087 ceph-mon[39120]: osdmap e382: 8 total, 8 up, 8 in 2024-02-13T14:23:08.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:23:07 smithi148 ceph-mon[31198]: osdmap e382: 8 total, 8 up, 8 in 2024-02-13T14:23:09.241 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:23:08 smithi087 ceph-mon[27372]: pgmap v529: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 288 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:23:09.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:23:08 smithi087 ceph-mon[27372]: osdmap e383: 8 total, 8 up, 8 in 2024-02-13T14:23:09.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:23:08 smithi087 ceph-mon[39120]: pgmap v529: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 288 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:23:09.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:23:08 smithi087 ceph-mon[39120]: osdmap e383: 8 total, 8 up, 8 in 2024-02-13T14:23:09.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:23:08 smithi148 ceph-mon[31198]: pgmap v529: 196 pgs: 32 creating+peering, 164 active+clean; 582 KiB data, 288 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:23:09.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:23:08 smithi148 ceph-mon[31198]: osdmap e383: 8 total, 8 up, 8 in 2024-02-13T14:23:10.241 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:23:09 smithi087 ceph-mon[27372]: osdmap e384: 8 total, 8 up, 8 in 2024-02-13T14:23:10.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:23:09 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/2699344162' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:23:10.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:23:09 smithi087 ceph-mon[27372]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-13T14:23:10.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:23:09 smithi087 ceph-mon[39120]: osdmap e384: 8 total, 8 up, 8 in 2024-02-13T14:23:10.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:23:09 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/2699344162' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:23:10.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:23:09 smithi087 ceph-mon[39120]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-13T14:23:10.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:23:09 smithi148 ceph-mon[31198]: osdmap e384: 8 total, 8 up, 8 in 2024-02-13T14:23:10.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:23:09 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/2699344162' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noup"}]: dispatch 2024-02-13T14:23:10.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:23:09 smithi148 ceph-mon[31198]: from='mgr.14562 172.21.15.87:0/3971453373' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-13T14:23:10.939 INFO:tasks.workunit.client.0.smithi087.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestIoCtxSelfManagedSnaps::test PASSED [ 94%] 2024-02-13T14:23:10.959 INFO:tasks.workunit.client.0.smithi087.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestCommand::test_monmap_dump PASSED [ 95%] 2024-02-13T14:23:10.976 INFO:tasks.workunit.client.0.smithi087.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestCommand::test_osd_bench PASSED [ 96%] 2024-02-13T14:23:11.241 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:23:10 smithi087 ceph-mon[27372]: pgmap v532: 196 pgs: 196 active+clean; 582 KiB data, 293 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 764 B/s wr, 2 op/s 2024-02-13T14:23:11.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:23:10 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/2699344162' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:23:11.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:23:10 smithi087 ceph-mon[27372]: osdmap e385: 8 total, 8 up, 8 in 2024-02-13T14:23:11.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:23:10 smithi087 ceph-mon[39120]: pgmap v532: 196 pgs: 196 active+clean; 582 KiB data, 293 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 764 B/s wr, 2 op/s 2024-02-13T14:23:11.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:23:10 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/2699344162' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:23:11.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:23:10 smithi087 ceph-mon[39120]: osdmap e385: 8 total, 8 up, 8 in 2024-02-13T14:23:11.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:23:10 smithi148 ceph-mon[31198]: pgmap v532: 196 pgs: 196 active+clean; 582 KiB data, 293 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 764 B/s wr, 2 op/s 2024-02-13T14:23:11.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:23:10 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/2699344162' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noup"}]': finished 2024-02-13T14:23:11.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:23:10 smithi148 ceph-mon[31198]: osdmap e385: 8 total, 8 up, 8 in 2024-02-13T14:23:11.948 INFO:tasks.workunit.client.0.smithi087.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestCommand::test_ceph_osd_pool_create_utf8 PASSED [ 97%] 2024-02-13T14:23:12.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:23:11 smithi087 ceph-mon[27372]: osdmap e386: 8 total, 8 up, 8 in 2024-02-13T14:23:12.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:23:11 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/493245935' entity='client.admin' cmd=[{"prefix": "mon dump"}]: dispatch 2024-02-13T14:23:12.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:23:11 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/493245935' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-13T14:23:12.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:23:11 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/493245935' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json", "epoch": 1003}]: dispatch 2024-02-13T14:23:12.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:23:11 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/4265413307' entity='client.admin' cmd=[{"prefix": "osd pool create", "pg_num": 16, "pool": "\u9ec5"}]: dispatch 2024-02-13T14:23:12.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:23:11 smithi087 ceph-mon[39120]: osdmap e386: 8 total, 8 up, 8 in 2024-02-13T14:23:12.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:23:11 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/493245935' entity='client.admin' cmd=[{"prefix": "mon dump"}]: dispatch 2024-02-13T14:23:12.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:23:11 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/493245935' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-13T14:23:12.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:23:11 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/493245935' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json", "epoch": 1003}]: dispatch 2024-02-13T14:23:12.243 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:23:11 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/4265413307' entity='client.admin' cmd=[{"prefix": "osd pool create", "pg_num": 16, "pool": "\u9ec5"}]: dispatch 2024-02-13T14:23:12.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:23:11 smithi148 ceph-mon[31198]: osdmap e386: 8 total, 8 up, 8 in 2024-02-13T14:23:12.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:23:11 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/493245935' entity='client.admin' cmd=[{"prefix": "mon dump"}]: dispatch 2024-02-13T14:23:12.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:23:11 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/493245935' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-13T14:23:12.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:23:11 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/493245935' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json", "epoch": 1003}]: dispatch 2024-02-13T14:23:12.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:23:11 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/4265413307' entity='client.admin' cmd=[{"prefix": "osd pool create", "pg_num": 16, "pool": "\u9ec5"}]: dispatch 2024-02-13T14:23:13.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:23:12 smithi087 ceph-mon[27372]: pgmap v535: 164 pgs: 164 active+clean; 582 KiB data, 293 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:23:13.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:23:12 smithi087 ceph-mon[27372]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T14:23:13.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:23:12 smithi087 ceph-mon[27372]: from='client.? 172.21.15.87:0/4265413307' entity='client.admin' cmd='[{"prefix": "osd pool create", "pg_num": 16, "pool": "\u9ec5"}]': finished 2024-02-13T14:23:13.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:23:12 smithi087 ceph-mon[27372]: osdmap e387: 8 total, 8 up, 8 in 2024-02-13T14:23:13.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:23:12 smithi087 ceph-mon[39120]: pgmap v535: 164 pgs: 164 active+clean; 582 KiB data, 293 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:23:13.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:23:12 smithi087 ceph-mon[39120]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T14:23:13.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:23:12 smithi087 ceph-mon[39120]: from='client.? 172.21.15.87:0/4265413307' entity='client.admin' cmd='[{"prefix": "osd pool create", "pg_num": 16, "pool": "\u9ec5"}]': finished 2024-02-13T14:23:13.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:23:12 smithi087 ceph-mon[39120]: osdmap e387: 8 total, 8 up, 8 in 2024-02-13T14:23:13.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:23:12 smithi148 ceph-mon[31198]: pgmap v535: 164 pgs: 164 active+clean; 582 KiB data, 293 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-13T14:23:13.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:23:12 smithi148 ceph-mon[31198]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-13T14:23:13.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:23:12 smithi148 ceph-mon[31198]: from='client.? 172.21.15.87:0/4265413307' entity='client.admin' cmd='[{"prefix": "osd pool create", "pg_num": 16, "pool": "\u9ec5"}]': finished 2024-02-13T14:23:13.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:23:12 smithi148 ceph-mon[31198]: osdmap e387: 8 total, 8 up, 8 in 2024-02-13T14:23:14.241 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:23:13 smithi087 ceph-mon[27372]: osdmap e388: 8 total, 8 up, 8 in 2024-02-13T14:23:14.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:23:13 smithi087 ceph-mon[39120]: osdmap e388: 8 total, 8 up, 8 in 2024-02-13T14:23:14.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:23:13 smithi148 ceph-mon[31198]: osdmap e388: 8 total, 8 up, 8 in 2024-02-13T14:23:15.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:23:14 smithi087 ceph-mon[39120]: pgmap v538: 212 pgs: 48 unknown, 164 active+clean; 582 KiB data, 293 MiB used, 715 GiB / 715 GiB avail 2024-02-13T14:23:15.242 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:23:14 smithi087 ceph-mon[39120]: osdmap e389: 8 total, 8 up, 8 in 2024-02-13T14:23:15.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:23:14 smithi087 ceph-mon[27372]: pgmap v538: 212 pgs: 48 unknown, 164 active+clean; 582 KiB data, 293 MiB used, 715 GiB / 715 GiB avail 2024-02-13T14:23:15.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:23:14 smithi087 ceph-mon[27372]: osdmap e389: 8 total, 8 up, 8 in 2024-02-13T14:23:15.368 INFO:journalctl@ceph.iscsi.iscsi.a.smithi148.stdout:Feb 13 14:23:15 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-iscsi-iscsi-a[59644]: debug there is no tcmu-runner data available 2024-02-13T14:23:15.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:23:14 smithi148 ceph-mon[31198]: pgmap v538: 212 pgs: 48 unknown, 164 active+clean; 582 KiB data, 293 MiB used, 715 GiB / 715 GiB avail 2024-02-13T14:23:15.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:23:14 smithi148 ceph-mon[31198]: osdmap e389: 8 total, 8 up, 8 in 2024-02-13T14:23:15.984 INFO:tasks.workunit.client.0.smithi087.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestWatchNotify::test PASSED [ 98%] 2024-02-13T14:23:16.241 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:23:15 smithi087 ceph-mon[39120]: osdmap e390: 8 total, 8 up, 8 in 2024-02-13T14:23:16.241 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:23:15 smithi087 ceph-mon[39120]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:23:16.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:23:15 smithi087 ceph-mon[27372]: osdmap e390: 8 total, 8 up, 8 in 2024-02-13T14:23:16.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:23:15 smithi087 ceph-mon[27372]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:23:16.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:23:15 smithi148 ceph-mon[31198]: osdmap e390: 8 total, 8 up, 8 in 2024-02-13T14:23:16.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:23:15 smithi148 ceph-mon[31198]: from='client.24428 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-13T14:23:17.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:23:16 smithi148 ceph-mon[31198]: pgmap v541: 212 pgs: 212 active+clean; 582 KiB data, 281 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 255 B/s wr, 1 op/s 2024-02-13T14:23:17.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:23:16 smithi148 ceph-mon[31198]: osdmap e391: 8 total, 8 up, 8 in 2024-02-13T14:23:17.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:23:16 smithi087 ceph-mon[27372]: pgmap v541: 212 pgs: 212 active+clean; 582 KiB data, 281 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 255 B/s wr, 1 op/s 2024-02-13T14:23:17.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:23:16 smithi087 ceph-mon[27372]: osdmap e391: 8 total, 8 up, 8 in 2024-02-13T14:23:17.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:23:16 smithi087 ceph-mon[39120]: pgmap v541: 212 pgs: 212 active+clean; 582 KiB data, 281 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 255 B/s wr, 1 op/s 2024-02-13T14:23:17.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:23:16 smithi087 ceph-mon[39120]: osdmap e391: 8 total, 8 up, 8 in 2024-02-13T14:23:17.492 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:23:17 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: ::ffff:172.21.15.148 - - [13/Feb/2024:14:23:17] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-13T14:23:18.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:23:18 smithi148 ceph-mon[31198]: osdmap e392: 8 total, 8 up, 8 in 2024-02-13T14:23:18.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:23:18 smithi148 ceph-mon[31198]: osdmap e393: 8 total, 8 up, 8 in 2024-02-13T14:23:18.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:23:18 smithi087 ceph-mon[27372]: osdmap e392: 8 total, 8 up, 8 in 2024-02-13T14:23:18.492 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:23:18 smithi087 ceph-mon[27372]: osdmap e393: 8 total, 8 up, 8 in 2024-02-13T14:23:18.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:23:18 smithi087 ceph-mon[39120]: osdmap e392: 8 total, 8 up, 8 in 2024-02-13T14:23:18.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:23:18 smithi087 ceph-mon[39120]: osdmap e393: 8 total, 8 up, 8 in 2024-02-13T14:23:19.368 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:23:19 smithi148 ceph-mon[31198]: pgmap v544: 212 pgs: 32 unknown, 180 active+clean; 582 KiB data, 281 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-13T14:23:19.369 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:23:19 smithi148 ceph-mon[31198]: osdmap e394: 8 total, 8 up, 8 in 2024-02-13T14:23:19.491 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:23:19 smithi087 ceph-mon[27372]: pgmap v544: 212 pgs: 32 unknown, 180 active+clean; 582 KiB data, 281 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-13T14:23:19.491 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:23:19 smithi087 ceph-mon[27372]: osdmap e394: 8 total, 8 up, 8 in 2024-02-13T14:23:19.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:23:19 smithi087 ceph-mon[39120]: pgmap v544: 212 pgs: 32 unknown, 180 active+clean; 582 KiB data, 281 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-13T14:23:19.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:23:19 smithi087 ceph-mon[39120]: osdmap e394: 8 total, 8 up, 8 in 2024-02-13T14:23:20.021 INFO:tasks.workunit.client.0.smithi087.stdout:../../../clone.client.0/src/test/pybind/test_rados.py::TestWatchNotify::test_aio_notify PASSED [100%] 2024-02-13T14:23:20.021 INFO:tasks.workunit.client.0.smithi087.stdout: 2024-02-13T14:23:20.021 INFO:tasks.workunit.client.0.smithi087.stdout:=============================== warnings summary =============================== 2024-02-13T14:23:20.021 INFO:tasks.workunit.client.0.smithi087.stdout:../../../clone.client.0/src/test/pybind/test_rados.py:210 2024-02-13T14:23:20.021 INFO:tasks.workunit.client.0.smithi087.stdout: /home/ubuntu/cephtest/clone.client.0/src/test/pybind/test_rados.py:210: DeprecationWarning: invalid escape sequence \- 2024-02-13T14:23:20.021 INFO:tasks.workunit.client.0.smithi087.stdout: assert re.match('[0-9a-f\-]{36}', fsid, re.I) 2024-02-13T14:23:20.021 INFO:tasks.workunit.client.0.smithi087.stdout: 2024-02-13T14:23:20.021 INFO:tasks.workunit.client.0.smithi087.stdout:../../../clone.client.0/src/test/pybind/test_rados.py:953 2024-02-13T14:23:20.021 INFO:tasks.workunit.client.0.smithi087.stdout: /home/ubuntu/cephtest/clone.client.0/src/test/pybind/test_rados.py:953: PytestUnknownMarkWarning: Unknown pytest.mark.wait - is this a typo? You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/stable/mark.html 2024-02-13T14:23:20.021 INFO:tasks.workunit.client.0.smithi087.stdout: @pytest.mark.wait 2024-02-13T14:23:20.022 INFO:tasks.workunit.client.0.smithi087.stdout: 2024-02-13T14:23:20.022 INFO:tasks.workunit.client.0.smithi087.stdout:../../../clone.client.0/src/test/pybind/test_rados.py:989 2024-02-13T14:23:20.022 INFO:tasks.workunit.client.0.smithi087.stdout: /home/ubuntu/cephtest/clone.client.0/src/test/pybind/test_rados.py:989: PytestUnknownMarkWarning: Unknown pytest.mark.wait - is this a typo? You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/stable/mark.html 2024-02-13T14:23:20.022 INFO:tasks.workunit.client.0.smithi087.stdout: @pytest.mark.wait 2024-02-13T14:23:20.022 INFO:tasks.workunit.client.0.smithi087.stdout: 2024-02-13T14:23:20.022 INFO:tasks.workunit.client.0.smithi087.stdout:../../../clone.client.0/src/test/pybind/test_rados.py:1017 2024-02-13T14:23:20.022 INFO:tasks.workunit.client.0.smithi087.stdout: /home/ubuntu/cephtest/clone.client.0/src/test/pybind/test_rados.py:1017: PytestUnknownMarkWarning: Unknown pytest.mark.wait - is this a typo? You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/stable/mark.html 2024-02-13T14:23:20.022 INFO:tasks.workunit.client.0.smithi087.stdout: @pytest.mark.wait 2024-02-13T14:23:20.022 INFO:tasks.workunit.client.0.smithi087.stdout: 2024-02-13T14:23:20.022 INFO:tasks.workunit.client.0.smithi087.stdout::210 2024-02-13T14:23:20.022 INFO:tasks.workunit.client.0.smithi087.stdout::210 2024-02-13T14:23:20.022 INFO:tasks.workunit.client.0.smithi087.stdout::210 2024-02-13T14:23:20.022 INFO:tasks.workunit.client.0.smithi087.stdout::210 2024-02-13T14:23:20.022 INFO:tasks.workunit.client.0.smithi087.stdout::210 2024-02-13T14:23:20.022 INFO:tasks.workunit.client.0.smithi087.stdout::210 2024-02-13T14:23:20.023 INFO:tasks.workunit.client.0.smithi087.stdout::210 2024-02-13T14:23:20.023 INFO:tasks.workunit.client.0.smithi087.stdout::210 2024-02-13T14:23:20.023 INFO:tasks.workunit.client.0.smithi087.stdout::210 2024-02-13T14:23:20.023 INFO:tasks.workunit.client.0.smithi087.stdout: :210: DeprecationWarning: invalid escape sequence \- 2024-02-13T14:23:20.023 INFO:tasks.workunit.client.0.smithi087.stdout: 2024-02-13T14:23:20.023 INFO:tasks.workunit.client.0.smithi087.stdout:-- Docs: https://docs.pytest.org/en/stable/warnings.html 2024-02-13T14:23:20.023 INFO:tasks.workunit.client.0.smithi087.stdout:================= 90 passed, 13 warnings in 324.62s (0:05:24) ================== 2024-02-13T14:23:20.047 INFO:tasks.workunit.client.0.smithi087.stderr:+ exit 0 2024-02-13T14:23:20.047 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-02-13T14:23:20.048 DEBUG:teuthology.orchestra.run.smithi087:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-02-13T14:23:20.081 INFO:tasks.workunit:Stopping ['rados/test_python.sh'] on client.0... 2024-02-13T14:23:20.081 DEBUG:teuthology.orchestra.run.smithi087:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-02-13T14:23:20.388 DEBUG:teuthology.parallel:result is None 2024-02-13T14:23:20.389 DEBUG:teuthology.orchestra.run.smithi087:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-02-13T14:23:20.416 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-02-13T14:23:20.416 DEBUG:teuthology.orchestra.run.smithi087:> rmdir -- /home/ubuntu/cephtest/mnt.0 2024-02-13T14:23:20.481 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2024-02-13T14:23:20.481 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-02-13T14:23:20.492 INFO:tasks.cephadm:Teardown begin 2024-02-13T14:23:20.492 DEBUG:teuthology.orchestra.run.smithi087:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-13T14:23:20.548 DEBUG:teuthology.orchestra.run.smithi148:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-13T14:23:20.586 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-02-13T14:23:20.586 DEBUG:teuthology.orchestra.run.smithi087:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-02-13T14:23:20.603 DEBUG:teuthology.orchestra.run.smithi148:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-02-13T14:23:20.643 INFO:tasks.cephadm:Stopping all daemons... 2024-02-13T14:23:20.644 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-02-13T14:23:20.644 DEBUG:teuthology.orchestra.run.smithi087:> sudo systemctl stop ceph-5c04b074-ca79-11ee-95b9-87774f69a715@mon.a 2024-02-13T14:23:20.972 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:23:20 smithi087 systemd[1]: Stopping Ceph mon.a for 5c04b074-ca79-11ee-95b9-87774f69a715... 2024-02-13T14:23:21.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:23:20 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mon-a[27349]: 2024-02-13T14:23:20.970+0000 7f6ff0a6a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-02-13T14:23:21.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:23:20 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mon-a[27349]: 2024-02-13T14:23:20.970+0000 7f6ff0a6a700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-02-13T14:23:21.242 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:23:21 smithi087 podman[81939]: 2024-02-13 14:23:21.021742465 +0000 UTC m=+0.199517976 container died 1c16bb4c95da4b569ba6767150c9c2d8fdc9b72940450e33defe962120ec75da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mon-a, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.2, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240131, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-13T14:23:21.242 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:23:20 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: [13/Feb/2024:14:23:20] ENGINE Bus STOPPING 2024-02-13T14:23:21.242 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:23:20 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: [13/Feb/2024:14:23:20] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-02-13T14:23:21.242 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:23:20 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: [13/Feb/2024:14:23:20] ENGINE Bus STOPPED 2024-02-13T14:23:21.243 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:23:20 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: [13/Feb/2024:14:23:20] ENGINE Bus STARTING 2024-02-13T14:23:21.243 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:23:21 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: [13/Feb/2024:14:23:21] ENGINE Serving on http://:::9283 2024-02-13T14:23:21.243 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:23:21 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: [13/Feb/2024:14:23:21] ENGINE Bus STARTED 2024-02-13T14:23:21.628 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:23:21 smithi087 podman[81939]: 2024-02-13 14:23:21.400913188 +0000 UTC m=+0.578688699 container cleanup 1c16bb4c95da4b569ba6767150c9c2d8fdc9b72940450e33defe962120ec75da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mon-a, org.label-schema.license=GPLv2, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, org.label-schema.vendor=CentOS, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.2, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240131, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-02-13T14:23:21.628 INFO:journalctl@ceph.mon.a.smithi087.stdout:Feb 13 14:23:21 smithi087 bash[81939]: ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mon-a 2024-02-13T14:23:21.771 DEBUG:teuthology.orchestra.run.smithi087:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5c04b074-ca79-11ee-95b9-87774f69a715@mon.a.service' 2024-02-13T14:23:21.819 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-13T14:23:21.819 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-02-13T14:23:21.819 INFO:tasks.cephadm.mon.b:Stopping mon.c... 2024-02-13T14:23:21.819 DEBUG:teuthology.orchestra.run.smithi087:> sudo systemctl stop ceph-5c04b074-ca79-11ee-95b9-87774f69a715@mon.c 2024-02-13T14:23:21.991 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:23:21 smithi087 systemd[1]: Stopping Ceph mon.c for 5c04b074-ca79-11ee-95b9-87774f69a715... 2024-02-13T14:23:22.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:23:22 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mon-c[39097]: 2024-02-13T14:23:22.124+0000 7fe94e8cf700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-02-13T14:23:22.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:23:22 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mon-c[39097]: 2024-02-13T14:23:22.124+0000 7fe94e8cf700 -1 mon.c@2(peon) e3 *** Got Signal Terminated *** 2024-02-13T14:23:22.492 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:23:22 smithi087 podman[82083]: 2024-02-13 14:23:22.18611075 +0000 UTC m=+0.184928551 container died b5ff86db5b9c08308bc58637575c2c304c52032b6049c3b84a37f665c46ed0b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mon-c, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240131, GIT_CLEAN=True, io.buildah.version=1.33.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-02-13T14:23:22.492 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:23:22 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: [13/Feb/2024:14:23:22] ENGINE Bus STOPPING 2024-02-13T14:23:22.492 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:23:22 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: [13/Feb/2024:14:23:22] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-02-13T14:23:22.492 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:23:22 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: [13/Feb/2024:14:23:22] ENGINE Bus STOPPED 2024-02-13T14:23:22.492 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:23:22 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: [13/Feb/2024:14:23:22] ENGINE Bus STARTING 2024-02-13T14:23:22.492 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:23:22 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: [13/Feb/2024:14:23:22] ENGINE Serving on http://:::9283 2024-02-13T14:23:22.493 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:23:22 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: [13/Feb/2024:14:23:22] ENGINE Bus STARTED 2024-02-13T14:23:22.941 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:23:22 smithi087 podman[82083]: 2024-02-13 14:23:22.6725486 +0000 UTC m=+0.671366406 container cleanup b5ff86db5b9c08308bc58637575c2c304c52032b6049c3b84a37f665c46ed0b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mon-c, GIT_CLEAN=True, org.label-schema.build-date=20240131, GIT_BRANCH=HEAD, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, io.buildah.version=1.33.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.schema-version=1.0) 2024-02-13T14:23:22.942 INFO:journalctl@ceph.mon.c.smithi087.stdout:Feb 13 14:23:22 smithi087 bash[82083]: ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mon-c 2024-02-13T14:23:23.088 DEBUG:teuthology.orchestra.run.smithi087:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5c04b074-ca79-11ee-95b9-87774f69a715@mon.c.service' 2024-02-13T14:23:23.124 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-13T14:23:23.124 INFO:tasks.cephadm.mon.b:Stopped mon.c 2024-02-13T14:23:23.125 INFO:tasks.cephadm.mon.b:Stopping mon.b... 2024-02-13T14:23:23.125 DEBUG:teuthology.orchestra.run.smithi148:> sudo systemctl stop ceph-5c04b074-ca79-11ee-95b9-87774f69a715@mon.b 2024-02-13T14:23:23.427 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:23:23 smithi148 systemd[1]: Stopping Ceph mon.b for 5c04b074-ca79-11ee-95b9-87774f69a715... 2024-02-13T14:23:23.868 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:23:23 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mon-b[31175]: 2024-02-13T14:23:23.425+0000 7f850250d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-02-13T14:23:23.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:23:23 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mon-b[31175]: 2024-02-13T14:23:23.425+0000 7f850250d700 -1 mon.b@1(peon) e3 *** Got Signal Terminated *** 2024-02-13T14:23:23.869 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:23:23 smithi148 podman[67283]: 2024-02-13 14:23:23.621101843 +0000 UTC m=+0.304103995 container died f226490f659a7511676a5098b1702367602d4eb76d1dae56d847f4f67b903696 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mon-b, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, io.buildah.version=1.33.2, org.label-schema.build-date=20240131, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, org.label-schema.schema-version=1.0) 2024-02-13T14:23:24.243 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:23:24 smithi148 podman[67283]: 2024-02-13 14:23:24.015187238 +0000 UTC m=+0.698189388 container cleanup f226490f659a7511676a5098b1702367602d4eb76d1dae56d847f4f67b903696 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mon-b, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240131, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-13T14:23:24.243 INFO:journalctl@ceph.mon.b.smithi148.stdout:Feb 13 14:23:24 smithi148 bash[67283]: ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mon-b 2024-02-13T14:23:24.396 DEBUG:teuthology.orchestra.run.smithi148:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5c04b074-ca79-11ee-95b9-87774f69a715@mon.b.service' 2024-02-13T14:23:24.445 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-13T14:23:24.445 INFO:tasks.cephadm.mon.b:Stopped mon.b 2024-02-13T14:23:24.445 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-02-13T14:23:24.445 DEBUG:teuthology.orchestra.run.smithi087:> sudo systemctl stop ceph-5c04b074-ca79-11ee-95b9-87774f69a715@mgr.y 2024-02-13T14:23:24.738 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:23:24 smithi087 systemd[1]: Stopping Ceph mgr.y for 5c04b074-ca79-11ee-95b9-87774f69a715... 2024-02-13T14:23:24.992 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:23:24 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: 2024-02-13T14:23:24.736+0000 7f2d8d50f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.y -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-13T14:23:24.992 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:23:24 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y[27633]: 2024-02-13T14:23:24.736+0000 7f2d8d50f700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-02-13T14:23:24.992 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:23:24 smithi087 podman[82225]: 2024-02-13 14:23:24.769194598 +0000 UTC m=+0.135532052 container died b3df15cc685cf1b1817b59ce4879a7fd631835234a3af3e093815880bf16935a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y, org.label-schema.build-date=20240131, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_BRANCH=HEAD, io.buildah.version=1.33.2, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-02-13T14:23:25.346 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:23:25 smithi087 podman[82225]: 2024-02-13 14:23:25.109941266 +0000 UTC m=+0.476278765 container cleanup b3df15cc685cf1b1817b59ce4879a7fd631835234a3af3e093815880bf16935a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, org.label-schema.build-date=20240131, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True) 2024-02-13T14:23:25.346 INFO:journalctl@ceph.mgr.y.smithi087.stdout:Feb 13 14:23:25 smithi087 bash[82225]: ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-y 2024-02-13T14:23:25.557 DEBUG:teuthology.orchestra.run.smithi087:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5c04b074-ca79-11ee-95b9-87774f69a715@mgr.y.service' 2024-02-13T14:23:25.594 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-13T14:23:25.595 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-02-13T14:23:25.595 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-02-13T14:23:25.595 DEBUG:teuthology.orchestra.run.smithi148:> sudo systemctl stop ceph-5c04b074-ca79-11ee-95b9-87774f69a715@mgr.x 2024-02-13T14:23:26.021 INFO:journalctl@ceph.mgr.x.smithi148.stdout:Feb 13 14:23:25 smithi148 systemd[1]: Stopping Ceph mgr.x for 5c04b074-ca79-11ee-95b9-87774f69a715... 2024-02-13T14:23:26.368 INFO:journalctl@ceph.mgr.x.smithi148.stdout:Feb 13 14:23:26 smithi148 podman[67414]: 2024-02-13 14:23:26.113429085 +0000 UTC m=+0.328611165 container died a4c89d7bac084a3590523e174031c318270badf7857a44b265729b0e5d1bb273 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-x, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.build-date=20240131, GIT_BRANCH=HEAD, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-13T14:23:26.756 INFO:journalctl@ceph.mgr.x.smithi148.stdout:Feb 13 14:23:26 smithi148 podman[67414]: 2024-02-13 14:23:26.51285129 +0000 UTC m=+0.728033378 container cleanup a4c89d7bac084a3590523e174031c318270badf7857a44b265729b0e5d1bb273 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-x, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, GIT_CLEAN=True, io.buildah.version=1.33.2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240131, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-02-13T14:23:26.756 INFO:journalctl@ceph.mgr.x.smithi148.stdout:Feb 13 14:23:26 smithi148 bash[67414]: ceph-5c04b074-ca79-11ee-95b9-87774f69a715-mgr-x 2024-02-13T14:23:26.887 DEBUG:teuthology.orchestra.run.smithi148:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5c04b074-ca79-11ee-95b9-87774f69a715@mgr.x.service' 2024-02-13T14:23:26.924 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-13T14:23:26.925 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-02-13T14:23:26.925 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-02-13T14:23:26.925 DEBUG:teuthology.orchestra.run.smithi087:> sudo systemctl stop ceph-5c04b074-ca79-11ee-95b9-87774f69a715@osd.0 2024-02-13T14:23:27.235 INFO:journalctl@ceph.osd.0.smithi087.stdout:Feb 13 14:23:26 smithi087 systemd[1]: Stopping Ceph osd.0 for 5c04b074-ca79-11ee-95b9-87774f69a715... 2024-02-13T14:23:27.491 INFO:journalctl@ceph.osd.0.smithi087.stdout:Feb 13 14:23:27 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-0[43500]: 2024-02-13T14:23:27.233+0000 7f0a970f2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-13T14:23:27.492 INFO:journalctl@ceph.osd.0.smithi087.stdout:Feb 13 14:23:27 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-0[43500]: 2024-02-13T14:23:27.233+0000 7f0a970f2700 -1 osd.0 395 *** Got signal Terminated *** 2024-02-13T14:23:27.492 INFO:journalctl@ceph.osd.0.smithi087.stdout:Feb 13 14:23:27 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-0[43500]: 2024-02-13T14:23:27.233+0000 7f0a970f2700 -1 osd.0 395 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-13T14:23:32.900 INFO:journalctl@ceph.osd.0.smithi087.stdout:Feb 13 14:23:32 smithi087 podman[82361]: 2024-02-13 14:23:32.621002276 +0000 UTC m=+5.506102986 container died 966a3358a2cf2e0b9bda521bf84eb95846e6e85a204e3bafb45325d762fbe2d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240131, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275) 2024-02-13T14:23:33.241 INFO:journalctl@ceph.osd.0.smithi087.stdout:Feb 13 14:23:32 smithi087 podman[82361]: 2024-02-13 14:23:32.989079821 +0000 UTC m=+5.874180529 container cleanup 966a3358a2cf2e0b9bda521bf84eb95846e6e85a204e3bafb45325d762fbe2d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-0, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240131, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.2, GIT_BRANCH=HEAD) 2024-02-13T14:23:33.241 INFO:journalctl@ceph.osd.0.smithi087.stdout:Feb 13 14:23:32 smithi087 bash[82361]: ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-0 2024-02-13T14:23:33.491 INFO:journalctl@ceph.osd.0.smithi087.stdout:Feb 13 14:23:33 smithi087 podman[82382]: 2024-02-13 14:23:33.240366294 +0000 UTC m=+0.622492236 container remove 966a3358a2cf2e0b9bda521bf84eb95846e6e85a204e3bafb45325d762fbe2d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-0, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.2, org.label-schema.build-date=20240131, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-02-13T14:23:33.869 INFO:journalctl@ceph.osd.0.smithi087.stdout:Feb 13 14:23:33 smithi087 podman[82484]: 2024-02-13 14:23:33.459954526 +0000 UTC m=+0.018463334 image pull 28480620e5edd298d55c0e6673dcbad9ed23edd37269028a11063c1b8925676a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f 2024-02-13T14:23:33.870 INFO:journalctl@ceph.osd.0.smithi087.stdout:Feb 13 14:23:33 smithi087 podman[82484]: 2024-02-13 14:23:33.566450742 +0000 UTC m=+0.124959547 container create 43f33cc6d327715b9adae9a56e8980add9283c9d170907380716c8cdeb731db6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-0-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.build-date=20240131, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, org.label-schema.schema-version=1.0) 2024-02-13T14:23:33.870 INFO:journalctl@ceph.osd.0.smithi087.stdout:Feb 13 14:23:33 smithi087 podman[82484]: 2024-02-13 14:23:33.777118631 +0000 UTC m=+0.335627445 container init 43f33cc6d327715b9adae9a56e8980add9283c9d170907380716c8cdeb731db6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-0-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240131, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5) 2024-02-13T14:23:33.870 INFO:journalctl@ceph.osd.0.smithi087.stdout:Feb 13 14:23:33 smithi087 podman[82484]: 2024-02-13 14:23:33.782916302 +0000 UTC m=+0.341425111 container start 43f33cc6d327715b9adae9a56e8980add9283c9d170907380716c8cdeb731db6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-0-deactivate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240131, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5) 2024-02-13T14:23:33.870 INFO:journalctl@ceph.osd.0.smithi087.stdout:Feb 13 14:23:33 smithi087 podman[82484]: 2024-02-13 14:23:33.868693561 +0000 UTC m=+0.427202379 container attach 43f33cc6d327715b9adae9a56e8980add9283c9d170907380716c8cdeb731db6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-0-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240131, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.2) 2024-02-13T14:23:34.242 INFO:journalctl@ceph.osd.0.smithi087.stdout:Feb 13 14:23:34 smithi087 podman[82484]: 2024-02-13 14:23:34.068204975 +0000 UTC m=+0.626713786 container died 43f33cc6d327715b9adae9a56e8980add9283c9d170907380716c8cdeb731db6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, ceph=True, org.label-schema.build-date=20240131, org.label-schema.vendor=CentOS, io.buildah.version=1.33.2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5) 2024-02-13T14:23:34.773 DEBUG:teuthology.orchestra.run.smithi087:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5c04b074-ca79-11ee-95b9-87774f69a715@osd.0.service' 2024-02-13T14:23:34.815 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-13T14:23:34.815 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-02-13T14:23:34.815 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-02-13T14:23:34.815 DEBUG:teuthology.orchestra.run.smithi087:> sudo systemctl stop ceph-5c04b074-ca79-11ee-95b9-87774f69a715@osd.1 2024-02-13T14:23:35.137 INFO:journalctl@ceph.osd.1.smithi087.stdout:Feb 13 14:23:34 smithi087 systemd[1]: Stopping Ceph osd.1 for 5c04b074-ca79-11ee-95b9-87774f69a715... 2024-02-13T14:23:35.491 INFO:journalctl@ceph.osd.1.smithi087.stdout:Feb 13 14:23:35 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-1[49621]: 2024-02-13T14:23:35.135+0000 7f213ceff700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-13T14:23:35.492 INFO:journalctl@ceph.osd.1.smithi087.stdout:Feb 13 14:23:35 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-1[49621]: 2024-02-13T14:23:35.135+0000 7f213ceff700 -1 osd.1 395 *** Got signal Terminated *** 2024-02-13T14:23:35.492 INFO:journalctl@ceph.osd.1.smithi087.stdout:Feb 13 14:23:35 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-1[49621]: 2024-02-13T14:23:35.135+0000 7f213ceff700 -1 osd.1 395 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-13T14:23:40.850 INFO:journalctl@ceph.osd.1.smithi087.stdout:Feb 13 14:23:40 smithi087 podman[82616]: 2024-02-13 14:23:40.56846559 +0000 UTC m=+5.530095147 container died e951ec521bd601c0f2b376df5cc27660a12810d1a3c92f6be59774584cd4c406 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-1, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240131) 2024-02-13T14:23:41.242 INFO:journalctl@ceph.osd.1.smithi087.stdout:Feb 13 14:23:40 smithi087 podman[82616]: 2024-02-13 14:23:40.849786043 +0000 UTC m=+5.811415598 container cleanup e951ec521bd601c0f2b376df5cc27660a12810d1a3c92f6be59774584cd4c406 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-1, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240131, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.2, org.label-schema.schema-version=1.0, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5) 2024-02-13T14:23:41.242 INFO:journalctl@ceph.osd.1.smithi087.stdout:Feb 13 14:23:40 smithi087 bash[82616]: ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-1 2024-02-13T14:23:41.242 INFO:journalctl@ceph.osd.1.smithi087.stdout:Feb 13 14:23:41 smithi087 podman[82639]: 2024-02-13 14:23:41.092652866 +0000 UTC m=+0.522781977 container remove e951ec521bd601c0f2b376df5cc27660a12810d1a3c92f6be59774584cd4c406 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-1, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240131, CEPH_POINT_RELEASE=, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-02-13T14:23:41.644 INFO:journalctl@ceph.osd.1.smithi087.stdout:Feb 13 14:23:41 smithi087 podman[82733]: 2024-02-13 14:23:41.291479424 +0000 UTC m=+0.017279679 image pull 28480620e5edd298d55c0e6673dcbad9ed23edd37269028a11063c1b8925676a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f 2024-02-13T14:23:41.644 INFO:journalctl@ceph.osd.1.smithi087.stdout:Feb 13 14:23:41 smithi087 podman[82733]: 2024-02-13 14:23:41.393770547 +0000 UTC m=+0.119570790 container create 0e2251f7965f263c081395af4ecd5fd3429623fe83fbd67fc8e001e8ba72c80c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-1-deactivate, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240131, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-13T14:23:41.644 INFO:journalctl@ceph.osd.1.smithi087.stdout:Feb 13 14:23:41 smithi087 podman[82733]: 2024-02-13 14:23:41.637917778 +0000 UTC m=+0.363718025 container init 0e2251f7965f263c081395af4ecd5fd3429623fe83fbd67fc8e001e8ba72c80c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-1-deactivate, ceph=True, io.buildah.version=1.33.2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240131) 2024-02-13T14:23:41.644 INFO:journalctl@ceph.osd.1.smithi087.stdout:Feb 13 14:23:41 smithi087 podman[82733]: 2024-02-13 14:23:41.643302423 +0000 UTC m=+0.369102668 container start 0e2251f7965f263c081395af4ecd5fd3429623fe83fbd67fc8e001e8ba72c80c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-1-deactivate, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240131, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-13T14:23:41.938 INFO:journalctl@ceph.osd.1.smithi087.stdout:Feb 13 14:23:41 smithi087 podman[82733]: 2024-02-13 14:23:41.746199419 +0000 UTC m=+0.471999685 container attach 0e2251f7965f263c081395af4ecd5fd3429623fe83fbd67fc8e001e8ba72c80c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-1-deactivate, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240131, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275) 2024-02-13T14:23:42.233 INFO:journalctl@ceph.osd.1.smithi087.stdout:Feb 13 14:23:41 smithi087 podman[82733]: 2024-02-13 14:23:41.938072863 +0000 UTC m=+0.663873112 container died 0e2251f7965f263c081395af4ecd5fd3429623fe83fbd67fc8e001e8ba72c80c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-1-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240131, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5) 2024-02-13T14:23:42.488 INFO:journalctl@ceph.osd.1.smithi087.stdout:Feb 13 14:23:42 smithi087 podman[82779]: 2024-02-13 14:23:42.474405416 +0000 UTC m=+0.531437413 container remove 0e2251f7965f263c081395af4ecd5fd3429623fe83fbd67fc8e001e8ba72c80c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.2, org.label-schema.schema-version=1.0, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.build-date=20240131, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-13T14:23:42.496 DEBUG:teuthology.orchestra.run.smithi087:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5c04b074-ca79-11ee-95b9-87774f69a715@osd.1.service' 2024-02-13T14:23:42.537 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-13T14:23:42.537 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-02-13T14:23:42.537 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-02-13T14:23:42.538 DEBUG:teuthology.orchestra.run.smithi087:> sudo systemctl stop ceph-5c04b074-ca79-11ee-95b9-87774f69a715@osd.2 2024-02-13T14:23:42.741 INFO:journalctl@ceph.osd.2.smithi087.stdout:Feb 13 14:23:42 smithi087 systemd[1]: Stopping Ceph osd.2 for 5c04b074-ca79-11ee-95b9-87774f69a715... 2024-02-13T14:23:43.242 INFO:journalctl@ceph.osd.2.smithi087.stdout:Feb 13 14:23:42 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-2[55599]: 2024-02-13T14:23:42.829+0000 7f77c3899700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-13T14:23:43.242 INFO:journalctl@ceph.osd.2.smithi087.stdout:Feb 13 14:23:42 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-2[55599]: 2024-02-13T14:23:42.829+0000 7f77c3899700 -1 osd.2 395 *** Got signal Terminated *** 2024-02-13T14:23:43.242 INFO:journalctl@ceph.osd.2.smithi087.stdout:Feb 13 14:23:42 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-2[55599]: 2024-02-13T14:23:42.829+0000 7f77c3899700 -1 osd.2 395 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-13T14:23:48.421 INFO:journalctl@ceph.osd.2.smithi087.stdout:Feb 13 14:23:48 smithi087 podman[82865]: 2024-02-13 14:23:48.149333682 +0000 UTC m=+5.393448483 container died d9a522f0e71a55509785f2e814d5ae2e2f19fb85910aea4af24b800b78f9e14c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-2, org.label-schema.license=GPLv2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240131, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-02-13T14:23:48.741 INFO:journalctl@ceph.osd.2.smithi087.stdout:Feb 13 14:23:48 smithi087 podman[82865]: 2024-02-13 14:23:48.502283763 +0000 UTC m=+5.746398570 container cleanup d9a522f0e71a55509785f2e814d5ae2e2f19fb85910aea4af24b800b78f9e14c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240131, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5) 2024-02-13T14:23:48.742 INFO:journalctl@ceph.osd.2.smithi087.stdout:Feb 13 14:23:48 smithi087 bash[82865]: ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-2 2024-02-13T14:23:49.108 INFO:journalctl@ceph.osd.2.smithi087.stdout:Feb 13 14:23:48 smithi087 podman[82887]: 2024-02-13 14:23:48.787041591 +0000 UTC m=+0.634370047 container remove d9a522f0e71a55509785f2e814d5ae2e2f19fb85910aea4af24b800b78f9e14c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240131, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, ceph=True, org.label-schema.license=GPLv2) 2024-02-13T14:23:49.374 INFO:journalctl@ceph.osd.2.smithi087.stdout:Feb 13 14:23:49 smithi087 podman[82981]: 2024-02-13 14:23:49.010889073 +0000 UTC m=+0.017940921 image pull 28480620e5edd298d55c0e6673dcbad9ed23edd37269028a11063c1b8925676a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f 2024-02-13T14:23:49.374 INFO:journalctl@ceph.osd.2.smithi087.stdout:Feb 13 14:23:49 smithi087 podman[82981]: 2024-02-13 14:23:49.112683716 +0000 UTC m=+0.119735556 container create 9bb6ffaa32f8ab444df1f51a9b2cf84c9b1652da662bd39ee120e05ae06f0429 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-2-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240131, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.2, GIT_BRANCH=HEAD, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-13T14:23:49.374 INFO:journalctl@ceph.osd.2.smithi087.stdout:Feb 13 14:23:49 smithi087 podman[82981]: 2024-02-13 14:23:49.290030991 +0000 UTC m=+0.297082829 container init 9bb6ffaa32f8ab444df1f51a9b2cf84c9b1652da662bd39ee120e05ae06f0429 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-2-deactivate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240131, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.schema-version=1.0, io.buildah.version=1.33.2, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, GIT_CLEAN=True) 2024-02-13T14:23:49.374 INFO:journalctl@ceph.osd.2.smithi087.stdout:Feb 13 14:23:49 smithi087 podman[82981]: 2024-02-13 14:23:49.295134264 +0000 UTC m=+0.302186107 container start 9bb6ffaa32f8ab444df1f51a9b2cf84c9b1652da662bd39ee120e05ae06f0429 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-2-deactivate, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, org.label-schema.build-date=20240131, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.2, GIT_CLEAN=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275) 2024-02-13T14:23:49.374 INFO:journalctl@ceph.osd.2.smithi087.stdout:Feb 13 14:23:49 smithi087 podman[82981]: 2024-02-13 14:23:49.37312322 +0000 UTC m=+0.380175065 container attach 9bb6ffaa32f8ab444df1f51a9b2cf84c9b1652da662bd39ee120e05ae06f0429 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-2-deactivate, ceph=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240131, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, io.buildah.version=1.33.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-13T14:23:49.742 INFO:journalctl@ceph.osd.2.smithi087.stdout:Feb 13 14:23:49 smithi087 podman[82981]: 2024-02-13 14:23:49.58571182 +0000 UTC m=+0.592763651 container died 9bb6ffaa32f8ab444df1f51a9b2cf84c9b1652da662bd39ee120e05ae06f0429 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-2-deactivate, GIT_CLEAN=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240131, io.buildah.version=1.33.2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-02-13T14:23:50.166 DEBUG:teuthology.orchestra.run.smithi087:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5c04b074-ca79-11ee-95b9-87774f69a715@osd.2.service' 2024-02-13T14:23:50.208 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-13T14:23:50.208 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-02-13T14:23:50.208 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-02-13T14:23:50.208 DEBUG:teuthology.orchestra.run.smithi087:> sudo systemctl stop ceph-5c04b074-ca79-11ee-95b9-87774f69a715@osd.3 2024-02-13T14:23:50.492 INFO:journalctl@ceph.osd.3.smithi087.stdout:Feb 13 14:23:50 smithi087 systemd[1]: Stopping Ceph osd.3 for 5c04b074-ca79-11ee-95b9-87774f69a715... 2024-02-13T14:23:50.992 INFO:journalctl@ceph.osd.3.smithi087.stdout:Feb 13 14:23:50 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-3[61781]: 2024-02-13T14:23:50.540+0000 7fe4692a4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-13T14:23:50.992 INFO:journalctl@ceph.osd.3.smithi087.stdout:Feb 13 14:23:50 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-3[61781]: 2024-02-13T14:23:50.540+0000 7fe4692a4700 -1 osd.3 395 *** Got signal Terminated *** 2024-02-13T14:23:50.992 INFO:journalctl@ceph.osd.3.smithi087.stdout:Feb 13 14:23:50 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-3[61781]: 2024-02-13T14:23:50.540+0000 7fe4692a4700 -1 osd.3 395 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-13T14:23:53.550 INFO:journalctl@ceph.osd.4.smithi148.stdout:Feb 13 14:23:53 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-4[36733]: 2024-02-13T14:23:53.239+0000 7f4ae8d87700 -1 osd.4 395 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-02-13T14:23:30.325417+0000 front 2024-02-13T14:23:30.325463+0000 (oldest deadline 2024-02-13T14:23:53.225223+0000) 2024-02-13T14:23:53.550 INFO:journalctl@ceph.osd.6.smithi148.stdout:Feb 13 14:23:53 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-6[48606]: 2024-02-13T14:23:53.461+0000 7f9ed7433700 -1 osd.6 395 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-02-13T14:23:26.938960+0000 front 2024-02-13T14:23:26.938983+0000 (oldest deadline 2024-02-13T14:23:52.838986+0000) 2024-02-13T14:23:53.868 INFO:journalctl@ceph.osd.5.smithi148.stdout:Feb 13 14:23:53 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-5[42659]: 2024-02-13T14:23:53.547+0000 7f97367bf700 -1 osd.5 395 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-02-13T14:23:31.860731+0000 front 2024-02-13T14:23:31.860700+0000 (oldest deadline 2024-02-13T14:23:52.960576+0000) 2024-02-13T14:23:54.484 INFO:journalctl@ceph.osd.4.smithi148.stdout:Feb 13 14:23:54 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-4[36733]: 2024-02-13T14:23:54.195+0000 7f4ae8d87700 -1 osd.4 395 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-02-13T14:23:30.325417+0000 front 2024-02-13T14:23:30.325463+0000 (oldest deadline 2024-02-13T14:23:53.225223+0000) 2024-02-13T14:23:54.491 INFO:journalctl@ceph.osd.3.smithi087.stdout:Feb 13 14:23:54 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-3[61781]: 2024-02-13T14:23:54.048+0000 7fe461890700 -1 osd.3 395 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-02-13T14:23:29.645856+0000 front 2024-02-13T14:23:29.645841+0000 (oldest deadline 2024-02-13T14:23:53.146008+0000) 2024-02-13T14:23:54.868 INFO:journalctl@ceph.osd.6.smithi148.stdout:Feb 13 14:23:54 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-6[48606]: 2024-02-13T14:23:54.483+0000 7f9ed7433700 -1 osd.6 395 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-02-13T14:23:26.938960+0000 front 2024-02-13T14:23:26.938983+0000 (oldest deadline 2024-02-13T14:23:52.838986+0000) 2024-02-13T14:23:54.869 INFO:journalctl@ceph.osd.5.smithi148.stdout:Feb 13 14:23:54 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-5[42659]: 2024-02-13T14:23:54.522+0000 7f97367bf700 -1 osd.5 395 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-02-13T14:23:31.860731+0000 front 2024-02-13T14:23:31.860700+0000 (oldest deadline 2024-02-13T14:23:52.960576+0000) 2024-02-13T14:23:55.491 INFO:journalctl@ceph.osd.3.smithi087.stdout:Feb 13 14:23:55 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-3[61781]: 2024-02-13T14:23:55.089+0000 7fe461890700 -1 osd.3 395 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-02-13T14:23:29.645856+0000 front 2024-02-13T14:23:29.645841+0000 (oldest deadline 2024-02-13T14:23:53.146008+0000) 2024-02-13T14:23:55.538 INFO:journalctl@ceph.osd.4.smithi148.stdout:Feb 13 14:23:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-4[36733]: 2024-02-13T14:23:55.236+0000 7f4ae8d87700 -1 osd.4 395 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-02-13T14:23:30.325417+0000 front 2024-02-13T14:23:30.325463+0000 (oldest deadline 2024-02-13T14:23:53.225223+0000) 2024-02-13T14:23:55.539 INFO:journalctl@ceph.osd.6.smithi148.stdout:Feb 13 14:23:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-6[48606]: 2024-02-13T14:23:55.459+0000 7f9ed7433700 -1 osd.6 395 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-02-13T14:23:26.938960+0000 front 2024-02-13T14:23:26.938983+0000 (oldest deadline 2024-02-13T14:23:52.838986+0000) 2024-02-13T14:23:55.868 INFO:journalctl@ceph.osd.5.smithi148.stdout:Feb 13 14:23:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-5[42659]: 2024-02-13T14:23:55.560+0000 7f97367bf700 -1 osd.5 395 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-02-13T14:23:31.860731+0000 front 2024-02-13T14:23:31.860700+0000 (oldest deadline 2024-02-13T14:23:52.960576+0000) 2024-02-13T14:23:55.869 INFO:journalctl@ceph.osd.7.smithi148.stdout:Feb 13 14:23:55 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-7[54686]: 2024-02-13T14:23:55.537+0000 7f4025619700 -1 osd.7 395 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-02-13T14:23:32.079364+0000 front 2024-02-13T14:23:32.079377+0000 (oldest deadline 2024-02-13T14:23:55.479861+0000) 2024-02-13T14:23:55.991 INFO:journalctl@ceph.osd.3.smithi087.stdout:Feb 13 14:23:55 smithi087 podman[83109]: 2024-02-13 14:23:55.945598214 +0000 UTC m=+5.520540293 container died 01f90b8487ad579c318ee401191273d7c51a463a5e20839c77a6ad4be3162f49 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-3, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, ceph=True, org.label-schema.build-date=20240131, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.2, maintainer=Guillaume Abrioux ) 2024-02-13T14:23:56.491 INFO:journalctl@ceph.osd.3.smithi087.stdout:Feb 13 14:23:56 smithi087 podman[83109]: 2024-02-13 14:23:56.287204954 +0000 UTC m=+5.862147028 container cleanup 01f90b8487ad579c318ee401191273d7c51a463a5e20839c77a6ad4be3162f49 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-3, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.2, GIT_CLEAN=True, ceph=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.build-date=20240131, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-02-13T14:23:56.491 INFO:journalctl@ceph.osd.3.smithi087.stdout:Feb 13 14:23:56 smithi087 bash[83109]: ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-3 2024-02-13T14:23:56.492 INFO:journalctl@ceph.osd.3.smithi087.stdout:Feb 13 14:23:56 smithi087 podman[83145]: 2024-02-13 14:23:56.44676305 +0000 UTC m=+0.496845100 container remove 01f90b8487ad579c318ee401191273d7c51a463a5e20839c77a6ad4be3162f49 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-3, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240131, org.label-schema.schema-version=1.0) 2024-02-13T14:23:56.549 INFO:journalctl@ceph.osd.4.smithi148.stdout:Feb 13 14:23:56 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-4[36733]: 2024-02-13T14:23:56.267+0000 7f4ae8d87700 -1 osd.4 395 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-02-13T14:23:30.325417+0000 front 2024-02-13T14:23:30.325463+0000 (oldest deadline 2024-02-13T14:23:53.225223+0000) 2024-02-13T14:23:56.549 INFO:journalctl@ceph.osd.6.smithi148.stdout:Feb 13 14:23:56 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-6[48606]: 2024-02-13T14:23:56.439+0000 7f9ed7433700 -1 osd.6 395 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-02-13T14:23:26.938960+0000 front 2024-02-13T14:23:26.938983+0000 (oldest deadline 2024-02-13T14:23:52.838986+0000) 2024-02-13T14:23:56.869 INFO:journalctl@ceph.osd.5.smithi148.stdout:Feb 13 14:23:56 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-5[42659]: 2024-02-13T14:23:56.547+0000 7f97367bf700 -1 osd.5 395 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-02-13T14:23:31.860731+0000 front 2024-02-13T14:23:31.860700+0000 (oldest deadline 2024-02-13T14:23:52.960576+0000) 2024-02-13T14:23:56.869 INFO:journalctl@ceph.osd.7.smithi148.stdout:Feb 13 14:23:56 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-7[54686]: 2024-02-13T14:23:56.583+0000 7f4025619700 -1 osd.7 395 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-02-13T14:23:32.079364+0000 front 2024-02-13T14:23:32.079377+0000 (oldest deadline 2024-02-13T14:23:55.479861+0000) 2024-02-13T14:23:57.125 INFO:journalctl@ceph.osd.3.smithi087.stdout:Feb 13 14:23:56 smithi087 podman[83236]: 2024-02-13 14:23:56.822830445 +0000 UTC m=+0.097005780 container create 36b9b34f5f194ece50d01dea06e7752fcc6ccf3f498e4c3aaf626e0bbc6bed88 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-3-deactivate, io.buildah.version=1.33.2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240131, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, maintainer=Guillaume Abrioux ) 2024-02-13T14:23:57.125 INFO:journalctl@ceph.osd.3.smithi087.stdout:Feb 13 14:23:56 smithi087 podman[83236]: 2024-02-13 14:23:56.74395626 +0000 UTC m=+0.018131595 image pull 28480620e5edd298d55c0e6673dcbad9ed23edd37269028a11063c1b8925676a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f 2024-02-13T14:23:57.125 INFO:journalctl@ceph.osd.3.smithi087.stdout:Feb 13 14:23:57 smithi087 podman[83236]: 2024-02-13 14:23:57.033247549 +0000 UTC m=+0.307422881 container init 36b9b34f5f194ece50d01dea06e7752fcc6ccf3f498e4c3aaf626e0bbc6bed88 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-3-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240131, io.buildah.version=1.33.2, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-02-13T14:23:57.125 INFO:journalctl@ceph.osd.3.smithi087.stdout:Feb 13 14:23:57 smithi087 podman[83236]: 2024-02-13 14:23:57.03897732 +0000 UTC m=+0.313152651 container start 36b9b34f5f194ece50d01dea06e7752fcc6ccf3f498e4c3aaf626e0bbc6bed88 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-3-deactivate, io.buildah.version=1.33.2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, maintainer=Guillaume Abrioux , RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, org.label-schema.build-date=20240131, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-13T14:23:57.126 INFO:journalctl@ceph.osd.3.smithi087.stdout:Feb 13 14:23:57 smithi087 podman[83236]: 2024-02-13 14:23:57.124783792 +0000 UTC m=+0.398959138 container attach 36b9b34f5f194ece50d01dea06e7752fcc6ccf3f498e4c3aaf626e0bbc6bed88 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-3-deactivate, org.label-schema.license=GPLv2, io.buildah.version=1.33.2, GIT_BRANCH=HEAD, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240131, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-02-13T14:23:57.491 INFO:journalctl@ceph.osd.3.smithi087.stdout:Feb 13 14:23:57 smithi087 podman[83236]: 2024-02-13 14:23:57.330603857 +0000 UTC m=+0.604779183 container died 36b9b34f5f194ece50d01dea06e7752fcc6ccf3f498e4c3aaf626e0bbc6bed88 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-3-deactivate, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, io.buildah.version=1.33.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240131, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-13T14:23:57.540 INFO:journalctl@ceph.osd.4.smithi148.stdout:Feb 13 14:23:57 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-4[36733]: 2024-02-13T14:23:57.255+0000 7f4ae8d87700 -1 osd.4 395 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-02-13T14:23:30.325417+0000 front 2024-02-13T14:23:30.325463+0000 (oldest deadline 2024-02-13T14:23:53.225223+0000) 2024-02-13T14:23:57.540 INFO:journalctl@ceph.osd.6.smithi148.stdout:Feb 13 14:23:57 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-6[48606]: 2024-02-13T14:23:57.487+0000 7f9ed7433700 -1 osd.6 395 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-02-13T14:23:26.938960+0000 front 2024-02-13T14:23:26.938983+0000 (oldest deadline 2024-02-13T14:23:52.838986+0000) 2024-02-13T14:23:57.822 DEBUG:teuthology.orchestra.run.smithi087:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5c04b074-ca79-11ee-95b9-87774f69a715@osd.3.service' 2024-02-13T14:23:57.848 INFO:journalctl@ceph.osd.3.smithi087.stdout:Feb 13 14:23:57 smithi087 podman[83282]: 2024-02-13 14:23:57.803030844 +0000 UTC m=+0.467677161 container remove 36b9b34f5f194ece50d01dea06e7752fcc6ccf3f498e4c3aaf626e0bbc6bed88 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-3-deactivate, org.label-schema.build-date=20240131, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-13T14:23:57.848 INFO:journalctl@ceph.osd.3.smithi087.stdout:Feb 13 14:23:57 smithi087 systemd[1]: ceph-5c04b074-ca79-11ee-95b9-87774f69a715@osd.3.service: Deactivated successfully. 2024-02-13T14:23:57.848 INFO:journalctl@ceph.osd.3.smithi087.stdout:Feb 13 14:23:57 smithi087 systemd[1]: Stopped Ceph osd.3 for 5c04b074-ca79-11ee-95b9-87774f69a715. 2024-02-13T14:23:57.848 INFO:journalctl@ceph.osd.3.smithi087.stdout:Feb 13 14:23:57 smithi087 systemd[1]: ceph-5c04b074-ca79-11ee-95b9-87774f69a715@osd.3.service: Consumed 15.903s CPU time. 2024-02-13T14:23:57.862 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-13T14:23:57.862 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-02-13T14:23:57.862 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-02-13T14:23:57.862 DEBUG:teuthology.orchestra.run.smithi148:> sudo systemctl stop ceph-5c04b074-ca79-11ee-95b9-87774f69a715@osd.4 2024-02-13T14:23:57.868 INFO:journalctl@ceph.osd.5.smithi148.stdout:Feb 13 14:23:57 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-5[42659]: 2024-02-13T14:23:57.539+0000 7f97367bf700 -1 osd.5 395 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-02-13T14:23:31.860731+0000 front 2024-02-13T14:23:31.860700+0000 (oldest deadline 2024-02-13T14:23:52.960576+0000) 2024-02-13T14:23:57.868 INFO:journalctl@ceph.osd.7.smithi148.stdout:Feb 13 14:23:57 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-7[54686]: 2024-02-13T14:23:57.592+0000 7f4025619700 -1 osd.7 395 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-02-13T14:23:32.079364+0000 front 2024-02-13T14:23:32.079377+0000 (oldest deadline 2024-02-13T14:23:55.479861+0000) 2024-02-13T14:23:58.159 INFO:journalctl@ceph.osd.4.smithi148.stdout:Feb 13 14:23:57 smithi148 systemd[1]: Stopping Ceph osd.4 for 5c04b074-ca79-11ee-95b9-87774f69a715... 2024-02-13T14:23:58.524 INFO:journalctl@ceph.osd.4.smithi148.stdout:Feb 13 14:23:58 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-4[36733]: 2024-02-13T14:23:58.157+0000 7f4af079b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-13T14:23:58.524 INFO:journalctl@ceph.osd.4.smithi148.stdout:Feb 13 14:23:58 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-4[36733]: 2024-02-13T14:23:58.157+0000 7f4af079b700 -1 osd.4 395 *** Got signal Terminated *** 2024-02-13T14:23:58.524 INFO:journalctl@ceph.osd.4.smithi148.stdout:Feb 13 14:23:58 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-4[36733]: 2024-02-13T14:23:58.157+0000 7f4af079b700 -1 osd.4 395 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-13T14:23:58.524 INFO:journalctl@ceph.osd.4.smithi148.stdout:Feb 13 14:23:58 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-4[36733]: 2024-02-13T14:23:58.284+0000 7f4ae8d87700 -1 osd.4 395 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-02-13T14:23:30.325417+0000 front 2024-02-13T14:23:30.325463+0000 (oldest deadline 2024-02-13T14:23:53.225223+0000) 2024-02-13T14:23:58.868 INFO:journalctl@ceph.osd.7.smithi148.stdout:Feb 13 14:23:58 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-7[54686]: 2024-02-13T14:23:58.554+0000 7f4025619700 -1 osd.7 395 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-02-13T14:23:32.079364+0000 front 2024-02-13T14:23:32.079377+0000 (oldest deadline 2024-02-13T14:23:55.479861+0000) 2024-02-13T14:23:58.869 INFO:journalctl@ceph.osd.5.smithi148.stdout:Feb 13 14:23:58 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-5[42659]: 2024-02-13T14:23:58.533+0000 7f97367bf700 -1 osd.5 395 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-02-13T14:23:31.860731+0000 front 2024-02-13T14:23:31.860700+0000 (oldest deadline 2024-02-13T14:23:52.960576+0000) 2024-02-13T14:23:58.869 INFO:journalctl@ceph.osd.6.smithi148.stdout:Feb 13 14:23:58 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-6[48606]: 2024-02-13T14:23:58.523+0000 7f9ed7433700 -1 osd.6 395 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-02-13T14:23:26.938960+0000 front 2024-02-13T14:23:26.938983+0000 (oldest deadline 2024-02-13T14:23:52.838986+0000) 2024-02-13T14:23:59.618 INFO:journalctl@ceph.osd.4.smithi148.stdout:Feb 13 14:23:59 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-4[36733]: 2024-02-13T14:23:59.308+0000 7f4ae8d87700 -1 osd.4 395 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-02-13T14:23:30.325417+0000 front 2024-02-13T14:23:30.325463+0000 (oldest deadline 2024-02-13T14:23:53.225223+0000) 2024-02-13T14:23:59.619 INFO:journalctl@ceph.osd.5.smithi148.stdout:Feb 13 14:23:59 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-5[42659]: 2024-02-13T14:23:59.551+0000 7f97367bf700 -1 osd.5 395 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-02-13T14:23:31.860731+0000 front 2024-02-13T14:23:31.860700+0000 (oldest deadline 2024-02-13T14:23:52.960576+0000) 2024-02-13T14:23:59.619 INFO:journalctl@ceph.osd.6.smithi148.stdout:Feb 13 14:23:59 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-6[48606]: 2024-02-13T14:23:59.526+0000 7f9ed7433700 -1 osd.6 395 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-02-13T14:23:26.938960+0000 front 2024-02-13T14:23:26.938983+0000 (oldest deadline 2024-02-13T14:23:52.838986+0000) 2024-02-13T14:23:59.619 INFO:journalctl@ceph.osd.7.smithi148.stdout:Feb 13 14:23:59 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-7[54686]: 2024-02-13T14:23:59.530+0000 7f4025619700 -1 osd.7 395 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-02-13T14:23:32.079364+0000 front 2024-02-13T14:23:32.079377+0000 (oldest deadline 2024-02-13T14:23:55.479861+0000) 2024-02-13T14:24:00.530 INFO:journalctl@ceph.osd.4.smithi148.stdout:Feb 13 14:24:00 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-4[36733]: 2024-02-13T14:24:00.266+0000 7f4ae8d87700 -1 osd.4 395 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-02-13T14:23:30.325417+0000 front 2024-02-13T14:23:30.325463+0000 (oldest deadline 2024-02-13T14:23:53.225223+0000) 2024-02-13T14:24:00.868 INFO:journalctl@ceph.osd.5.smithi148.stdout:Feb 13 14:24:00 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-5[42659]: 2024-02-13T14:24:00.594+0000 7f97367bf700 -1 osd.5 395 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-02-13T14:23:31.860731+0000 front 2024-02-13T14:23:31.860700+0000 (oldest deadline 2024-02-13T14:23:52.960576+0000) 2024-02-13T14:24:00.869 INFO:journalctl@ceph.osd.6.smithi148.stdout:Feb 13 14:24:00 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-6[48606]: 2024-02-13T14:24:00.528+0000 7f9ed7433700 -1 osd.6 395 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-02-13T14:23:26.938960+0000 front 2024-02-13T14:23:26.938983+0000 (oldest deadline 2024-02-13T14:23:52.838986+0000) 2024-02-13T14:24:00.869 INFO:journalctl@ceph.osd.7.smithi148.stdout:Feb 13 14:24:00 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-7[54686]: 2024-02-13T14:24:00.538+0000 7f4025619700 -1 osd.7 395 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-02-13T14:23:32.079364+0000 front 2024-02-13T14:23:32.079377+0000 (oldest deadline 2024-02-13T14:23:55.479861+0000) 2024-02-13T14:24:01.561 INFO:journalctl@ceph.osd.6.smithi148.stdout:Feb 13 14:24:01 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-6[48606]: 2024-02-13T14:24:01.534+0000 7f9ed7433700 -1 osd.6 395 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-02-13T14:23:26.938960+0000 front 2024-02-13T14:23:26.938983+0000 (oldest deadline 2024-02-13T14:23:52.838986+0000) 2024-02-13T14:24:01.561 INFO:journalctl@ceph.osd.4.smithi148.stdout:Feb 13 14:24:01 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-4[36733]: 2024-02-13T14:24:01.286+0000 7f4ae8d87700 -1 osd.4 395 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-02-13T14:23:30.325417+0000 front 2024-02-13T14:23:30.325463+0000 (oldest deadline 2024-02-13T14:23:53.225223+0000) 2024-02-13T14:24:01.562 INFO:journalctl@ceph.osd.4.smithi148.stdout:Feb 13 14:24:01 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-4[36733]: 2024-02-13T14:24:01.287+0000 7f4ae8d87700 -1 osd.4 395 heartbeat_check: no reply from 172.21.15.87:6814 osd.1 since back 2024-02-13T14:23:39.126308+0000 front 2024-02-13T14:23:39.126324+0000 (oldest deadline 2024-02-13T14:24:00.826194+0000) 2024-02-13T14:24:01.562 INFO:journalctl@ceph.osd.7.smithi148.stdout:Feb 13 14:24:01 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-7[54686]: 2024-02-13T14:24:01.531+0000 7f4025619700 -1 osd.7 395 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-02-13T14:23:32.079364+0000 front 2024-02-13T14:23:32.079377+0000 (oldest deadline 2024-02-13T14:23:55.479861+0000) 2024-02-13T14:24:01.562 INFO:journalctl@ceph.osd.7.smithi148.stdout:Feb 13 14:24:01 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-7[54686]: 2024-02-13T14:24:01.531+0000 7f4025619700 -1 osd.7 395 heartbeat_check: no reply from 172.21.15.87:6814 osd.1 since back 2024-02-13T14:23:35.480357+0000 front 2024-02-13T14:23:35.480420+0000 (oldest deadline 2024-02-13T14:24:01.380377+0000) 2024-02-13T14:24:01.868 INFO:journalctl@ceph.osd.5.smithi148.stdout:Feb 13 14:24:01 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-5[42659]: 2024-02-13T14:24:01.559+0000 7f97367bf700 -1 osd.5 395 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-02-13T14:23:31.860731+0000 front 2024-02-13T14:23:31.860700+0000 (oldest deadline 2024-02-13T14:23:52.960576+0000) 2024-02-13T14:24:02.618 INFO:journalctl@ceph.osd.6.smithi148.stdout:Feb 13 14:24:02 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-6[48606]: 2024-02-13T14:24:02.524+0000 7f9ed7433700 -1 osd.6 395 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-02-13T14:23:26.938960+0000 front 2024-02-13T14:23:26.938983+0000 (oldest deadline 2024-02-13T14:23:52.838986+0000) 2024-02-13T14:24:02.619 INFO:journalctl@ceph.osd.4.smithi148.stdout:Feb 13 14:24:02 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-4[36733]: 2024-02-13T14:24:02.332+0000 7f4ae8d87700 -1 osd.4 395 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-02-13T14:23:30.325417+0000 front 2024-02-13T14:23:30.325463+0000 (oldest deadline 2024-02-13T14:23:53.225223+0000) 2024-02-13T14:24:02.619 INFO:journalctl@ceph.osd.4.smithi148.stdout:Feb 13 14:24:02 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-4[36733]: 2024-02-13T14:24:02.332+0000 7f4ae8d87700 -1 osd.4 395 heartbeat_check: no reply from 172.21.15.87:6814 osd.1 since back 2024-02-13T14:23:39.126308+0000 front 2024-02-13T14:23:39.126324+0000 (oldest deadline 2024-02-13T14:24:00.826194+0000) 2024-02-13T14:24:02.619 INFO:journalctl@ceph.osd.5.smithi148.stdout:Feb 13 14:24:02 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-5[42659]: 2024-02-13T14:24:02.560+0000 7f97367bf700 -1 osd.5 395 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-02-13T14:23:31.860731+0000 front 2024-02-13T14:23:31.860700+0000 (oldest deadline 2024-02-13T14:23:52.960576+0000) 2024-02-13T14:24:02.619 INFO:journalctl@ceph.osd.5.smithi148.stdout:Feb 13 14:24:02 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-5[42659]: 2024-02-13T14:24:02.560+0000 7f97367bf700 -1 osd.5 395 heartbeat_check: no reply from 172.21.15.87:6814 osd.1 since back 2024-02-13T14:23:39.962085+0000 front 2024-02-13T14:23:39.961987+0000 (oldest deadline 2024-02-13T14:24:01.661991+0000) 2024-02-13T14:24:02.619 INFO:journalctl@ceph.osd.7.smithi148.stdout:Feb 13 14:24:02 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-7[54686]: 2024-02-13T14:24:02.514+0000 7f4025619700 -1 osd.7 395 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-02-13T14:23:32.079364+0000 front 2024-02-13T14:23:32.079377+0000 (oldest deadline 2024-02-13T14:23:55.479861+0000) 2024-02-13T14:24:02.619 INFO:journalctl@ceph.osd.7.smithi148.stdout:Feb 13 14:24:02 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-7[54686]: 2024-02-13T14:24:02.514+0000 7f4025619700 -1 osd.7 395 heartbeat_check: no reply from 172.21.15.87:6814 osd.1 since back 2024-02-13T14:23:35.480357+0000 front 2024-02-13T14:23:35.480420+0000 (oldest deadline 2024-02-13T14:24:01.380377+0000) 2024-02-13T14:24:03.849 INFO:journalctl@ceph.osd.4.smithi148.stdout:Feb 13 14:24:03 smithi148 podman[67550]: 2024-02-13 14:24:03.611666846 +0000 UTC m=+5.557356877 container died cf2ef7fce9c073733d50814408f16f754d5e8ebd9c871d116a9ee3ad946b5d8b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-4, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240131, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.2, maintainer=Guillaume Abrioux , RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275) 2024-02-13T14:24:03.849 INFO:journalctl@ceph.osd.5.smithi148.stdout:Feb 13 14:24:03 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-5[42659]: 2024-02-13T14:24:03.525+0000 7f97367bf700 -1 osd.5 395 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-02-13T14:23:31.860731+0000 front 2024-02-13T14:23:31.860700+0000 (oldest deadline 2024-02-13T14:23:52.960576+0000) 2024-02-13T14:24:03.849 INFO:journalctl@ceph.osd.5.smithi148.stdout:Feb 13 14:24:03 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-5[42659]: 2024-02-13T14:24:03.525+0000 7f97367bf700 -1 osd.5 395 heartbeat_check: no reply from 172.21.15.87:6814 osd.1 since back 2024-02-13T14:23:39.962085+0000 front 2024-02-13T14:23:39.961987+0000 (oldest deadline 2024-02-13T14:24:01.661991+0000) 2024-02-13T14:24:03.850 INFO:journalctl@ceph.osd.6.smithi148.stdout:Feb 13 14:24:03 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-6[48606]: 2024-02-13T14:24:03.500+0000 7f9ed7433700 -1 osd.6 395 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-02-13T14:23:26.938960+0000 front 2024-02-13T14:23:26.938983+0000 (oldest deadline 2024-02-13T14:23:52.838986+0000) 2024-02-13T14:24:03.850 INFO:journalctl@ceph.osd.6.smithi148.stdout:Feb 13 14:24:03 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-6[48606]: 2024-02-13T14:24:03.500+0000 7f9ed7433700 -1 osd.6 395 heartbeat_check: no reply from 172.21.15.87:6814 osd.1 since back 2024-02-13T14:23:39.741076+0000 front 2024-02-13T14:23:39.741053+0000 (oldest deadline 2024-02-13T14:24:02.541342+0000) 2024-02-13T14:24:03.850 INFO:journalctl@ceph.osd.7.smithi148.stdout:Feb 13 14:24:03 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-7[54686]: 2024-02-13T14:24:03.499+0000 7f4025619700 -1 osd.7 395 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-02-13T14:23:32.079364+0000 front 2024-02-13T14:23:32.079377+0000 (oldest deadline 2024-02-13T14:23:55.479861+0000) 2024-02-13T14:24:03.850 INFO:journalctl@ceph.osd.7.smithi148.stdout:Feb 13 14:24:03 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-7[54686]: 2024-02-13T14:24:03.500+0000 7f4025619700 -1 osd.7 395 heartbeat_check: no reply from 172.21.15.87:6814 osd.1 since back 2024-02-13T14:23:35.480357+0000 front 2024-02-13T14:23:35.480420+0000 (oldest deadline 2024-02-13T14:24:01.380377+0000) 2024-02-13T14:24:03.991 INFO:journalctl@ceph.alertmanager.a.smithi087.stdout:Feb 13 14:24:03 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-alertmanager-a[78198]: ts=2024-02-13T14:24:03.531Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi087.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.87:8443: connect: connection refused" 2024-02-13T14:24:03.992 INFO:journalctl@ceph.alertmanager.a.smithi087.stdout:Feb 13 14:24:03 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-alertmanager-a[78198]: ts=2024-02-13T14:24:03.532Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi148.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.148:8443: connect: connection refused" 2024-02-13T14:24:04.118 INFO:journalctl@ceph.osd.4.smithi148.stdout:Feb 13 14:24:03 smithi148 podman[67550]: 2024-02-13 14:24:03.9385604 +0000 UTC m=+5.884250431 container cleanup cf2ef7fce9c073733d50814408f16f754d5e8ebd9c871d116a9ee3ad946b5d8b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-4, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, org.label-schema.build-date=20240131, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275) 2024-02-13T14:24:04.119 INFO:journalctl@ceph.osd.4.smithi148.stdout:Feb 13 14:24:03 smithi148 bash[67550]: ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-4 2024-02-13T14:24:04.492 INFO:journalctl@ceph.osd.4.smithi148.stdout:Feb 13 14:24:04 smithi148 podman[67574]: 2024-02-13 14:24:04.123093875 +0000 UTC m=+0.512759132 container remove cf2ef7fce9c073733d50814408f16f754d5e8ebd9c871d116a9ee3ad946b5d8b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-4, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.2, CEPH_POINT_RELEASE=, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240131, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-02-13T14:24:04.802 INFO:journalctl@ceph.osd.5.smithi148.stdout:Feb 13 14:24:04 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-5[42659]: 2024-02-13T14:24:04.560+0000 7f97367bf700 -1 osd.5 395 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-02-13T14:23:31.860731+0000 front 2024-02-13T14:23:31.860700+0000 (oldest deadline 2024-02-13T14:23:52.960576+0000) 2024-02-13T14:24:04.802 INFO:journalctl@ceph.osd.5.smithi148.stdout:Feb 13 14:24:04 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-5[42659]: 2024-02-13T14:24:04.560+0000 7f97367bf700 -1 osd.5 395 heartbeat_check: no reply from 172.21.15.87:6814 osd.1 since back 2024-02-13T14:23:39.962085+0000 front 2024-02-13T14:23:39.961987+0000 (oldest deadline 2024-02-13T14:24:01.661991+0000) 2024-02-13T14:24:04.803 INFO:journalctl@ceph.osd.6.smithi148.stdout:Feb 13 14:24:04 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-6[48606]: 2024-02-13T14:24:04.490+0000 7f9ed7433700 -1 osd.6 395 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-02-13T14:23:26.938960+0000 front 2024-02-13T14:23:26.938983+0000 (oldest deadline 2024-02-13T14:23:52.838986+0000) 2024-02-13T14:24:04.803 INFO:journalctl@ceph.osd.6.smithi148.stdout:Feb 13 14:24:04 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-6[48606]: 2024-02-13T14:24:04.490+0000 7f9ed7433700 -1 osd.6 395 heartbeat_check: no reply from 172.21.15.87:6814 osd.1 since back 2024-02-13T14:23:39.741076+0000 front 2024-02-13T14:23:39.741053+0000 (oldest deadline 2024-02-13T14:24:02.541342+0000) 2024-02-13T14:24:04.803 INFO:journalctl@ceph.osd.4.smithi148.stdout:Feb 13 14:24:04 smithi148 podman[67673]: 2024-02-13 14:24:04.426137223 +0000 UTC m=+0.017855561 image pull 28480620e5edd298d55c0e6673dcbad9ed23edd37269028a11063c1b8925676a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f 2024-02-13T14:24:04.803 INFO:journalctl@ceph.osd.4.smithi148.stdout:Feb 13 14:24:04 smithi148 podman[67673]: 2024-02-13 14:24:04.524025766 +0000 UTC m=+0.115744097 container create 27fbc67ff241b6e3b12aeeaecba27924965ecfbe3567301c17dd075a40b57390 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-4-deactivate, org.label-schema.build-date=20240131, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, io.buildah.version=1.33.2) 2024-02-13T14:24:04.803 INFO:journalctl@ceph.osd.7.smithi148.stdout:Feb 13 14:24:04 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-7[54686]: 2024-02-13T14:24:04.501+0000 7f4025619700 -1 osd.7 395 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-02-13T14:23:32.079364+0000 front 2024-02-13T14:23:32.079377+0000 (oldest deadline 2024-02-13T14:23:55.479861+0000) 2024-02-13T14:24:04.803 INFO:journalctl@ceph.osd.7.smithi148.stdout:Feb 13 14:24:04 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-7[54686]: 2024-02-13T14:24:04.501+0000 7f4025619700 -1 osd.7 395 heartbeat_check: no reply from 172.21.15.87:6814 osd.1 since back 2024-02-13T14:23:35.480357+0000 front 2024-02-13T14:23:35.480420+0000 (oldest deadline 2024-02-13T14:24:01.380377+0000) 2024-02-13T14:24:05.103 INFO:journalctl@ceph.osd.4.smithi148.stdout:Feb 13 14:24:04 smithi148 podman[67673]: 2024-02-13 14:24:04.801604667 +0000 UTC m=+0.393322997 container init 27fbc67ff241b6e3b12aeeaecba27924965ecfbe3567301c17dd075a40b57390 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-4-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240131, io.buildah.version=1.33.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-13T14:24:05.104 INFO:journalctl@ceph.osd.4.smithi148.stdout:Feb 13 14:24:04 smithi148 podman[67673]: 2024-02-13 14:24:04.807460414 +0000 UTC m=+0.399178744 container start 27fbc67ff241b6e3b12aeeaecba27924965ecfbe3567301c17dd075a40b57390 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-4-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, io.buildah.version=1.33.2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240131, org.label-schema.license=GPLv2, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, ceph=True) 2024-02-13T14:24:05.104 INFO:journalctl@ceph.osd.4.smithi148.stdout:Feb 13 14:24:04 smithi148 podman[67673]: 2024-02-13 14:24:04.893180917 +0000 UTC m=+0.484899250 container attach 27fbc67ff241b6e3b12aeeaecba27924965ecfbe3567301c17dd075a40b57390 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-4-deactivate, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240131, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.2, org.label-schema.vendor=CentOS) 2024-02-13T14:24:05.368 INFO:journalctl@ceph.osd.4.smithi148.stdout:Feb 13 14:24:05 smithi148 podman[67673]: 2024-02-13 14:24:05.103460224 +0000 UTC m=+0.695178548 container died 27fbc67ff241b6e3b12aeeaecba27924965ecfbe3567301c17dd075a40b57390 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-4-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240131, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_BRANCH=HEAD, io.buildah.version=1.33.2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-02-13T14:24:05.672 INFO:journalctl@ceph.osd.5.smithi148.stdout:Feb 13 14:24:05 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-5[42659]: 2024-02-13T14:24:05.516+0000 7f97367bf700 -1 osd.5 395 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-02-13T14:23:31.860731+0000 front 2024-02-13T14:23:31.860700+0000 (oldest deadline 2024-02-13T14:23:52.960576+0000) 2024-02-13T14:24:05.672 INFO:journalctl@ceph.osd.5.smithi148.stdout:Feb 13 14:24:05 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-5[42659]: 2024-02-13T14:24:05.516+0000 7f97367bf700 -1 osd.5 395 heartbeat_check: no reply from 172.21.15.87:6814 osd.1 since back 2024-02-13T14:23:39.962085+0000 front 2024-02-13T14:23:39.961987+0000 (oldest deadline 2024-02-13T14:24:01.661991+0000) 2024-02-13T14:24:05.672 INFO:journalctl@ceph.osd.6.smithi148.stdout:Feb 13 14:24:05 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-6[48606]: 2024-02-13T14:24:05.467+0000 7f9ed7433700 -1 osd.6 395 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-02-13T14:23:26.938960+0000 front 2024-02-13T14:23:26.938983+0000 (oldest deadline 2024-02-13T14:23:52.838986+0000) 2024-02-13T14:24:05.673 INFO:journalctl@ceph.osd.6.smithi148.stdout:Feb 13 14:24:05 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-6[48606]: 2024-02-13T14:24:05.467+0000 7f9ed7433700 -1 osd.6 395 heartbeat_check: no reply from 172.21.15.87:6814 osd.1 since back 2024-02-13T14:23:39.741076+0000 front 2024-02-13T14:23:39.741053+0000 (oldest deadline 2024-02-13T14:24:02.541342+0000) 2024-02-13T14:24:05.673 INFO:journalctl@ceph.osd.7.smithi148.stdout:Feb 13 14:24:05 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-7[54686]: 2024-02-13T14:24:05.511+0000 7f4025619700 -1 osd.7 395 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-02-13T14:23:32.079364+0000 front 2024-02-13T14:23:32.079377+0000 (oldest deadline 2024-02-13T14:23:55.479861+0000) 2024-02-13T14:24:05.673 INFO:journalctl@ceph.osd.7.smithi148.stdout:Feb 13 14:24:05 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-7[54686]: 2024-02-13T14:24:05.511+0000 7f4025619700 -1 osd.7 395 heartbeat_check: no reply from 172.21.15.87:6814 osd.1 since back 2024-02-13T14:23:35.480357+0000 front 2024-02-13T14:23:35.480420+0000 (oldest deadline 2024-02-13T14:24:01.380377+0000) 2024-02-13T14:24:05.691 DEBUG:teuthology.orchestra.run.smithi148:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5c04b074-ca79-11ee-95b9-87774f69a715@osd.4.service' 2024-02-13T14:24:05.774 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-13T14:24:05.774 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-02-13T14:24:05.774 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-02-13T14:24:05.774 DEBUG:teuthology.orchestra.run.smithi148:> sudo systemctl stop ceph-5c04b074-ca79-11ee-95b9-87774f69a715@osd.5 2024-02-13T14:24:06.111 INFO:journalctl@ceph.osd.5.smithi148.stdout:Feb 13 14:24:05 smithi148 systemd[1]: Stopping Ceph osd.5 for 5c04b074-ca79-11ee-95b9-87774f69a715... 2024-02-13T14:24:06.368 INFO:journalctl@ceph.osd.5.smithi148.stdout:Feb 13 14:24:06 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-5[42659]: 2024-02-13T14:24:06.109+0000 7f973e1d3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-13T14:24:06.368 INFO:journalctl@ceph.osd.5.smithi148.stdout:Feb 13 14:24:06 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-5[42659]: 2024-02-13T14:24:06.109+0000 7f973e1d3700 -1 osd.5 395 *** Got signal Terminated *** 2024-02-13T14:24:06.369 INFO:journalctl@ceph.osd.5.smithi148.stdout:Feb 13 14:24:06 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-5[42659]: 2024-02-13T14:24:06.109+0000 7f973e1d3700 -1 osd.5 395 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-13T14:24:06.869 INFO:journalctl@ceph.osd.5.smithi148.stdout:Feb 13 14:24:06 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-5[42659]: 2024-02-13T14:24:06.544+0000 7f97367bf700 -1 osd.5 395 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-02-13T14:23:31.860731+0000 front 2024-02-13T14:23:31.860700+0000 (oldest deadline 2024-02-13T14:23:52.960576+0000) 2024-02-13T14:24:06.869 INFO:journalctl@ceph.osd.5.smithi148.stdout:Feb 13 14:24:06 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-5[42659]: 2024-02-13T14:24:06.544+0000 7f97367bf700 -1 osd.5 395 heartbeat_check: no reply from 172.21.15.87:6814 osd.1 since back 2024-02-13T14:23:39.962085+0000 front 2024-02-13T14:23:39.961987+0000 (oldest deadline 2024-02-13T14:24:01.661991+0000) 2024-02-13T14:24:06.869 INFO:journalctl@ceph.osd.6.smithi148.stdout:Feb 13 14:24:06 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-6[48606]: 2024-02-13T14:24:06.422+0000 7f9ed7433700 -1 osd.6 395 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-02-13T14:23:26.938960+0000 front 2024-02-13T14:23:26.938983+0000 (oldest deadline 2024-02-13T14:23:52.838986+0000) 2024-02-13T14:24:06.869 INFO:journalctl@ceph.osd.6.smithi148.stdout:Feb 13 14:24:06 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-6[48606]: 2024-02-13T14:24:06.422+0000 7f9ed7433700 -1 osd.6 395 heartbeat_check: no reply from 172.21.15.87:6814 osd.1 since back 2024-02-13T14:23:39.741076+0000 front 2024-02-13T14:23:39.741053+0000 (oldest deadline 2024-02-13T14:24:02.541342+0000) 2024-02-13T14:24:06.869 INFO:journalctl@ceph.osd.7.smithi148.stdout:Feb 13 14:24:06 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-7[54686]: 2024-02-13T14:24:06.512+0000 7f4025619700 -1 osd.7 395 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-02-13T14:23:32.079364+0000 front 2024-02-13T14:23:32.079377+0000 (oldest deadline 2024-02-13T14:23:55.479861+0000) 2024-02-13T14:24:06.870 INFO:journalctl@ceph.osd.7.smithi148.stdout:Feb 13 14:24:06 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-7[54686]: 2024-02-13T14:24:06.512+0000 7f4025619700 -1 osd.7 395 heartbeat_check: no reply from 172.21.15.87:6814 osd.1 since back 2024-02-13T14:23:35.480357+0000 front 2024-02-13T14:23:35.480420+0000 (oldest deadline 2024-02-13T14:24:01.380377+0000) 2024-02-13T14:24:07.868 INFO:journalctl@ceph.osd.5.smithi148.stdout:Feb 13 14:24:07 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-5[42659]: 2024-02-13T14:24:07.534+0000 7f97367bf700 -1 osd.5 395 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-02-13T14:23:31.860731+0000 front 2024-02-13T14:23:31.860700+0000 (oldest deadline 2024-02-13T14:23:52.960576+0000) 2024-02-13T14:24:07.869 INFO:journalctl@ceph.osd.5.smithi148.stdout:Feb 13 14:24:07 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-5[42659]: 2024-02-13T14:24:07.534+0000 7f97367bf700 -1 osd.5 395 heartbeat_check: no reply from 172.21.15.87:6814 osd.1 since back 2024-02-13T14:23:39.962085+0000 front 2024-02-13T14:23:39.961987+0000 (oldest deadline 2024-02-13T14:24:01.661991+0000) 2024-02-13T14:24:07.869 INFO:journalctl@ceph.osd.6.smithi148.stdout:Feb 13 14:24:07 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-6[48606]: 2024-02-13T14:24:07.391+0000 7f9ed7433700 -1 osd.6 395 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-02-13T14:23:26.938960+0000 front 2024-02-13T14:23:26.938983+0000 (oldest deadline 2024-02-13T14:23:52.838986+0000) 2024-02-13T14:24:07.869 INFO:journalctl@ceph.osd.6.smithi148.stdout:Feb 13 14:24:07 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-6[48606]: 2024-02-13T14:24:07.391+0000 7f9ed7433700 -1 osd.6 395 heartbeat_check: no reply from 172.21.15.87:6814 osd.1 since back 2024-02-13T14:23:39.741076+0000 front 2024-02-13T14:23:39.741053+0000 (oldest deadline 2024-02-13T14:24:02.541342+0000) 2024-02-13T14:24:07.869 INFO:journalctl@ceph.osd.7.smithi148.stdout:Feb 13 14:24:07 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-7[54686]: 2024-02-13T14:24:07.497+0000 7f4025619700 -1 osd.7 395 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-02-13T14:23:32.079364+0000 front 2024-02-13T14:23:32.079377+0000 (oldest deadline 2024-02-13T14:23:55.479861+0000) 2024-02-13T14:24:07.869 INFO:journalctl@ceph.osd.7.smithi148.stdout:Feb 13 14:24:07 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-7[54686]: 2024-02-13T14:24:07.497+0000 7f4025619700 -1 osd.7 395 heartbeat_check: no reply from 172.21.15.87:6814 osd.1 since back 2024-02-13T14:23:35.480357+0000 front 2024-02-13T14:23:35.480420+0000 (oldest deadline 2024-02-13T14:24:01.380377+0000) 2024-02-13T14:24:08.618 INFO:journalctl@ceph.osd.5.smithi148.stdout:Feb 13 14:24:08 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-5[42659]: 2024-02-13T14:24:08.571+0000 7f97367bf700 -1 osd.5 395 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-02-13T14:23:31.860731+0000 front 2024-02-13T14:23:31.860700+0000 (oldest deadline 2024-02-13T14:23:52.960576+0000) 2024-02-13T14:24:08.619 INFO:journalctl@ceph.osd.5.smithi148.stdout:Feb 13 14:24:08 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-5[42659]: 2024-02-13T14:24:08.571+0000 7f97367bf700 -1 osd.5 395 heartbeat_check: no reply from 172.21.15.87:6814 osd.1 since back 2024-02-13T14:23:39.962085+0000 front 2024-02-13T14:23:39.961987+0000 (oldest deadline 2024-02-13T14:24:01.661991+0000) 2024-02-13T14:24:08.619 INFO:journalctl@ceph.osd.6.smithi148.stdout:Feb 13 14:24:08 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-6[48606]: 2024-02-13T14:24:08.362+0000 7f9ed7433700 -1 osd.6 395 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-02-13T14:23:26.938960+0000 front 2024-02-13T14:23:26.938983+0000 (oldest deadline 2024-02-13T14:23:52.838986+0000) 2024-02-13T14:24:08.619 INFO:journalctl@ceph.osd.6.smithi148.stdout:Feb 13 14:24:08 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-6[48606]: 2024-02-13T14:24:08.362+0000 7f9ed7433700 -1 osd.6 395 heartbeat_check: no reply from 172.21.15.87:6814 osd.1 since back 2024-02-13T14:23:39.741076+0000 front 2024-02-13T14:23:39.741053+0000 (oldest deadline 2024-02-13T14:24:02.541342+0000) 2024-02-13T14:24:08.619 INFO:journalctl@ceph.osd.7.smithi148.stdout:Feb 13 14:24:08 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-7[54686]: 2024-02-13T14:24:08.490+0000 7f4025619700 -1 osd.7 395 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-02-13T14:23:32.079364+0000 front 2024-02-13T14:23:32.079377+0000 (oldest deadline 2024-02-13T14:23:55.479861+0000) 2024-02-13T14:24:08.619 INFO:journalctl@ceph.osd.7.smithi148.stdout:Feb 13 14:24:08 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-7[54686]: 2024-02-13T14:24:08.490+0000 7f4025619700 -1 osd.7 395 heartbeat_check: no reply from 172.21.15.87:6814 osd.1 since back 2024-02-13T14:23:35.480357+0000 front 2024-02-13T14:23:35.480420+0000 (oldest deadline 2024-02-13T14:24:01.380377+0000) 2024-02-13T14:24:09.868 INFO:journalctl@ceph.osd.6.smithi148.stdout:Feb 13 14:24:09 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-6[48606]: 2024-02-13T14:24:09.378+0000 7f9ed7433700 -1 osd.6 395 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-02-13T14:23:26.938960+0000 front 2024-02-13T14:23:26.938983+0000 (oldest deadline 2024-02-13T14:23:52.838986+0000) 2024-02-13T14:24:09.869 INFO:journalctl@ceph.osd.6.smithi148.stdout:Feb 13 14:24:09 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-6[48606]: 2024-02-13T14:24:09.378+0000 7f9ed7433700 -1 osd.6 395 heartbeat_check: no reply from 172.21.15.87:6814 osd.1 since back 2024-02-13T14:23:39.741076+0000 front 2024-02-13T14:23:39.741053+0000 (oldest deadline 2024-02-13T14:24:02.541342+0000) 2024-02-13T14:24:09.869 INFO:journalctl@ceph.osd.5.smithi148.stdout:Feb 13 14:24:09 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-5[42659]: 2024-02-13T14:24:09.592+0000 7f97367bf700 -1 osd.5 395 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-02-13T14:23:31.860731+0000 front 2024-02-13T14:23:31.860700+0000 (oldest deadline 2024-02-13T14:23:52.960576+0000) 2024-02-13T14:24:09.869 INFO:journalctl@ceph.osd.5.smithi148.stdout:Feb 13 14:24:09 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-5[42659]: 2024-02-13T14:24:09.592+0000 7f97367bf700 -1 osd.5 395 heartbeat_check: no reply from 172.21.15.87:6814 osd.1 since back 2024-02-13T14:23:39.962085+0000 front 2024-02-13T14:23:39.961987+0000 (oldest deadline 2024-02-13T14:24:01.661991+0000) 2024-02-13T14:24:09.869 INFO:journalctl@ceph.osd.7.smithi148.stdout:Feb 13 14:24:09 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-7[54686]: 2024-02-13T14:24:09.484+0000 7f4025619700 -1 osd.7 395 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-02-13T14:23:32.079364+0000 front 2024-02-13T14:23:32.079377+0000 (oldest deadline 2024-02-13T14:23:55.479861+0000) 2024-02-13T14:24:09.869 INFO:journalctl@ceph.osd.7.smithi148.stdout:Feb 13 14:24:09 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-7[54686]: 2024-02-13T14:24:09.484+0000 7f4025619700 -1 osd.7 395 heartbeat_check: no reply from 172.21.15.87:6814 osd.1 since back 2024-02-13T14:23:35.480357+0000 front 2024-02-13T14:23:35.480420+0000 (oldest deadline 2024-02-13T14:24:01.380377+0000) 2024-02-13T14:24:10.868 INFO:journalctl@ceph.osd.6.smithi148.stdout:Feb 13 14:24:10 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-6[48606]: 2024-02-13T14:24:10.423+0000 7f9ed7433700 -1 osd.6 395 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-02-13T14:23:26.938960+0000 front 2024-02-13T14:23:26.938983+0000 (oldest deadline 2024-02-13T14:23:52.838986+0000) 2024-02-13T14:24:10.868 INFO:journalctl@ceph.osd.6.smithi148.stdout:Feb 13 14:24:10 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-6[48606]: 2024-02-13T14:24:10.423+0000 7f9ed7433700 -1 osd.6 395 heartbeat_check: no reply from 172.21.15.87:6814 osd.1 since back 2024-02-13T14:23:39.741076+0000 front 2024-02-13T14:23:39.741053+0000 (oldest deadline 2024-02-13T14:24:02.541342+0000) 2024-02-13T14:24:10.869 INFO:journalctl@ceph.osd.7.smithi148.stdout:Feb 13 14:24:10 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-7[54686]: 2024-02-13T14:24:10.439+0000 7f4025619700 -1 osd.7 395 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-02-13T14:23:32.079364+0000 front 2024-02-13T14:23:32.079377+0000 (oldest deadline 2024-02-13T14:23:55.479861+0000) 2024-02-13T14:24:10.869 INFO:journalctl@ceph.osd.7.smithi148.stdout:Feb 13 14:24:10 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-7[54686]: 2024-02-13T14:24:10.439+0000 7f4025619700 -1 osd.7 395 heartbeat_check: no reply from 172.21.15.87:6814 osd.1 since back 2024-02-13T14:23:35.480357+0000 front 2024-02-13T14:23:35.480420+0000 (oldest deadline 2024-02-13T14:24:01.380377+0000) 2024-02-13T14:24:10.869 INFO:journalctl@ceph.osd.5.smithi148.stdout:Feb 13 14:24:10 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-5[42659]: 2024-02-13T14:24:10.566+0000 7f97367bf700 -1 osd.5 395 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-02-13T14:23:31.860731+0000 front 2024-02-13T14:23:31.860700+0000 (oldest deadline 2024-02-13T14:23:52.960576+0000) 2024-02-13T14:24:10.869 INFO:journalctl@ceph.osd.5.smithi148.stdout:Feb 13 14:24:10 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-5[42659]: 2024-02-13T14:24:10.566+0000 7f97367bf700 -1 osd.5 395 heartbeat_check: no reply from 172.21.15.87:6814 osd.1 since back 2024-02-13T14:23:39.962085+0000 front 2024-02-13T14:23:39.961987+0000 (oldest deadline 2024-02-13T14:24:01.661991+0000) 2024-02-13T14:24:10.869 INFO:journalctl@ceph.osd.5.smithi148.stdout:Feb 13 14:24:10 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-5[42659]: 2024-02-13T14:24:10.566+0000 7f97367bf700 -1 osd.5 395 heartbeat_check: no reply from 172.21.15.87:6822 osd.2 since back 2024-02-13T14:23:46.263575+0000 front 2024-02-13T14:23:46.263436+0000 (oldest deadline 2024-02-13T14:24:10.363425+0000) 2024-02-13T14:24:11.752 INFO:journalctl@ceph.osd.5.smithi148.stdout:Feb 13 14:24:11 smithi148 podman[67804]: 2024-02-13 14:24:11.449743176 +0000 UTC m=+5.449524860 container died ec4f44410b031c56b686a3055913cba849532c9ebcd040fce15098dd848fa0b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-5, io.buildah.version=1.33.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240131, GIT_CLEAN=True) 2024-02-13T14:24:11.752 INFO:journalctl@ceph.osd.6.smithi148.stdout:Feb 13 14:24:11 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-6[48606]: 2024-02-13T14:24:11.375+0000 7f9ed7433700 -1 osd.6 395 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-02-13T14:23:26.938960+0000 front 2024-02-13T14:23:26.938983+0000 (oldest deadline 2024-02-13T14:23:52.838986+0000) 2024-02-13T14:24:11.752 INFO:journalctl@ceph.osd.6.smithi148.stdout:Feb 13 14:24:11 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-6[48606]: 2024-02-13T14:24:11.375+0000 7f9ed7433700 -1 osd.6 395 heartbeat_check: no reply from 172.21.15.87:6814 osd.1 since back 2024-02-13T14:23:39.741076+0000 front 2024-02-13T14:23:39.741053+0000 (oldest deadline 2024-02-13T14:24:02.541342+0000) 2024-02-13T14:24:11.752 INFO:journalctl@ceph.osd.7.smithi148.stdout:Feb 13 14:24:11 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-7[54686]: 2024-02-13T14:24:11.480+0000 7f4025619700 -1 osd.7 395 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-02-13T14:23:32.079364+0000 front 2024-02-13T14:23:32.079377+0000 (oldest deadline 2024-02-13T14:23:55.479861+0000) 2024-02-13T14:24:11.752 INFO:journalctl@ceph.osd.7.smithi148.stdout:Feb 13 14:24:11 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-7[54686]: 2024-02-13T14:24:11.480+0000 7f4025619700 -1 osd.7 395 heartbeat_check: no reply from 172.21.15.87:6814 osd.1 since back 2024-02-13T14:23:35.480357+0000 front 2024-02-13T14:23:35.480420+0000 (oldest deadline 2024-02-13T14:24:01.380377+0000) 2024-02-13T14:24:11.753 INFO:journalctl@ceph.osd.7.smithi148.stdout:Feb 13 14:24:11 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-7[54686]: 2024-02-13T14:24:11.480+0000 7f4025619700 -1 osd.7 395 heartbeat_check: no reply from 172.21.15.87:6822 osd.2 since back 2024-02-13T14:23:45.481305+0000 front 2024-02-13T14:23:45.481157+0000 (oldest deadline 2024-02-13T14:24:11.381166+0000) 2024-02-13T14:24:12.109 INFO:journalctl@ceph.osd.5.smithi148.stdout:Feb 13 14:24:11 smithi148 podman[67804]: 2024-02-13 14:24:11.849217458 +0000 UTC m=+5.848999142 container cleanup ec4f44410b031c56b686a3055913cba849532c9ebcd040fce15098dd848fa0b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240131, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True) 2024-02-13T14:24:12.109 INFO:journalctl@ceph.osd.5.smithi148.stdout:Feb 13 14:24:11 smithi148 bash[67804]: ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-5 2024-02-13T14:24:12.368 INFO:journalctl@ceph.osd.5.smithi148.stdout:Feb 13 14:24:12 smithi148 podman[67827]: 2024-02-13 14:24:12.108654067 +0000 UTC m=+0.652351402 container remove ec4f44410b031c56b686a3055913cba849532c9ebcd040fce15098dd848fa0b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-5, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.build-date=20240131, org.label-schema.vendor=CentOS, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.2, CEPH_POINT_RELEASE=) 2024-02-13T14:24:12.745 INFO:journalctl@ceph.osd.6.smithi148.stdout:Feb 13 14:24:12 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-6[48606]: 2024-02-13T14:24:12.424+0000 7f9ed7433700 -1 osd.6 395 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-02-13T14:23:26.938960+0000 front 2024-02-13T14:23:26.938983+0000 (oldest deadline 2024-02-13T14:23:52.838986+0000) 2024-02-13T14:24:12.745 INFO:journalctl@ceph.osd.6.smithi148.stdout:Feb 13 14:24:12 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-6[48606]: 2024-02-13T14:24:12.424+0000 7f9ed7433700 -1 osd.6 395 heartbeat_check: no reply from 172.21.15.87:6814 osd.1 since back 2024-02-13T14:23:39.741076+0000 front 2024-02-13T14:23:39.741053+0000 (oldest deadline 2024-02-13T14:24:02.541342+0000) 2024-02-13T14:24:12.746 INFO:journalctl@ceph.osd.7.smithi148.stdout:Feb 13 14:24:12 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-7[54686]: 2024-02-13T14:24:12.511+0000 7f4025619700 -1 osd.7 395 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-02-13T14:23:32.079364+0000 front 2024-02-13T14:23:32.079377+0000 (oldest deadline 2024-02-13T14:23:55.479861+0000) 2024-02-13T14:24:12.746 INFO:journalctl@ceph.osd.7.smithi148.stdout:Feb 13 14:24:12 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-7[54686]: 2024-02-13T14:24:12.511+0000 7f4025619700 -1 osd.7 395 heartbeat_check: no reply from 172.21.15.87:6814 osd.1 since back 2024-02-13T14:23:35.480357+0000 front 2024-02-13T14:23:35.480420+0000 (oldest deadline 2024-02-13T14:24:01.380377+0000) 2024-02-13T14:24:12.746 INFO:journalctl@ceph.osd.7.smithi148.stdout:Feb 13 14:24:12 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-7[54686]: 2024-02-13T14:24:12.511+0000 7f4025619700 -1 osd.7 395 heartbeat_check: no reply from 172.21.15.87:6822 osd.2 since back 2024-02-13T14:23:45.481305+0000 front 2024-02-13T14:23:45.481157+0000 (oldest deadline 2024-02-13T14:24:11.381166+0000) 2024-02-13T14:24:12.746 INFO:journalctl@ceph.osd.5.smithi148.stdout:Feb 13 14:24:12 smithi148 podman[67926]: 2024-02-13 14:24:12.370598018 +0000 UTC m=+0.018048803 image pull 28480620e5edd298d55c0e6673dcbad9ed23edd37269028a11063c1b8925676a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f 2024-02-13T14:24:12.746 INFO:journalctl@ceph.osd.5.smithi148.stdout:Feb 13 14:24:12 smithi148 podman[67926]: 2024-02-13 14:24:12.476089106 +0000 UTC m=+0.123539879 container create b133a9b8ba02fecf7c5fdc6789e4d4d55c0e60fbae7c4b8ef1f03a21a8bf3c14 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-5-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.2, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240131) 2024-02-13T14:24:12.746 INFO:journalctl@ceph.osd.5.smithi148.stdout:Feb 13 14:24:12 smithi148 podman[67926]: 2024-02-13 14:24:12.669794953 +0000 UTC m=+0.317245727 container init b133a9b8ba02fecf7c5fdc6789e4d4d55c0e60fbae7c4b8ef1f03a21a8bf3c14 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-5-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240131) 2024-02-13T14:24:12.746 INFO:journalctl@ceph.osd.5.smithi148.stdout:Feb 13 14:24:12 smithi148 podman[67926]: 2024-02-13 14:24:12.674865793 +0000 UTC m=+0.322316568 container start b133a9b8ba02fecf7c5fdc6789e4d4d55c0e60fbae7c4b8ef1f03a21a8bf3c14 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240131, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-02-13T14:24:12.746 INFO:journalctl@ceph.osd.5.smithi148.stdout:Feb 13 14:24:12 smithi148 podman[67926]: 2024-02-13 14:24:12.744837437 +0000 UTC m=+0.392288216 container attach b133a9b8ba02fecf7c5fdc6789e4d4d55c0e60fbae7c4b8ef1f03a21a8bf3c14 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-5-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, org.label-schema.build-date=20240131, org.label-schema.vendor=CentOS, io.buildah.version=1.33.2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275) 2024-02-13T14:24:13.119 INFO:journalctl@ceph.osd.5.smithi148.stdout:Feb 13 14:24:12 smithi148 podman[67926]: 2024-02-13 14:24:12.974560239 +0000 UTC m=+0.622011021 container died b133a9b8ba02fecf7c5fdc6789e4d4d55c0e60fbae7c4b8ef1f03a21a8bf3c14 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-5-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.2, org.label-schema.build-date=20240131) 2024-02-13T14:24:13.551 INFO:journalctl@ceph.osd.6.smithi148.stdout:Feb 13 14:24:13 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-6[48606]: 2024-02-13T14:24:13.468+0000 7f9ed7433700 -1 osd.6 395 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-02-13T14:23:26.938960+0000 front 2024-02-13T14:23:26.938983+0000 (oldest deadline 2024-02-13T14:23:52.838986+0000) 2024-02-13T14:24:13.551 INFO:journalctl@ceph.osd.6.smithi148.stdout:Feb 13 14:24:13 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-6[48606]: 2024-02-13T14:24:13.468+0000 7f9ed7433700 -1 osd.6 395 heartbeat_check: no reply from 172.21.15.87:6814 osd.1 since back 2024-02-13T14:23:39.741076+0000 front 2024-02-13T14:23:39.741053+0000 (oldest deadline 2024-02-13T14:24:02.541342+0000) 2024-02-13T14:24:13.551 INFO:journalctl@ceph.osd.6.smithi148.stdout:Feb 13 14:24:13 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-6[48606]: 2024-02-13T14:24:13.468+0000 7f9ed7433700 -1 osd.6 395 heartbeat_check: no reply from 172.21.15.87:6822 osd.2 since back 2024-02-13T14:23:47.142816+0000 front 2024-02-13T14:23:47.142804+0000 (oldest deadline 2024-02-13T14:24:13.042764+0000) 2024-02-13T14:24:13.551 INFO:journalctl@ceph.osd.7.smithi148.stdout:Feb 13 14:24:13 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-7[54686]: 2024-02-13T14:24:13.513+0000 7f4025619700 -1 osd.7 395 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-02-13T14:23:32.079364+0000 front 2024-02-13T14:23:32.079377+0000 (oldest deadline 2024-02-13T14:23:55.479861+0000) 2024-02-13T14:24:13.551 INFO:journalctl@ceph.osd.7.smithi148.stdout:Feb 13 14:24:13 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-7[54686]: 2024-02-13T14:24:13.513+0000 7f4025619700 -1 osd.7 395 heartbeat_check: no reply from 172.21.15.87:6814 osd.1 since back 2024-02-13T14:23:35.480357+0000 front 2024-02-13T14:23:35.480420+0000 (oldest deadline 2024-02-13T14:24:01.380377+0000) 2024-02-13T14:24:13.551 INFO:journalctl@ceph.osd.7.smithi148.stdout:Feb 13 14:24:13 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-7[54686]: 2024-02-13T14:24:13.513+0000 7f4025619700 -1 osd.7 395 heartbeat_check: no reply from 172.21.15.87:6822 osd.2 since back 2024-02-13T14:23:45.481305+0000 front 2024-02-13T14:23:45.481157+0000 (oldest deadline 2024-02-13T14:24:11.381166+0000) 2024-02-13T14:24:13.552 INFO:journalctl@ceph.osd.5.smithi148.stdout:Feb 13 14:24:13 smithi148 podman[67972]: 2024-02-13 14:24:13.548168875 +0000 UTC m=+0.568242826 container remove b133a9b8ba02fecf7c5fdc6789e4d4d55c0e60fbae7c4b8ef1f03a21a8bf3c14 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-5-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, io.buildah.version=1.33.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240131, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.vendor=CentOS) 2024-02-13T14:24:13.569 DEBUG:teuthology.orchestra.run.smithi148:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5c04b074-ca79-11ee-95b9-87774f69a715@osd.5.service' 2024-02-13T14:24:13.610 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-13T14:24:13.610 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-02-13T14:24:13.610 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-02-13T14:24:13.610 DEBUG:teuthology.orchestra.run.smithi148:> sudo systemctl stop ceph-5c04b074-ca79-11ee-95b9-87774f69a715@osd.6 2024-02-13T14:24:13.869 INFO:journalctl@ceph.osd.6.smithi148.stdout:Feb 13 14:24:13 smithi148 systemd[1]: Stopping Ceph osd.6 for 5c04b074-ca79-11ee-95b9-87774f69a715... 2024-02-13T14:24:13.992 INFO:journalctl@ceph.alertmanager.a.smithi087.stdout:Feb 13 14:24:13 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-alertmanager-a[78198]: ts=2024-02-13T14:24:13.531Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=2 err="ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://smithi087.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.87:8443: connect: connection refused; ceph-dashboard/webhook[1]: notify retry canceled after 8 attempts: Post \"https://smithi148.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.148:8443: connect: connection refused" 2024-02-13T14:24:13.992 INFO:journalctl@ceph.alertmanager.a.smithi087.stdout:Feb 13 14:24:13 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-alertmanager-a[78198]: ts=2024-02-13T14:24:13.532Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi087.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.87:8443: connect: connection refused" 2024-02-13T14:24:13.992 INFO:journalctl@ceph.alertmanager.a.smithi087.stdout:Feb 13 14:24:13 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-alertmanager-a[78198]: ts=2024-02-13T14:24:13.533Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi148.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.148:8443: connect: connection refused" 2024-02-13T14:24:14.368 INFO:journalctl@ceph.osd.6.smithi148.stdout:Feb 13 14:24:13 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-6[48606]: 2024-02-13T14:24:13.978+0000 7f9edee47700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-13T14:24:14.368 INFO:journalctl@ceph.osd.6.smithi148.stdout:Feb 13 14:24:13 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-6[48606]: 2024-02-13T14:24:13.978+0000 7f9edee47700 -1 osd.6 395 *** Got signal Terminated *** 2024-02-13T14:24:14.369 INFO:journalctl@ceph.osd.6.smithi148.stdout:Feb 13 14:24:13 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-6[48606]: 2024-02-13T14:24:13.978+0000 7f9edee47700 -1 osd.6 395 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-13T14:24:14.869 INFO:journalctl@ceph.osd.6.smithi148.stdout:Feb 13 14:24:14 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-6[48606]: 2024-02-13T14:24:14.494+0000 7f9ed7433700 -1 osd.6 395 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-02-13T14:23:26.938960+0000 front 2024-02-13T14:23:26.938983+0000 (oldest deadline 2024-02-13T14:23:52.838986+0000) 2024-02-13T14:24:14.869 INFO:journalctl@ceph.osd.6.smithi148.stdout:Feb 13 14:24:14 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-6[48606]: 2024-02-13T14:24:14.494+0000 7f9ed7433700 -1 osd.6 395 heartbeat_check: no reply from 172.21.15.87:6814 osd.1 since back 2024-02-13T14:23:39.741076+0000 front 2024-02-13T14:23:39.741053+0000 (oldest deadline 2024-02-13T14:24:02.541342+0000) 2024-02-13T14:24:14.869 INFO:journalctl@ceph.osd.6.smithi148.stdout:Feb 13 14:24:14 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-6[48606]: 2024-02-13T14:24:14.494+0000 7f9ed7433700 -1 osd.6 395 heartbeat_check: no reply from 172.21.15.87:6822 osd.2 since back 2024-02-13T14:23:47.142816+0000 front 2024-02-13T14:23:47.142804+0000 (oldest deadline 2024-02-13T14:24:13.042764+0000) 2024-02-13T14:24:14.869 INFO:journalctl@ceph.osd.7.smithi148.stdout:Feb 13 14:24:14 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-7[54686]: 2024-02-13T14:24:14.529+0000 7f4025619700 -1 osd.7 395 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-02-13T14:23:32.079364+0000 front 2024-02-13T14:23:32.079377+0000 (oldest deadline 2024-02-13T14:23:55.479861+0000) 2024-02-13T14:24:14.869 INFO:journalctl@ceph.osd.7.smithi148.stdout:Feb 13 14:24:14 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-7[54686]: 2024-02-13T14:24:14.529+0000 7f4025619700 -1 osd.7 395 heartbeat_check: no reply from 172.21.15.87:6814 osd.1 since back 2024-02-13T14:23:35.480357+0000 front 2024-02-13T14:23:35.480420+0000 (oldest deadline 2024-02-13T14:24:01.380377+0000) 2024-02-13T14:24:14.869 INFO:journalctl@ceph.osd.7.smithi148.stdout:Feb 13 14:24:14 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-7[54686]: 2024-02-13T14:24:14.529+0000 7f4025619700 -1 osd.7 395 heartbeat_check: no reply from 172.21.15.87:6822 osd.2 since back 2024-02-13T14:23:45.481305+0000 front 2024-02-13T14:23:45.481157+0000 (oldest deadline 2024-02-13T14:24:11.381166+0000) 2024-02-13T14:24:15.868 INFO:journalctl@ceph.osd.6.smithi148.stdout:Feb 13 14:24:15 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-6[48606]: 2024-02-13T14:24:15.510+0000 7f9ed7433700 -1 osd.6 395 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-02-13T14:23:26.938960+0000 front 2024-02-13T14:23:26.938983+0000 (oldest deadline 2024-02-13T14:23:52.838986+0000) 2024-02-13T14:24:15.869 INFO:journalctl@ceph.osd.6.smithi148.stdout:Feb 13 14:24:15 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-6[48606]: 2024-02-13T14:24:15.510+0000 7f9ed7433700 -1 osd.6 395 heartbeat_check: no reply from 172.21.15.87:6814 osd.1 since back 2024-02-13T14:23:39.741076+0000 front 2024-02-13T14:23:39.741053+0000 (oldest deadline 2024-02-13T14:24:02.541342+0000) 2024-02-13T14:24:15.869 INFO:journalctl@ceph.osd.6.smithi148.stdout:Feb 13 14:24:15 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-6[48606]: 2024-02-13T14:24:15.510+0000 7f9ed7433700 -1 osd.6 395 heartbeat_check: no reply from 172.21.15.87:6822 osd.2 since back 2024-02-13T14:23:47.142816+0000 front 2024-02-13T14:23:47.142804+0000 (oldest deadline 2024-02-13T14:24:13.042764+0000) 2024-02-13T14:24:15.869 INFO:journalctl@ceph.osd.7.smithi148.stdout:Feb 13 14:24:15 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-7[54686]: 2024-02-13T14:24:15.547+0000 7f4025619700 -1 osd.7 395 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-02-13T14:23:32.079364+0000 front 2024-02-13T14:23:32.079377+0000 (oldest deadline 2024-02-13T14:23:55.479861+0000) 2024-02-13T14:24:15.869 INFO:journalctl@ceph.osd.7.smithi148.stdout:Feb 13 14:24:15 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-7[54686]: 2024-02-13T14:24:15.547+0000 7f4025619700 -1 osd.7 395 heartbeat_check: no reply from 172.21.15.87:6814 osd.1 since back 2024-02-13T14:23:35.480357+0000 front 2024-02-13T14:23:35.480420+0000 (oldest deadline 2024-02-13T14:24:01.380377+0000) 2024-02-13T14:24:15.870 INFO:journalctl@ceph.osd.7.smithi148.stdout:Feb 13 14:24:15 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-7[54686]: 2024-02-13T14:24:15.547+0000 7f4025619700 -1 osd.7 395 heartbeat_check: no reply from 172.21.15.87:6822 osd.2 since back 2024-02-13T14:23:45.481305+0000 front 2024-02-13T14:23:45.481157+0000 (oldest deadline 2024-02-13T14:24:11.381166+0000) 2024-02-13T14:24:15.870 INFO:journalctl@ceph.osd.7.smithi148.stdout:Feb 13 14:24:15 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-7[54686]: 2024-02-13T14:24:15.547+0000 7f4025619700 -1 osd.7 395 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24409.0:493 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e395) 2024-02-13T14:24:16.868 INFO:journalctl@ceph.osd.6.smithi148.stdout:Feb 13 14:24:16 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-6[48606]: 2024-02-13T14:24:16.463+0000 7f9ed7433700 -1 osd.6 395 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-02-13T14:23:26.938960+0000 front 2024-02-13T14:23:26.938983+0000 (oldest deadline 2024-02-13T14:23:52.838986+0000) 2024-02-13T14:24:16.869 INFO:journalctl@ceph.osd.6.smithi148.stdout:Feb 13 14:24:16 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-6[48606]: 2024-02-13T14:24:16.463+0000 7f9ed7433700 -1 osd.6 395 heartbeat_check: no reply from 172.21.15.87:6814 osd.1 since back 2024-02-13T14:23:39.741076+0000 front 2024-02-13T14:23:39.741053+0000 (oldest deadline 2024-02-13T14:24:02.541342+0000) 2024-02-13T14:24:16.869 INFO:journalctl@ceph.osd.6.smithi148.stdout:Feb 13 14:24:16 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-6[48606]: 2024-02-13T14:24:16.463+0000 7f9ed7433700 -1 osd.6 395 heartbeat_check: no reply from 172.21.15.87:6822 osd.2 since back 2024-02-13T14:23:47.142816+0000 front 2024-02-13T14:23:47.142804+0000 (oldest deadline 2024-02-13T14:24:13.042764+0000) 2024-02-13T14:24:16.869 INFO:journalctl@ceph.osd.7.smithi148.stdout:Feb 13 14:24:16 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-7[54686]: 2024-02-13T14:24:16.542+0000 7f4025619700 -1 osd.7 395 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-02-13T14:23:32.079364+0000 front 2024-02-13T14:23:32.079377+0000 (oldest deadline 2024-02-13T14:23:55.479861+0000) 2024-02-13T14:24:16.869 INFO:journalctl@ceph.osd.7.smithi148.stdout:Feb 13 14:24:16 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-7[54686]: 2024-02-13T14:24:16.542+0000 7f4025619700 -1 osd.7 395 heartbeat_check: no reply from 172.21.15.87:6814 osd.1 since back 2024-02-13T14:23:35.480357+0000 front 2024-02-13T14:23:35.480420+0000 (oldest deadline 2024-02-13T14:24:01.380377+0000) 2024-02-13T14:24:16.869 INFO:journalctl@ceph.osd.7.smithi148.stdout:Feb 13 14:24:16 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-7[54686]: 2024-02-13T14:24:16.542+0000 7f4025619700 -1 osd.7 395 heartbeat_check: no reply from 172.21.15.87:6822 osd.2 since back 2024-02-13T14:23:45.481305+0000 front 2024-02-13T14:23:45.481157+0000 (oldest deadline 2024-02-13T14:24:11.381166+0000) 2024-02-13T14:24:16.869 INFO:journalctl@ceph.osd.7.smithi148.stdout:Feb 13 14:24:16 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-7[54686]: 2024-02-13T14:24:16.542+0000 7f4025619700 -1 osd.7 395 heartbeat_check: no reply from 172.21.15.87:6830 osd.3 since back 2024-02-13T14:23:52.982720+0000 front 2024-02-13T14:23:52.982697+0000 (oldest deadline 2024-02-13T14:24:16.482807+0000) 2024-02-13T14:24:16.869 INFO:journalctl@ceph.osd.7.smithi148.stdout:Feb 13 14:24:16 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-7[54686]: 2024-02-13T14:24:16.542+0000 7f4025619700 -1 osd.7 395 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24409.0:493 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e395) 2024-02-13T14:24:17.869 INFO:journalctl@ceph.osd.6.smithi148.stdout:Feb 13 14:24:17 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-6[48606]: 2024-02-13T14:24:17.490+0000 7f9ed7433700 -1 osd.6 395 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-02-13T14:23:26.938960+0000 front 2024-02-13T14:23:26.938983+0000 (oldest deadline 2024-02-13T14:23:52.838986+0000) 2024-02-13T14:24:17.869 INFO:journalctl@ceph.osd.6.smithi148.stdout:Feb 13 14:24:17 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-6[48606]: 2024-02-13T14:24:17.490+0000 7f9ed7433700 -1 osd.6 395 heartbeat_check: no reply from 172.21.15.87:6814 osd.1 since back 2024-02-13T14:23:39.741076+0000 front 2024-02-13T14:23:39.741053+0000 (oldest deadline 2024-02-13T14:24:02.541342+0000) 2024-02-13T14:24:17.869 INFO:journalctl@ceph.osd.6.smithi148.stdout:Feb 13 14:24:17 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-6[48606]: 2024-02-13T14:24:17.490+0000 7f9ed7433700 -1 osd.6 395 heartbeat_check: no reply from 172.21.15.87:6822 osd.2 since back 2024-02-13T14:23:47.142816+0000 front 2024-02-13T14:23:47.142804+0000 (oldest deadline 2024-02-13T14:24:13.042764+0000) 2024-02-13T14:24:17.869 INFO:journalctl@ceph.osd.7.smithi148.stdout:Feb 13 14:24:17 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-7[54686]: 2024-02-13T14:24:17.523+0000 7f4025619700 -1 osd.7 395 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-02-13T14:23:32.079364+0000 front 2024-02-13T14:23:32.079377+0000 (oldest deadline 2024-02-13T14:23:55.479861+0000) 2024-02-13T14:24:17.869 INFO:journalctl@ceph.osd.7.smithi148.stdout:Feb 13 14:24:17 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-7[54686]: 2024-02-13T14:24:17.523+0000 7f4025619700 -1 osd.7 395 heartbeat_check: no reply from 172.21.15.87:6814 osd.1 since back 2024-02-13T14:23:35.480357+0000 front 2024-02-13T14:23:35.480420+0000 (oldest deadline 2024-02-13T14:24:01.380377+0000) 2024-02-13T14:24:17.869 INFO:journalctl@ceph.osd.7.smithi148.stdout:Feb 13 14:24:17 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-7[54686]: 2024-02-13T14:24:17.523+0000 7f4025619700 -1 osd.7 395 heartbeat_check: no reply from 172.21.15.87:6822 osd.2 since back 2024-02-13T14:23:45.481305+0000 front 2024-02-13T14:23:45.481157+0000 (oldest deadline 2024-02-13T14:24:11.381166+0000) 2024-02-13T14:24:17.869 INFO:journalctl@ceph.osd.7.smithi148.stdout:Feb 13 14:24:17 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-7[54686]: 2024-02-13T14:24:17.523+0000 7f4025619700 -1 osd.7 395 heartbeat_check: no reply from 172.21.15.87:6830 osd.3 since back 2024-02-13T14:23:52.982720+0000 front 2024-02-13T14:23:52.982697+0000 (oldest deadline 2024-02-13T14:24:16.482807+0000) 2024-02-13T14:24:17.869 INFO:journalctl@ceph.osd.7.smithi148.stdout:Feb 13 14:24:17 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-7[54686]: 2024-02-13T14:24:17.523+0000 7f4025619700 -1 osd.7 395 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24409.0:493 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e395) 2024-02-13T14:24:18.849 INFO:journalctl@ceph.osd.6.smithi148.stdout:Feb 13 14:24:18 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-6[48606]: 2024-02-13T14:24:18.463+0000 7f9ed7433700 -1 osd.6 395 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-02-13T14:23:26.938960+0000 front 2024-02-13T14:23:26.938983+0000 (oldest deadline 2024-02-13T14:23:52.838986+0000) 2024-02-13T14:24:18.850 INFO:journalctl@ceph.osd.6.smithi148.stdout:Feb 13 14:24:18 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-6[48606]: 2024-02-13T14:24:18.463+0000 7f9ed7433700 -1 osd.6 395 heartbeat_check: no reply from 172.21.15.87:6814 osd.1 since back 2024-02-13T14:23:39.741076+0000 front 2024-02-13T14:23:39.741053+0000 (oldest deadline 2024-02-13T14:24:02.541342+0000) 2024-02-13T14:24:18.850 INFO:journalctl@ceph.osd.6.smithi148.stdout:Feb 13 14:24:18 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-6[48606]: 2024-02-13T14:24:18.463+0000 7f9ed7433700 -1 osd.6 395 heartbeat_check: no reply from 172.21.15.87:6822 osd.2 since back 2024-02-13T14:23:47.142816+0000 front 2024-02-13T14:23:47.142804+0000 (oldest deadline 2024-02-13T14:24:13.042764+0000) 2024-02-13T14:24:18.850 INFO:journalctl@ceph.osd.6.smithi148.stdout:Feb 13 14:24:18 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-6[48606]: 2024-02-13T14:24:18.463+0000 7f9ed7433700 -1 osd.6 395 heartbeat_check: no reply from 172.21.15.87:6830 osd.3 since back 2024-02-13T14:23:53.043277+0000 front 2024-02-13T14:23:53.043311+0000 (oldest deadline 2024-02-13T14:24:17.643782+0000) 2024-02-13T14:24:18.850 INFO:journalctl@ceph.osd.7.smithi148.stdout:Feb 13 14:24:18 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-7[54686]: 2024-02-13T14:24:18.526+0000 7f4025619700 -1 osd.7 395 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-02-13T14:23:32.079364+0000 front 2024-02-13T14:23:32.079377+0000 (oldest deadline 2024-02-13T14:23:55.479861+0000) 2024-02-13T14:24:18.850 INFO:journalctl@ceph.osd.7.smithi148.stdout:Feb 13 14:24:18 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-7[54686]: 2024-02-13T14:24:18.526+0000 7f4025619700 -1 osd.7 395 heartbeat_check: no reply from 172.21.15.87:6814 osd.1 since back 2024-02-13T14:23:35.480357+0000 front 2024-02-13T14:23:35.480420+0000 (oldest deadline 2024-02-13T14:24:01.380377+0000) 2024-02-13T14:24:18.850 INFO:journalctl@ceph.osd.7.smithi148.stdout:Feb 13 14:24:18 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-7[54686]: 2024-02-13T14:24:18.526+0000 7f4025619700 -1 osd.7 395 heartbeat_check: no reply from 172.21.15.87:6822 osd.2 since back 2024-02-13T14:23:45.481305+0000 front 2024-02-13T14:23:45.481157+0000 (oldest deadline 2024-02-13T14:24:11.381166+0000) 2024-02-13T14:24:18.850 INFO:journalctl@ceph.osd.7.smithi148.stdout:Feb 13 14:24:18 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-7[54686]: 2024-02-13T14:24:18.526+0000 7f4025619700 -1 osd.7 395 heartbeat_check: no reply from 172.21.15.87:6830 osd.3 since back 2024-02-13T14:23:52.982720+0000 front 2024-02-13T14:23:52.982697+0000 (oldest deadline 2024-02-13T14:24:16.482807+0000) 2024-02-13T14:24:18.850 INFO:journalctl@ceph.osd.7.smithi148.stdout:Feb 13 14:24:18 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-7[54686]: 2024-02-13T14:24:18.526+0000 7f4025619700 -1 osd.7 395 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24409.0:493 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e395) 2024-02-13T14:24:19.119 INFO:journalctl@ceph.prometheus.a.smithi148.stdout:Feb 13 14:24:18 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-prometheus-a[65166]: ts=2024-02-13T14:24:18.849Z caller=refresh.go:99 level=error component="discovery manager scrape" discovery=http config=ceph-exporter msg="Unable to refresh target groups" err="Get \"http://172.21.15.87:8765/sd/prometheus/sd-config?service=ceph-exporter\": dial tcp 172.21.15.87:8765: connect: connection refused" 2024-02-13T14:24:19.119 INFO:journalctl@ceph.prometheus.a.smithi148.stdout:Feb 13 14:24:18 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-prometheus-a[65166]: ts=2024-02-13T14:24:18.849Z caller=refresh.go:99 level=error component="discovery manager scrape" discovery=http config=node msg="Unable to refresh target groups" err="Get \"http://172.21.15.87:8765/sd/prometheus/sd-config?service=node-exporter\": dial tcp 172.21.15.87:8765: connect: connection refused" 2024-02-13T14:24:19.119 INFO:journalctl@ceph.prometheus.a.smithi148.stdout:Feb 13 14:24:18 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-prometheus-a[65166]: ts=2024-02-13T14:24:18.851Z caller=refresh.go:99 level=error component="discovery manager scrape" discovery=http config=ceph msg="Unable to refresh target groups" err="Get \"http://172.21.15.87:8765/sd/prometheus/sd-config?service=mgr-prometheus\": dial tcp 172.21.15.87:8765: connect: connection refused" 2024-02-13T14:24:19.119 INFO:journalctl@ceph.prometheus.a.smithi148.stdout:Feb 13 14:24:18 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-prometheus-a[65166]: ts=2024-02-13T14:24:18.851Z caller=refresh.go:99 level=error component="discovery manager notify" discovery=http config=config-0 msg="Unable to refresh target groups" err="Get \"http://172.21.15.87:8765/sd/prometheus/sd-config?service=alertmanager\": dial tcp 172.21.15.87:8765: connect: connection refused" 2024-02-13T14:24:19.712 INFO:journalctl@ceph.osd.6.smithi148.stdout:Feb 13 14:24:19 smithi148 podman[68056]: 2024-02-13 14:24:19.409677682 +0000 UTC m=+5.574011464 container died a344ac3ef6b7e0f7c2e5abd8e8603dfaefde50e232dc9df5c54258bb831c2f0f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-6, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240131, GIT_BRANCH=HEAD, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-02-13T14:24:19.712 INFO:journalctl@ceph.osd.7.smithi148.stdout:Feb 13 14:24:19 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-7[54686]: 2024-02-13T14:24:19.518+0000 7f4025619700 -1 osd.7 395 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-02-13T14:23:32.079364+0000 front 2024-02-13T14:23:32.079377+0000 (oldest deadline 2024-02-13T14:23:55.479861+0000) 2024-02-13T14:24:19.712 INFO:journalctl@ceph.osd.7.smithi148.stdout:Feb 13 14:24:19 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-7[54686]: 2024-02-13T14:24:19.518+0000 7f4025619700 -1 osd.7 395 heartbeat_check: no reply from 172.21.15.87:6814 osd.1 since back 2024-02-13T14:23:35.480357+0000 front 2024-02-13T14:23:35.480420+0000 (oldest deadline 2024-02-13T14:24:01.380377+0000) 2024-02-13T14:24:19.713 INFO:journalctl@ceph.osd.7.smithi148.stdout:Feb 13 14:24:19 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-7[54686]: 2024-02-13T14:24:19.518+0000 7f4025619700 -1 osd.7 395 heartbeat_check: no reply from 172.21.15.87:6822 osd.2 since back 2024-02-13T14:23:45.481305+0000 front 2024-02-13T14:23:45.481157+0000 (oldest deadline 2024-02-13T14:24:11.381166+0000) 2024-02-13T14:24:19.713 INFO:journalctl@ceph.osd.7.smithi148.stdout:Feb 13 14:24:19 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-7[54686]: 2024-02-13T14:24:19.518+0000 7f4025619700 -1 osd.7 395 heartbeat_check: no reply from 172.21.15.87:6830 osd.3 since back 2024-02-13T14:23:52.982720+0000 front 2024-02-13T14:23:52.982697+0000 (oldest deadline 2024-02-13T14:24:16.482807+0000) 2024-02-13T14:24:19.713 INFO:journalctl@ceph.osd.7.smithi148.stdout:Feb 13 14:24:19 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-7[54686]: 2024-02-13T14:24:19.518+0000 7f4025619700 -1 osd.7 395 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24409.0:493 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e395) 2024-02-13T14:24:20.053 INFO:journalctl@ceph.osd.6.smithi148.stdout:Feb 13 14:24:19 smithi148 podman[68056]: 2024-02-13 14:24:19.817971928 +0000 UTC m=+5.982305710 container cleanup a344ac3ef6b7e0f7c2e5abd8e8603dfaefde50e232dc9df5c54258bb831c2f0f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-6, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240131, GIT_BRANCH=HEAD, io.buildah.version=1.33.2, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-13T14:24:20.053 INFO:journalctl@ceph.osd.6.smithi148.stdout:Feb 13 14:24:19 smithi148 bash[68056]: ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-6 2024-02-13T14:24:20.368 INFO:journalctl@ceph.osd.6.smithi148.stdout:Feb 13 14:24:20 smithi148 podman[68079]: 2024-02-13 14:24:20.052395103 +0000 UTC m=+0.637571429 container remove a344ac3ef6b7e0f7c2e5abd8e8603dfaefde50e232dc9df5c54258bb831c2f0f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-6, org.label-schema.build-date=20240131, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, CEPH_POINT_RELEASE=, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, ceph=True, org.label-schema.license=GPLv2) 2024-02-13T14:24:20.673 INFO:journalctl@ceph.osd.7.smithi148.stdout:Feb 13 14:24:20 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-7[54686]: 2024-02-13T14:24:20.491+0000 7f4025619700 -1 osd.7 395 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-02-13T14:23:32.079364+0000 front 2024-02-13T14:23:32.079377+0000 (oldest deadline 2024-02-13T14:23:55.479861+0000) 2024-02-13T14:24:20.673 INFO:journalctl@ceph.osd.7.smithi148.stdout:Feb 13 14:24:20 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-7[54686]: 2024-02-13T14:24:20.491+0000 7f4025619700 -1 osd.7 395 heartbeat_check: no reply from 172.21.15.87:6814 osd.1 since back 2024-02-13T14:23:35.480357+0000 front 2024-02-13T14:23:35.480420+0000 (oldest deadline 2024-02-13T14:24:01.380377+0000) 2024-02-13T14:24:20.673 INFO:journalctl@ceph.osd.7.smithi148.stdout:Feb 13 14:24:20 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-7[54686]: 2024-02-13T14:24:20.491+0000 7f4025619700 -1 osd.7 395 heartbeat_check: no reply from 172.21.15.87:6822 osd.2 since back 2024-02-13T14:23:45.481305+0000 front 2024-02-13T14:23:45.481157+0000 (oldest deadline 2024-02-13T14:24:11.381166+0000) 2024-02-13T14:24:20.673 INFO:journalctl@ceph.osd.7.smithi148.stdout:Feb 13 14:24:20 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-7[54686]: 2024-02-13T14:24:20.491+0000 7f4025619700 -1 osd.7 395 heartbeat_check: no reply from 172.21.15.87:6830 osd.3 since back 2024-02-13T14:23:52.982720+0000 front 2024-02-13T14:23:52.982697+0000 (oldest deadline 2024-02-13T14:24:16.482807+0000) 2024-02-13T14:24:20.673 INFO:journalctl@ceph.osd.7.smithi148.stdout:Feb 13 14:24:20 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-7[54686]: 2024-02-13T14:24:20.491+0000 7f4025619700 -1 osd.7 395 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24409.0:493 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e395) 2024-02-13T14:24:20.673 INFO:journalctl@ceph.osd.6.smithi148.stdout:Feb 13 14:24:20 smithi148 podman[68177]: 2024-02-13 14:24:20.286557988 +0000 UTC m=+0.017985134 image pull 28480620e5edd298d55c0e6673dcbad9ed23edd37269028a11063c1b8925676a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f 2024-02-13T14:24:20.673 INFO:journalctl@ceph.osd.6.smithi148.stdout:Feb 13 14:24:20 smithi148 podman[68177]: 2024-02-13 14:24:20.386284782 +0000 UTC m=+0.117711916 container create 33fef8d754032ea1cb63ac1a2d2d7b8f284f8b49e8e1d40149c893e2e1b69494 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-6-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240131, io.buildah.version=1.33.2, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-02-13T14:24:20.673 INFO:journalctl@ceph.osd.6.smithi148.stdout:Feb 13 14:24:20 smithi148 podman[68177]: 2024-02-13 14:24:20.580275714 +0000 UTC m=+0.311702860 container init 33fef8d754032ea1cb63ac1a2d2d7b8f284f8b49e8e1d40149c893e2e1b69494 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-6-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240131, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-02-13T14:24:20.673 INFO:journalctl@ceph.osd.6.smithi148.stdout:Feb 13 14:24:20 smithi148 podman[68177]: 2024-02-13 14:24:20.586007323 +0000 UTC m=+0.317434457 container start 33fef8d754032ea1cb63ac1a2d2d7b8f284f8b49e8e1d40149c893e2e1b69494 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-6-deactivate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240131, io.buildah.version=1.33.2, maintainer=Guillaume Abrioux , RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-13T14:24:20.673 INFO:journalctl@ceph.osd.6.smithi148.stdout:Feb 13 14:24:20 smithi148 podman[68177]: 2024-02-13 14:24:20.67206674 +0000 UTC m=+0.403493886 container attach 33fef8d754032ea1cb63ac1a2d2d7b8f284f8b49e8e1d40149c893e2e1b69494 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-6-deactivate, ceph=True, GIT_CLEAN=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240131, io.buildah.version=1.33.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, org.label-schema.vendor=CentOS) 2024-02-13T14:24:21.118 INFO:journalctl@ceph.osd.6.smithi148.stdout:Feb 13 14:24:20 smithi148 podman[68177]: 2024-02-13 14:24:20.879782438 +0000 UTC m=+0.611209575 container died 33fef8d754032ea1cb63ac1a2d2d7b8f284f8b49e8e1d40149c893e2e1b69494 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-6-deactivate, org.label-schema.build-date=20240131, io.buildah.version=1.33.2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5) 2024-02-13T14:24:21.498 DEBUG:teuthology.orchestra.run.smithi148:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5c04b074-ca79-11ee-95b9-87774f69a715@osd.6.service' 2024-02-13T14:24:21.542 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-13T14:24:21.543 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-02-13T14:24:21.543 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-02-13T14:24:21.543 DEBUG:teuthology.orchestra.run.smithi148:> sudo systemctl stop ceph-5c04b074-ca79-11ee-95b9-87774f69a715@osd.7 2024-02-13T14:24:21.868 INFO:journalctl@ceph.osd.7.smithi148.stdout:Feb 13 14:24:21 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-7[54686]: 2024-02-13T14:24:21.483+0000 7f4025619700 -1 osd.7 395 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-02-13T14:23:32.079364+0000 front 2024-02-13T14:23:32.079377+0000 (oldest deadline 2024-02-13T14:23:55.479861+0000) 2024-02-13T14:24:21.869 INFO:journalctl@ceph.osd.7.smithi148.stdout:Feb 13 14:24:21 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-7[54686]: 2024-02-13T14:24:21.483+0000 7f4025619700 -1 osd.7 395 heartbeat_check: no reply from 172.21.15.87:6814 osd.1 since back 2024-02-13T14:23:35.480357+0000 front 2024-02-13T14:23:35.480420+0000 (oldest deadline 2024-02-13T14:24:01.380377+0000) 2024-02-13T14:24:21.869 INFO:journalctl@ceph.osd.7.smithi148.stdout:Feb 13 14:24:21 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-7[54686]: 2024-02-13T14:24:21.483+0000 7f4025619700 -1 osd.7 395 heartbeat_check: no reply from 172.21.15.87:6822 osd.2 since back 2024-02-13T14:23:45.481305+0000 front 2024-02-13T14:23:45.481157+0000 (oldest deadline 2024-02-13T14:24:11.381166+0000) 2024-02-13T14:24:21.869 INFO:journalctl@ceph.osd.7.smithi148.stdout:Feb 13 14:24:21 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-7[54686]: 2024-02-13T14:24:21.483+0000 7f4025619700 -1 osd.7 395 heartbeat_check: no reply from 172.21.15.87:6830 osd.3 since back 2024-02-13T14:23:52.982720+0000 front 2024-02-13T14:23:52.982697+0000 (oldest deadline 2024-02-13T14:24:16.482807+0000) 2024-02-13T14:24:21.869 INFO:journalctl@ceph.osd.7.smithi148.stdout:Feb 13 14:24:21 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-7[54686]: 2024-02-13T14:24:21.483+0000 7f4025619700 -1 osd.7 395 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24409.0:493 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e395) 2024-02-13T14:24:21.869 INFO:journalctl@ceph.osd.7.smithi148.stdout:Feb 13 14:24:21 smithi148 systemd[1]: Stopping Ceph osd.7 for 5c04b074-ca79-11ee-95b9-87774f69a715... 2024-02-13T14:24:22.368 INFO:journalctl@ceph.osd.7.smithi148.stdout:Feb 13 14:24:21 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-7[54686]: 2024-02-13T14:24:21.897+0000 7f402d02d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-13T14:24:22.368 INFO:journalctl@ceph.osd.7.smithi148.stdout:Feb 13 14:24:21 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-7[54686]: 2024-02-13T14:24:21.897+0000 7f402d02d700 -1 osd.7 395 *** Got signal Terminated *** 2024-02-13T14:24:22.368 INFO:journalctl@ceph.osd.7.smithi148.stdout:Feb 13 14:24:21 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-7[54686]: 2024-02-13T14:24:21.897+0000 7f402d02d700 -1 osd.7 395 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-13T14:24:22.868 INFO:journalctl@ceph.osd.7.smithi148.stdout:Feb 13 14:24:22 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-7[54686]: 2024-02-13T14:24:22.492+0000 7f4025619700 -1 osd.7 395 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-02-13T14:23:32.079364+0000 front 2024-02-13T14:23:32.079377+0000 (oldest deadline 2024-02-13T14:23:55.479861+0000) 2024-02-13T14:24:22.869 INFO:journalctl@ceph.osd.7.smithi148.stdout:Feb 13 14:24:22 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-7[54686]: 2024-02-13T14:24:22.492+0000 7f4025619700 -1 osd.7 395 heartbeat_check: no reply from 172.21.15.87:6814 osd.1 since back 2024-02-13T14:23:35.480357+0000 front 2024-02-13T14:23:35.480420+0000 (oldest deadline 2024-02-13T14:24:01.380377+0000) 2024-02-13T14:24:22.869 INFO:journalctl@ceph.osd.7.smithi148.stdout:Feb 13 14:24:22 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-7[54686]: 2024-02-13T14:24:22.492+0000 7f4025619700 -1 osd.7 395 heartbeat_check: no reply from 172.21.15.87:6822 osd.2 since back 2024-02-13T14:23:45.481305+0000 front 2024-02-13T14:23:45.481157+0000 (oldest deadline 2024-02-13T14:24:11.381166+0000) 2024-02-13T14:24:22.869 INFO:journalctl@ceph.osd.7.smithi148.stdout:Feb 13 14:24:22 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-7[54686]: 2024-02-13T14:24:22.492+0000 7f4025619700 -1 osd.7 395 heartbeat_check: no reply from 172.21.15.87:6830 osd.3 since back 2024-02-13T14:23:52.982720+0000 front 2024-02-13T14:23:52.982697+0000 (oldest deadline 2024-02-13T14:24:16.482807+0000) 2024-02-13T14:24:22.869 INFO:journalctl@ceph.osd.7.smithi148.stdout:Feb 13 14:24:22 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-7[54686]: 2024-02-13T14:24:22.492+0000 7f4025619700 -1 osd.7 395 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24409.0:493 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e395) 2024-02-13T14:24:23.868 INFO:journalctl@ceph.osd.7.smithi148.stdout:Feb 13 14:24:23 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-7[54686]: 2024-02-13T14:24:23.507+0000 7f4025619700 -1 osd.7 395 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-02-13T14:23:32.079364+0000 front 2024-02-13T14:23:32.079377+0000 (oldest deadline 2024-02-13T14:23:55.479861+0000) 2024-02-13T14:24:23.869 INFO:journalctl@ceph.osd.7.smithi148.stdout:Feb 13 14:24:23 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-7[54686]: 2024-02-13T14:24:23.507+0000 7f4025619700 -1 osd.7 395 heartbeat_check: no reply from 172.21.15.87:6814 osd.1 since back 2024-02-13T14:23:35.480357+0000 front 2024-02-13T14:23:35.480420+0000 (oldest deadline 2024-02-13T14:24:01.380377+0000) 2024-02-13T14:24:23.869 INFO:journalctl@ceph.osd.7.smithi148.stdout:Feb 13 14:24:23 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-7[54686]: 2024-02-13T14:24:23.507+0000 7f4025619700 -1 osd.7 395 heartbeat_check: no reply from 172.21.15.87:6822 osd.2 since back 2024-02-13T14:23:45.481305+0000 front 2024-02-13T14:23:45.481157+0000 (oldest deadline 2024-02-13T14:24:11.381166+0000) 2024-02-13T14:24:23.869 INFO:journalctl@ceph.osd.7.smithi148.stdout:Feb 13 14:24:23 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-7[54686]: 2024-02-13T14:24:23.507+0000 7f4025619700 -1 osd.7 395 heartbeat_check: no reply from 172.21.15.87:6830 osd.3 since back 2024-02-13T14:23:52.982720+0000 front 2024-02-13T14:23:52.982697+0000 (oldest deadline 2024-02-13T14:24:16.482807+0000) 2024-02-13T14:24:23.869 INFO:journalctl@ceph.osd.7.smithi148.stdout:Feb 13 14:24:23 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-7[54686]: 2024-02-13T14:24:23.508+0000 7f4025619700 -1 osd.7 395 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24409.0:493 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e395) 2024-02-13T14:24:23.991 INFO:journalctl@ceph.alertmanager.a.smithi087.stdout:Feb 13 14:24:23 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-alertmanager-a[78198]: ts=2024-02-13T14:24:23.532Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=2 err="ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://smithi087.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.87:8443: connect: connection refused; ceph-dashboard/webhook[1]: notify retry canceled after 8 attempts: Post \"https://smithi148.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.148:8443: connect: connection refused" 2024-02-13T14:24:23.992 INFO:journalctl@ceph.alertmanager.a.smithi087.stdout:Feb 13 14:24:23 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-alertmanager-a[78198]: ts=2024-02-13T14:24:23.532Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi087.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.87:8443: connect: connection refused" 2024-02-13T14:24:23.992 INFO:journalctl@ceph.alertmanager.a.smithi087.stdout:Feb 13 14:24:23 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-alertmanager-a[78198]: ts=2024-02-13T14:24:23.533Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi148.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.148:8443: connect: connection refused" 2024-02-13T14:24:24.869 INFO:journalctl@ceph.osd.7.smithi148.stdout:Feb 13 14:24:24 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-7[54686]: 2024-02-13T14:24:24.521+0000 7f4025619700 -1 osd.7 395 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-02-13T14:23:32.079364+0000 front 2024-02-13T14:23:32.079377+0000 (oldest deadline 2024-02-13T14:23:55.479861+0000) 2024-02-13T14:24:24.869 INFO:journalctl@ceph.osd.7.smithi148.stdout:Feb 13 14:24:24 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-7[54686]: 2024-02-13T14:24:24.521+0000 7f4025619700 -1 osd.7 395 heartbeat_check: no reply from 172.21.15.87:6814 osd.1 since back 2024-02-13T14:23:35.480357+0000 front 2024-02-13T14:23:35.480420+0000 (oldest deadline 2024-02-13T14:24:01.380377+0000) 2024-02-13T14:24:24.869 INFO:journalctl@ceph.osd.7.smithi148.stdout:Feb 13 14:24:24 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-7[54686]: 2024-02-13T14:24:24.521+0000 7f4025619700 -1 osd.7 395 heartbeat_check: no reply from 172.21.15.87:6822 osd.2 since back 2024-02-13T14:23:45.481305+0000 front 2024-02-13T14:23:45.481157+0000 (oldest deadline 2024-02-13T14:24:11.381166+0000) 2024-02-13T14:24:24.869 INFO:journalctl@ceph.osd.7.smithi148.stdout:Feb 13 14:24:24 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-7[54686]: 2024-02-13T14:24:24.521+0000 7f4025619700 -1 osd.7 395 heartbeat_check: no reply from 172.21.15.87:6830 osd.3 since back 2024-02-13T14:23:52.982720+0000 front 2024-02-13T14:23:52.982697+0000 (oldest deadline 2024-02-13T14:24:16.482807+0000) 2024-02-13T14:24:24.869 INFO:journalctl@ceph.osd.7.smithi148.stdout:Feb 13 14:24:24 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-7[54686]: 2024-02-13T14:24:24.521+0000 7f4025619700 -1 osd.7 395 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24409.0:493 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e395) 2024-02-13T14:24:25.869 INFO:journalctl@ceph.osd.7.smithi148.stdout:Feb 13 14:24:25 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-7[54686]: 2024-02-13T14:24:25.501+0000 7f4025619700 -1 osd.7 395 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-02-13T14:23:32.079364+0000 front 2024-02-13T14:23:32.079377+0000 (oldest deadline 2024-02-13T14:23:55.479861+0000) 2024-02-13T14:24:25.869 INFO:journalctl@ceph.osd.7.smithi148.stdout:Feb 13 14:24:25 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-7[54686]: 2024-02-13T14:24:25.501+0000 7f4025619700 -1 osd.7 395 heartbeat_check: no reply from 172.21.15.87:6814 osd.1 since back 2024-02-13T14:23:35.480357+0000 front 2024-02-13T14:23:35.480420+0000 (oldest deadline 2024-02-13T14:24:01.380377+0000) 2024-02-13T14:24:25.869 INFO:journalctl@ceph.osd.7.smithi148.stdout:Feb 13 14:24:25 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-7[54686]: 2024-02-13T14:24:25.501+0000 7f4025619700 -1 osd.7 395 heartbeat_check: no reply from 172.21.15.87:6822 osd.2 since back 2024-02-13T14:23:45.481305+0000 front 2024-02-13T14:23:45.481157+0000 (oldest deadline 2024-02-13T14:24:11.381166+0000) 2024-02-13T14:24:25.869 INFO:journalctl@ceph.osd.7.smithi148.stdout:Feb 13 14:24:25 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-7[54686]: 2024-02-13T14:24:25.501+0000 7f4025619700 -1 osd.7 395 heartbeat_check: no reply from 172.21.15.87:6830 osd.3 since back 2024-02-13T14:23:52.982720+0000 front 2024-02-13T14:23:52.982697+0000 (oldest deadline 2024-02-13T14:24:16.482807+0000) 2024-02-13T14:24:25.869 INFO:journalctl@ceph.osd.7.smithi148.stdout:Feb 13 14:24:25 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-7[54686]: 2024-02-13T14:24:25.501+0000 7f4025619700 -1 osd.7 395 heartbeat_check: no reply from 172.21.15.148:6804 osd.4 since back 2024-02-13T14:24:00.583495+0000 front 2024-02-13T14:24:00.583581+0000 (oldest deadline 2024-02-13T14:24:24.683547+0000) 2024-02-13T14:24:25.869 INFO:journalctl@ceph.osd.7.smithi148.stdout:Feb 13 14:24:25 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-7[54686]: 2024-02-13T14:24:25.501+0000 7f4025619700 -1 osd.7 395 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24409.0:493 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e395) 2024-02-13T14:24:26.789 INFO:journalctl@ceph.osd.7.smithi148.stdout:Feb 13 14:24:26 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-7[54686]: 2024-02-13T14:24:26.526+0000 7f4025619700 -1 osd.7 395 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-02-13T14:23:32.079364+0000 front 2024-02-13T14:23:32.079377+0000 (oldest deadline 2024-02-13T14:23:55.479861+0000) 2024-02-13T14:24:26.790 INFO:journalctl@ceph.osd.7.smithi148.stdout:Feb 13 14:24:26 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-7[54686]: 2024-02-13T14:24:26.526+0000 7f4025619700 -1 osd.7 395 heartbeat_check: no reply from 172.21.15.87:6814 osd.1 since back 2024-02-13T14:23:35.480357+0000 front 2024-02-13T14:23:35.480420+0000 (oldest deadline 2024-02-13T14:24:01.380377+0000) 2024-02-13T14:24:26.790 INFO:journalctl@ceph.osd.7.smithi148.stdout:Feb 13 14:24:26 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-7[54686]: 2024-02-13T14:24:26.526+0000 7f4025619700 -1 osd.7 395 heartbeat_check: no reply from 172.21.15.87:6822 osd.2 since back 2024-02-13T14:23:45.481305+0000 front 2024-02-13T14:23:45.481157+0000 (oldest deadline 2024-02-13T14:24:11.381166+0000) 2024-02-13T14:24:26.790 INFO:journalctl@ceph.osd.7.smithi148.stdout:Feb 13 14:24:26 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-7[54686]: 2024-02-13T14:24:26.526+0000 7f4025619700 -1 osd.7 395 heartbeat_check: no reply from 172.21.15.87:6830 osd.3 since back 2024-02-13T14:23:52.982720+0000 front 2024-02-13T14:23:52.982697+0000 (oldest deadline 2024-02-13T14:24:16.482807+0000) 2024-02-13T14:24:26.790 INFO:journalctl@ceph.osd.7.smithi148.stdout:Feb 13 14:24:26 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-7[54686]: 2024-02-13T14:24:26.526+0000 7f4025619700 -1 osd.7 395 heartbeat_check: no reply from 172.21.15.148:6804 osd.4 since back 2024-02-13T14:24:00.583495+0000 front 2024-02-13T14:24:00.583581+0000 (oldest deadline 2024-02-13T14:24:24.683547+0000) 2024-02-13T14:24:26.790 INFO:journalctl@ceph.osd.7.smithi148.stdout:Feb 13 14:24:26 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-7[54686]: 2024-02-13T14:24:26.526+0000 7f4025619700 -1 osd.7 395 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24409.0:493 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e395) 2024-02-13T14:24:27.745 INFO:journalctl@ceph.osd.7.smithi148.stdout:Feb 13 14:24:27 smithi148 podman[68307]: 2024-02-13 14:24:27.429221776 +0000 UTC m=+5.662384134 container died 3b63c1fbaaaf67f7d3714632000998bf30f8c8a26a0f04db01a091a455f55f93 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-7, io.buildah.version=1.33.2, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240131, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.schema-version=1.0) 2024-02-13T14:24:28.004 INFO:journalctl@ceph.osd.7.smithi148.stdout:Feb 13 14:24:27 smithi148 podman[68307]: 2024-02-13 14:24:27.74416734 +0000 UTC m=+5.977329691 container cleanup 3b63c1fbaaaf67f7d3714632000998bf30f8c8a26a0f04db01a091a455f55f93 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-7, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240131, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.2, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-13T14:24:28.004 INFO:journalctl@ceph.osd.7.smithi148.stdout:Feb 13 14:24:27 smithi148 bash[68307]: ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-7 2024-02-13T14:24:28.004 INFO:journalctl@ceph.osd.7.smithi148.stdout:Feb 13 14:24:27 smithi148 podman[68338]: 2024-02-13 14:24:27.978640746 +0000 UTC m=+0.544852706 container remove 3b63c1fbaaaf67f7d3714632000998bf30f8c8a26a0f04db01a091a455f55f93 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-7, org.label-schema.build-date=20240131, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.2, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, maintainer=Guillaume Abrioux ) 2024-02-13T14:24:28.682 INFO:journalctl@ceph.osd.7.smithi148.stdout:Feb 13 14:24:28 smithi148 podman[68437]: 2024-02-13 14:24:28.301578016 +0000 UTC m=+0.018811966 image pull 28480620e5edd298d55c0e6673dcbad9ed23edd37269028a11063c1b8925676a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f 2024-02-13T14:24:28.682 INFO:journalctl@ceph.osd.7.smithi148.stdout:Feb 13 14:24:28 smithi148 podman[68437]: 2024-02-13 14:24:28.413114248 +0000 UTC m=+0.130348200 container create 7d745f126130658005413e116675edfcc2e580c7a86e2ed2f90360b3e8cdeb40 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-7-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240131, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, GIT_CLEAN=True, io.buildah.version=1.33.2, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-02-13T14:24:28.682 INFO:journalctl@ceph.osd.7.smithi148.stdout:Feb 13 14:24:28 smithi148 podman[68437]: 2024-02-13 14:24:28.59885264 +0000 UTC m=+0.316086596 container init 7d745f126130658005413e116675edfcc2e580c7a86e2ed2f90360b3e8cdeb40 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-7-deactivate, org.label-schema.build-date=20240131, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.2, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, CEPH_POINT_RELEASE=) 2024-02-13T14:24:28.682 INFO:journalctl@ceph.osd.7.smithi148.stdout:Feb 13 14:24:28 smithi148 podman[68437]: 2024-02-13 14:24:28.604713397 +0000 UTC m=+0.321947353 container start 7d745f126130658005413e116675edfcc2e580c7a86e2ed2f90360b3e8cdeb40 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-7-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.2, ceph=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240131) 2024-02-13T14:24:28.683 INFO:journalctl@ceph.osd.7.smithi148.stdout:Feb 13 14:24:28 smithi148 podman[68437]: 2024-02-13 14:24:28.681685912 +0000 UTC m=+0.398919863 container attach 7d745f126130658005413e116675edfcc2e580c7a86e2ed2f90360b3e8cdeb40 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-7-deactivate, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_BRANCH=HEAD, org.label-schema.build-date=20240131, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-13T14:24:29.118 INFO:journalctl@ceph.osd.7.smithi148.stdout:Feb 13 14:24:28 smithi148 podman[68437]: 2024-02-13 14:24:28.900903525 +0000 UTC m=+0.618137482 container died 7d745f126130658005413e116675edfcc2e580c7a86e2ed2f90360b3e8cdeb40 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-7-deactivate, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240131, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.2) 2024-02-13T14:24:29.439 DEBUG:teuthology.orchestra.run.smithi148:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5c04b074-ca79-11ee-95b9-87774f69a715@osd.7.service' 2024-02-13T14:24:29.465 INFO:journalctl@ceph.osd.7.smithi148.stdout:Feb 13 14:24:29 smithi148 podman[68483]: 2024-02-13 14:24:29.409351719 +0000 UTC m=+0.498016896 container remove 7d745f126130658005413e116675edfcc2e580c7a86e2ed2f90360b3e8cdeb40 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-osd-7-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, maintainer=Guillaume Abrioux , GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.schema-version=1.0, io.buildah.version=1.33.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240131, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-02-13T14:24:29.466 INFO:journalctl@ceph.osd.7.smithi148.stdout:Feb 13 14:24:29 smithi148 systemd[1]: ceph-5c04b074-ca79-11ee-95b9-87774f69a715@osd.7.service: Deactivated successfully. 2024-02-13T14:24:29.466 INFO:journalctl@ceph.osd.7.smithi148.stdout:Feb 13 14:24:29 smithi148 systemd[1]: Stopped Ceph osd.7 for 5c04b074-ca79-11ee-95b9-87774f69a715. 2024-02-13T14:24:29.466 INFO:journalctl@ceph.osd.7.smithi148.stdout:Feb 13 14:24:29 smithi148 systemd[1]: ceph-5c04b074-ca79-11ee-95b9-87774f69a715@osd.7.service: Consumed 15.119s CPU time. 2024-02-13T14:24:29.481 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-13T14:24:29.481 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-02-13T14:24:29.481 INFO:tasks.cephadm.ceph.rgw.foo.a:Stopping rgw.foo.a... 2024-02-13T14:24:29.481 DEBUG:teuthology.orchestra.run.smithi087:> sudo systemctl stop ceph-5c04b074-ca79-11ee-95b9-87774f69a715@rgw.foo.a 2024-02-13T14:24:29.991 INFO:journalctl@ceph.rgw.foo.a.smithi087.stdout:Feb 13 14:24:29 smithi087 systemd[1]: Stopping Ceph rgw.foo.a for 5c04b074-ca79-11ee-95b9-87774f69a715... 2024-02-13T14:24:29.992 INFO:journalctl@ceph.rgw.foo.a.smithi087.stdout:Feb 13 14:24:29 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-rgw-foo-a[67192]: 2024-02-13T14:24:29.754+0000 7f103e60c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/radosgw -n client.rgw.foo.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-13T14:24:29.992 INFO:journalctl@ceph.rgw.foo.a.smithi087.stdout:Feb 13 14:24:29 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-rgw-foo-a[67192]: 2024-02-13T14:24:29.754+0000 7f104ce7dac0 -1 shutting down 2024-02-13T14:24:33.992 INFO:journalctl@ceph.alertmanager.a.smithi087.stdout:Feb 13 14:24:33 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-alertmanager-a[78198]: ts=2024-02-13T14:24:33.532Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=3 err="ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://smithi148.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.148:8443: connect: connection refused; ceph-dashboard/webhook[0]: notify retry canceled after 8 attempts: Post \"https://smithi087.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.87:8443: connect: connection refused" 2024-02-13T14:24:33.992 INFO:journalctl@ceph.alertmanager.a.smithi087.stdout:Feb 13 14:24:33 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-alertmanager-a[78198]: ts=2024-02-13T14:24:33.533Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi087.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.87:8443: connect: connection refused" 2024-02-13T14:24:33.992 INFO:journalctl@ceph.alertmanager.a.smithi087.stdout:Feb 13 14:24:33 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-alertmanager-a[78198]: ts=2024-02-13T14:24:33.534Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi148.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.148:8443: connect: connection refused" 2024-02-13T14:24:40.054 INFO:journalctl@ceph.rgw.foo.a.smithi087.stdout:Feb 13 14:24:39 smithi087 bash[83369]: time="2024-02-13T14:24:39Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-5c04b074-ca79-11ee-95b9-87774f69a715-rgw-foo-a in 10 seconds, resorting to SIGKILL" 2024-02-13T14:24:40.054 INFO:journalctl@ceph.rgw.foo.a.smithi087.stdout:Feb 13 14:24:39 smithi087 podman[83369]: 2024-02-13 14:24:39.779034314 +0000 UTC m=+10.106591614 container died 77dce37a552b69f34de726522f2be80384bea6754b5d4d82a38a3bb53b6d2ee8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-rgw-foo-a, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240131, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, io.buildah.version=1.33.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-13T14:24:40.421 INFO:journalctl@ceph.rgw.foo.a.smithi087.stdout:Feb 13 14:24:40 smithi087 podman[83369]: 2024-02-13 14:24:40.143594042 +0000 UTC m=+10.471151343 container cleanup 77dce37a552b69f34de726522f2be80384bea6754b5d4d82a38a3bb53b6d2ee8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-rgw-foo-a, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240131, org.label-schema.license=GPLv2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.2, maintainer=Guillaume Abrioux , RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5) 2024-02-13T14:24:40.421 INFO:journalctl@ceph.rgw.foo.a.smithi087.stdout:Feb 13 14:24:40 smithi087 bash[83369]: ceph-5c04b074-ca79-11ee-95b9-87774f69a715-rgw-foo-a 2024-02-13T14:24:40.587 DEBUG:teuthology.orchestra.run.smithi087:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5c04b074-ca79-11ee-95b9-87774f69a715@rgw.foo.a.service' 2024-02-13T14:24:40.627 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-13T14:24:40.628 INFO:tasks.cephadm.ceph.rgw.foo.a:Stopped rgw.foo.a 2024-02-13T14:24:40.628 INFO:tasks.cephadm.prometheus.a:Stopping prometheus.a... 2024-02-13T14:24:40.628 DEBUG:teuthology.orchestra.run.smithi148:> sudo systemctl stop ceph-5c04b074-ca79-11ee-95b9-87774f69a715@prometheus.a 2024-02-13T14:24:40.914 INFO:journalctl@ceph.prometheus.a.smithi148.stdout:Feb 13 14:24:40 smithi148 systemd[1]: Stopping Ceph prometheus.a for 5c04b074-ca79-11ee-95b9-87774f69a715... 2024-02-13T14:24:41.244 INFO:journalctl@ceph.prometheus.a.smithi148.stdout:Feb 13 14:24:40 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-prometheus-a[65166]: ts=2024-02-13T14:24:40.913Z caller=main.go:840 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-02-13T14:24:41.245 INFO:journalctl@ceph.prometheus.a.smithi148.stdout:Feb 13 14:24:40 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-prometheus-a[65166]: ts=2024-02-13T14:24:40.913Z caller=main.go:864 level=info msg="Stopping scrape discovery manager..." 2024-02-13T14:24:41.245 INFO:journalctl@ceph.prometheus.a.smithi148.stdout:Feb 13 14:24:40 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-prometheus-a[65166]: ts=2024-02-13T14:24:40.913Z caller=main.go:878 level=info msg="Stopping notify discovery manager..." 2024-02-13T14:24:41.245 INFO:journalctl@ceph.prometheus.a.smithi148.stdout:Feb 13 14:24:40 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-prometheus-a[65166]: ts=2024-02-13T14:24:40.913Z caller=manager.go:988 level=info component="rule manager" msg="Stopping rule manager..." 2024-02-13T14:24:41.245 INFO:journalctl@ceph.prometheus.a.smithi148.stdout:Feb 13 14:24:40 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-prometheus-a[65166]: ts=2024-02-13T14:24:40.913Z caller=main.go:860 level=info msg="Scrape discovery manager stopped" 2024-02-13T14:24:41.245 INFO:journalctl@ceph.prometheus.a.smithi148.stdout:Feb 13 14:24:40 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-prometheus-a[65166]: ts=2024-02-13T14:24:40.913Z caller=main.go:874 level=info msg="Notify discovery manager stopped" 2024-02-13T14:24:41.245 INFO:journalctl@ceph.prometheus.a.smithi148.stdout:Feb 13 14:24:40 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-prometheus-a[65166]: ts=2024-02-13T14:24:40.913Z caller=manager.go:998 level=info component="rule manager" msg="Rule manager stopped" 2024-02-13T14:24:41.245 INFO:journalctl@ceph.prometheus.a.smithi148.stdout:Feb 13 14:24:40 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-prometheus-a[65166]: ts=2024-02-13T14:24:40.913Z caller=main.go:915 level=info msg="Stopping scrape manager..." 2024-02-13T14:24:41.245 INFO:journalctl@ceph.prometheus.a.smithi148.stdout:Feb 13 14:24:40 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-prometheus-a[65166]: ts=2024-02-13T14:24:40.914Z caller=main.go:907 level=info msg="Scrape manager stopped" 2024-02-13T14:24:41.245 INFO:journalctl@ceph.prometheus.a.smithi148.stdout:Feb 13 14:24:40 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-prometheus-a[65166]: ts=2024-02-13T14:24:40.914Z caller=notifier.go:606 level=info component=notifier msg="Stopping notification manager..." 2024-02-13T14:24:41.245 INFO:journalctl@ceph.prometheus.a.smithi148.stdout:Feb 13 14:24:40 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-prometheus-a[65166]: ts=2024-02-13T14:24:40.914Z caller=main.go:1135 level=info msg="Notifier manager stopped" 2024-02-13T14:24:41.245 INFO:journalctl@ceph.prometheus.a.smithi148.stdout:Feb 13 14:24:40 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-prometheus-a[65166]: ts=2024-02-13T14:24:40.914Z caller=main.go:1147 level=info msg="See you next time!" 2024-02-13T14:24:41.245 INFO:journalctl@ceph.prometheus.a.smithi148.stdout:Feb 13 14:24:40 smithi148 podman[68569]: 2024-02-13 14:24:40.924091302 +0000 UTC m=+0.108440155 container died 9e8219ab6c3626065eb826335a46d80b0311ba678f304b58499cbf5213ec454e (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-02-13T14:24:41.245 INFO:journalctl@ceph.prometheus.a.smithi148.stdout:Feb 13 14:24:41 smithi148 podman[68569]: 2024-02-13 14:24:41.243800154 +0000 UTC m=+0.428149007 container cleanup 9e8219ab6c3626065eb826335a46d80b0311ba678f304b58499cbf5213ec454e (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-02-13T14:24:41.245 INFO:journalctl@ceph.prometheus.a.smithi148.stdout:Feb 13 14:24:41 smithi148 bash[68569]: ceph-5c04b074-ca79-11ee-95b9-87774f69a715-prometheus-a 2024-02-13T14:24:41.618 INFO:journalctl@ceph.prometheus.a.smithi148.stdout:Feb 13 14:24:41 smithi148 podman[68589]: 2024-02-13 14:24:41.41181721 +0000 UTC m=+0.490725641 container remove 9e8219ab6c3626065eb826335a46d80b0311ba678f304b58499cbf5213ec454e (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-02-13T14:24:41.649 DEBUG:teuthology.orchestra.run.smithi148:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5c04b074-ca79-11ee-95b9-87774f69a715@prometheus.a.service' 2024-02-13T14:24:41.686 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-13T14:24:41.686 INFO:tasks.cephadm.prometheus.a:Stopped prometheus.a 2024-02-13T14:24:41.686 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 5c04b074-ca79-11ee-95b9-87774f69a715 --force --keep-logs 2024-02-13T14:24:41.875 INFO:teuthology.orchestra.run.smithi087.stdout:Deleting cluster with fsid: 5c04b074-ca79-11ee-95b9-87774f69a715 2024-02-13T14:24:43.719 INFO:journalctl@ceph.alertmanager.a.smithi087.stdout:Feb 13 14:24:43 smithi087 systemd[1]: Stopping Ceph alertmanager.a for 5c04b074-ca79-11ee-95b9-87774f69a715... 2024-02-13T14:24:43.719 INFO:journalctl@ceph.alertmanager.a.smithi087.stdout:Feb 13 14:24:43 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-alertmanager-a[78198]: ts=2024-02-13T14:24:43.533Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=3 err="ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://smithi148.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.148:8443: connect: connection refused; ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://smithi087.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.87:8443: connect: connection refused" 2024-02-13T14:24:43.719 INFO:journalctl@ceph.alertmanager.a.smithi087.stdout:Feb 13 14:24:43 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-alertmanager-a[78198]: ts=2024-02-13T14:24:43.534Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi087.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.87:8443: connect: connection refused" 2024-02-13T14:24:43.719 INFO:journalctl@ceph.alertmanager.a.smithi087.stdout:Feb 13 14:24:43 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-alertmanager-a[78198]: ts=2024-02-13T14:24:43.534Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi148.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.148:8443: connect: connection refused" 2024-02-13T14:24:43.719 INFO:journalctl@ceph.alertmanager.a.smithi087.stdout:Feb 13 14:24:43 smithi087 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-alertmanager-a[78198]: ts=2024-02-13T14:24:43.718Z caller=main.go:583 level=info msg="Received SIGTERM, exiting gracefully..." 2024-02-13T14:24:43.992 INFO:journalctl@ceph.alertmanager.a.smithi087.stdout:Feb 13 14:24:43 smithi087 podman[83889]: 2024-02-13 14:24:43.75923155 +0000 UTC m=+0.129611400 container died 5ff9960e3532efa0da23b7e5ec58abb52f1db388d33b0836d78fde2c36a85582 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-02-13T14:24:44.317 INFO:journalctl@ceph.alertmanager.a.smithi087.stdout:Feb 13 14:24:44 smithi087 podman[83889]: 2024-02-13 14:24:44.09898457 +0000 UTC m=+0.469364421 container cleanup 5ff9960e3532efa0da23b7e5ec58abb52f1db388d33b0836d78fde2c36a85582 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-02-13T14:24:44.318 INFO:journalctl@ceph.alertmanager.a.smithi087.stdout:Feb 13 14:24:44 smithi087 bash[83889]: ceph-5c04b074-ca79-11ee-95b9-87774f69a715-alertmanager-a 2024-02-13T14:24:44.318 INFO:journalctl@ceph.alertmanager.a.smithi087.stdout:Feb 13 14:24:44 smithi087 podman[83908]: 2024-02-13 14:24:44.233198491 +0000 UTC m=+0.468266355 container remove 5ff9960e3532efa0da23b7e5ec58abb52f1db388d33b0836d78fde2c36a85582 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-02-13T14:24:44.318 INFO:journalctl@ceph.alertmanager.a.smithi087.stdout:Feb 13 14:24:44 smithi087 podman[83908]: 2024-02-13 14:24:44.317289789 +0000 UTC m=+0.552357665 volume remove a271d9f9aefb5aeb0c3be1410be445f36ff9ee7b92445149df260944afcf4da5 2024-02-13T14:24:44.736 INFO:journalctl@ceph.alertmanager.a.smithi087.stdout:Feb 13 14:24:44 smithi087 systemd[1]: ceph-5c04b074-ca79-11ee-95b9-87774f69a715@alertmanager.a.service: Deactivated successfully. 2024-02-13T14:24:44.736 INFO:journalctl@ceph.alertmanager.a.smithi087.stdout:Feb 13 14:24:44 smithi087 systemd[1]: Stopped Ceph alertmanager.a for 5c04b074-ca79-11ee-95b9-87774f69a715. 2024-02-13T14:24:44.736 INFO:journalctl@ceph.alertmanager.a.smithi087.stdout:Feb 13 14:24:44 smithi087 systemd[1]: ceph-5c04b074-ca79-11ee-95b9-87774f69a715@alertmanager.a.service: Consumed 1.250s CPU time. 2024-02-13T14:24:44.991 INFO:journalctl@ceph.node-exporter.a.smithi087.stdout:Feb 13 14:24:44 smithi087 systemd[1]: Stopping Ceph node-exporter.a for 5c04b074-ca79-11ee-95b9-87774f69a715... 2024-02-13T14:24:44.992 INFO:journalctl@ceph.node-exporter.a.smithi087.stdout:Feb 13 14:24:44 smithi087 podman[84023]: 2024-02-13 14:24:44.983523229 +0000 UTC m=+0.114773439 container died af62e3ac6b0a15606ddf42ea6f3d4efe42d8a32c8dcf2425a6279a31597f7d08 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-node-exporter-a, maintainer=The Prometheus Authors ) 2024-02-13T14:24:45.588 INFO:journalctl@ceph.node-exporter.a.smithi087.stdout:Feb 13 14:24:45 smithi087 podman[84023]: 2024-02-13 14:24:45.395150114 +0000 UTC m=+0.526400334 container cleanup af62e3ac6b0a15606ddf42ea6f3d4efe42d8a32c8dcf2425a6279a31597f7d08 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-node-exporter-a, maintainer=The Prometheus Authors ) 2024-02-13T14:24:45.588 INFO:journalctl@ceph.node-exporter.a.smithi087.stdout:Feb 13 14:24:45 smithi087 bash[84023]: ceph-5c04b074-ca79-11ee-95b9-87774f69a715-node-exporter-a 2024-02-13T14:24:45.948 INFO:journalctl@ceph.node-exporter.a.smithi087.stdout:Feb 13 14:24:45 smithi087 podman[84042]: 2024-02-13 14:24:45.587839112 +0000 UTC m=+0.609018420 container remove af62e3ac6b0a15606ddf42ea6f3d4efe42d8a32c8dcf2425a6279a31597f7d08 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-node-exporter-a, maintainer=The Prometheus Authors ) 2024-02-13T14:24:45.948 INFO:journalctl@ceph.node-exporter.a.smithi087.stdout:Feb 13 14:24:45 smithi087 systemd[1]: ceph-5c04b074-ca79-11ee-95b9-87774f69a715@node-exporter.a.service: Main process exited, code=exited, status=143/n/a 2024-02-13T14:24:45.948 INFO:journalctl@ceph.node-exporter.a.smithi087.stdout:Feb 13 14:24:45 smithi087 systemd[1]: ceph-5c04b074-ca79-11ee-95b9-87774f69a715@node-exporter.a.service: Failed with result 'exit-code'. 2024-02-13T14:24:45.948 INFO:journalctl@ceph.node-exporter.a.smithi087.stdout:Feb 13 14:24:45 smithi087 systemd[1]: Stopped Ceph node-exporter.a for 5c04b074-ca79-11ee-95b9-87774f69a715. 2024-02-13T14:24:45.948 INFO:journalctl@ceph.node-exporter.a.smithi087.stdout:Feb 13 14:24:45 smithi087 systemd[1]: ceph-5c04b074-ca79-11ee-95b9-87774f69a715@node-exporter.a.service: Consumed 2.438s CPU time. 2024-02-13T14:24:46.435 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 5c04b074-ca79-11ee-95b9-87774f69a715 --force --keep-logs 2024-02-13T14:24:46.627 INFO:teuthology.orchestra.run.smithi148.stdout:Deleting cluster with fsid: 5c04b074-ca79-11ee-95b9-87774f69a715 2024-02-13T14:24:48.118 INFO:journalctl@ceph.iscsi.iscsi.a.smithi148.stdout:Feb 13 14:24:47 smithi148 systemd[1]: Stopping Ceph iscsi.iscsi.a for 5c04b074-ca79-11ee-95b9-87774f69a715... 2024-02-13T14:24:48.618 INFO:journalctl@ceph.iscsi.iscsi.a.smithi148.stdout:Feb 13 14:24:48 smithi148 ceph-5c04b074-ca79-11ee-95b9-87774f69a715-iscsi-iscsi-a[59644]: debug Shutdown received 2024-02-13T14:24:58.457 INFO:journalctl@ceph.iscsi.iscsi.a.smithi148.stdout:Feb 13 14:24:58 smithi148 bash[68988]: time="2024-02-13T14:24:58Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-5c04b074-ca79-11ee-95b9-87774f69a715-iscsi-iscsi-a in 10 seconds, resorting to SIGKILL" 2024-02-13T14:24:58.457 INFO:journalctl@ceph.iscsi.iscsi.a.smithi148.stdout:Feb 13 14:24:58 smithi148 podman[68988]: 2024-02-13 14:24:58.176239263 +0000 UTC m=+10.136431072 container died fb74c3c1636a61cb3b30bc126b6e04cf9a478cad3ab539e2c543e2e32f4f5a8c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-iscsi-iscsi-a, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, ceph=True, io.buildah.version=1.33.2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240131, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-02-13T14:24:58.816 INFO:journalctl@ceph.iscsi.iscsi.a.smithi148.stdout:Feb 13 14:24:58 smithi148 podman[68988]: 2024-02-13 14:24:58.538413371 +0000 UTC m=+10.498605221 container cleanup fb74c3c1636a61cb3b30bc126b6e04cf9a478cad3ab539e2c543e2e32f4f5a8c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-iscsi-iscsi-a, ceph=True, io.buildah.version=1.33.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, org.label-schema.build-date=20240131, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-02-13T14:24:58.816 INFO:journalctl@ceph.iscsi.iscsi.a.smithi148.stdout:Feb 13 14:24:58 smithi148 bash[68988]: ceph-5c04b074-ca79-11ee-95b9-87774f69a715-iscsi-iscsi-a 2024-02-13T14:24:59.118 INFO:journalctl@ceph.iscsi.iscsi.a.smithi148.stdout:Feb 13 14:24:58 smithi148 podman[69013]: 2024-02-13 14:24:58.814279141 +0000 UTC m=+0.639086499 container remove fb74c3c1636a61cb3b30bc126b6e04cf9a478cad3ab539e2c543e2e32f4f5a8c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-iscsi-iscsi-a, org.label-schema.license=GPLv2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240131, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, io.buildah.version=1.33.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-02-13T14:24:59.118 INFO:journalctl@ceph.iscsi.iscsi.a.smithi148.stdout:Feb 13 14:24:58 smithi148 systemd[1]: ceph-5c04b074-ca79-11ee-95b9-87774f69a715@iscsi.iscsi.a.service: Main process exited, code=exited, status=137/n/a 2024-02-13T14:24:59.118 INFO:journalctl@ceph.iscsi.iscsi.a.smithi148.stdout:Feb 13 14:24:58 smithi148 systemd[1]: ceph-5c04b074-ca79-11ee-95b9-87774f69a715@iscsi.iscsi.a.service: Failed with result 'exit-code'. 2024-02-13T14:24:59.119 INFO:journalctl@ceph.iscsi.iscsi.a.smithi148.stdout:Feb 13 14:24:58 smithi148 systemd[1]: Stopped Ceph iscsi.iscsi.a for 5c04b074-ca79-11ee-95b9-87774f69a715. 2024-02-13T14:24:59.119 INFO:journalctl@ceph.iscsi.iscsi.a.smithi148.stdout:Feb 13 14:24:58 smithi148 systemd[1]: ceph-5c04b074-ca79-11ee-95b9-87774f69a715@iscsi.iscsi.a.service: Consumed 2.727s CPU time. 2024-02-13T14:25:00.728 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:25:00 smithi148 systemd[1]: Stopping Ceph grafana.a for 5c04b074-ca79-11ee-95b9-87774f69a715... 2024-02-13T14:25:01.005 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:25:00 smithi148 podman[69302]: 2024-02-13 14:25:00.725972704 +0000 UTC m=+0.117922464 container died c87535293afd6bf0cc3f025d1f7af9f40a3411572d29b658fa7fbec31f287d2b (image=quay.io/ceph/ceph-grafana:9.4.12, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a, architecture=x86_64, io.openshift.tags=base rhel8, com.redhat.component=ubi8-container, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, io.openshift.expose-services=, maintainer=Paul Cuzner , vendor=Red Hat, Inc., summary=Grafana Container configured for Ceph mgr/dashboard integration, distribution-scope=public, vcs-type=git, name=ubi8, description=Ceph Grafana Container, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, io.buildah.version=1.32.0, io.k8s.display-name=Red Hat Universal Base Image 8, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, release=239.1651231664, version=8.5, build-date=2022-04-29T11:48:11.055897, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly.) 2024-02-13T14:25:01.328 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:25:01 smithi148 podman[69302]: 2024-02-13 14:25:01.070511895 +0000 UTC m=+0.462461639 container cleanup c87535293afd6bf0cc3f025d1f7af9f40a3411572d29b658fa7fbec31f287d2b (image=quay.io/ceph/ceph-grafana:9.4.12, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a, io.openshift.tags=base rhel8, description=Ceph Grafana Container, io.openshift.expose-services=, io.k8s.display-name=Red Hat Universal Base Image 8, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, build-date=2022-04-29T11:48:11.055897, architecture=x86_64, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, distribution-scope=public, vendor=Red Hat, Inc., com.redhat.component=ubi8-container, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, vcs-type=git, version=8.5, release=239.1651231664, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., io.buildah.version=1.32.0, name=ubi8, summary=Grafana Container configured for Ceph mgr/dashboard integration, maintainer=Paul Cuzner ) 2024-02-13T14:25:01.328 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:25:01 smithi148 bash[69302]: ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a 2024-02-13T14:25:01.618 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:25:01 smithi148 podman[69321]: 2024-02-13 14:25:01.33818139 +0000 UTC m=+0.615136170 container remove c87535293afd6bf0cc3f025d1f7af9f40a3411572d29b658fa7fbec31f287d2b (image=quay.io/ceph/ceph-grafana:9.4.12, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-grafana-a, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., distribution-scope=public, io.openshift.expose-services=, io.buildah.version=1.32.0, name=ubi8, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, summary=Grafana Container configured for Ceph mgr/dashboard integration, architecture=x86_64, com.redhat.component=ubi8-container, vcs-type=git, vendor=Red Hat, Inc., description=Ceph Grafana Container, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, io.openshift.tags=base rhel8, version=8.5, maintainer=Paul Cuzner , com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, io.k8s.display-name=Red Hat Universal Base Image 8, release=239.1651231664, build-date=2022-04-29T11:48:11.055897, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI) 2024-02-13T14:25:01.619 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:25:01 smithi148 systemd[1]: ceph-5c04b074-ca79-11ee-95b9-87774f69a715@grafana.a.service: Main process exited, code=exited, status=143/n/a 2024-02-13T14:25:01.619 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:25:01 smithi148 systemd[1]: ceph-5c04b074-ca79-11ee-95b9-87774f69a715@grafana.a.service: Failed with result 'exit-code'. 2024-02-13T14:25:01.619 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:25:01 smithi148 systemd[1]: Stopped Ceph grafana.a for 5c04b074-ca79-11ee-95b9-87774f69a715. 2024-02-13T14:25:01.619 INFO:journalctl@ceph.grafana.a.smithi148.stdout:Feb 13 14:25:01 smithi148 systemd[1]: ceph-5c04b074-ca79-11ee-95b9-87774f69a715@grafana.a.service: Consumed 6.265s CPU time. 2024-02-13T14:25:02.118 INFO:journalctl@ceph.node-exporter.b.smithi148.stdout:Feb 13 14:25:01 smithi148 systemd[1]: Stopping Ceph node-exporter.b for 5c04b074-ca79-11ee-95b9-87774f69a715... 2024-02-13T14:25:02.119 INFO:journalctl@ceph.node-exporter.b.smithi148.stdout:Feb 13 14:25:01 smithi148 podman[69440]: 2024-02-13 14:25:01.954793477 +0000 UTC m=+0.109386681 container died d7d208c49b58e9e0c15345a8dd874a1b8e1f3222f758b3be3e848a81fd094bc1 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-node-exporter-b, maintainer=The Prometheus Authors ) 2024-02-13T14:25:02.459 INFO:journalctl@ceph.node-exporter.b.smithi148.stdout:Feb 13 14:25:02 smithi148 podman[69440]: 2024-02-13 14:25:02.274105332 +0000 UTC m=+0.428698536 container cleanup d7d208c49b58e9e0c15345a8dd874a1b8e1f3222f758b3be3e848a81fd094bc1 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-node-exporter-b, maintainer=The Prometheus Authors ) 2024-02-13T14:25:02.459 INFO:journalctl@ceph.node-exporter.b.smithi148.stdout:Feb 13 14:25:02 smithi148 bash[69440]: ceph-5c04b074-ca79-11ee-95b9-87774f69a715-node-exporter-b 2024-02-13T14:25:02.837 INFO:journalctl@ceph.node-exporter.b.smithi148.stdout:Feb 13 14:25:02 smithi148 podman[69459]: 2024-02-13 14:25:02.458593401 +0000 UTC m=+0.505538216 container remove d7d208c49b58e9e0c15345a8dd874a1b8e1f3222f758b3be3e848a81fd094bc1 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-5c04b074-ca79-11ee-95b9-87774f69a715-node-exporter-b, maintainer=The Prometheus Authors ) 2024-02-13T14:25:02.837 INFO:journalctl@ceph.node-exporter.b.smithi148.stdout:Feb 13 14:25:02 smithi148 systemd[1]: ceph-5c04b074-ca79-11ee-95b9-87774f69a715@node-exporter.b.service: Main process exited, code=exited, status=143/n/a 2024-02-13T14:25:02.838 INFO:journalctl@ceph.node-exporter.b.smithi148.stdout:Feb 13 14:25:02 smithi148 systemd[1]: ceph-5c04b074-ca79-11ee-95b9-87774f69a715@node-exporter.b.service: Failed with result 'exit-code'. 2024-02-13T14:25:02.838 INFO:journalctl@ceph.node-exporter.b.smithi148.stdout:Feb 13 14:25:02 smithi148 systemd[1]: Stopped Ceph node-exporter.b for 5c04b074-ca79-11ee-95b9-87774f69a715. 2024-02-13T14:25:02.838 INFO:journalctl@ceph.node-exporter.b.smithi148.stdout:Feb 13 14:25:02 smithi148 systemd[1]: ceph-5c04b074-ca79-11ee-95b9-87774f69a715@node-exporter.b.service: Consumed 2.472s CPU time. 2024-02-13T14:25:03.321 DEBUG:teuthology.orchestra.run.smithi087:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-13T14:25:03.356 DEBUG:teuthology.orchestra.run.smithi148:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-13T14:25:03.386 INFO:tasks.cephadm:Archiving crash dumps... 2024-02-13T14:25:03.388 DEBUG:teuthology.misc:Transferring archived files from smithi087:/var/lib/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/crash to /home/teuthworker/archive/phlogistonjohn-2024-02-13_11:41:43-orch:cephadm-wip-phlogistonjohn-testing-2024-02-12-1654-distro-default-smithi/7558012/remote/smithi087/crash 2024-02-13T14:25:03.389 DEBUG:teuthology.orchestra.run.smithi087:> sudo tar c -f - -C /var/lib/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/crash -- . 2024-02-13T14:25:03.441 INFO:teuthology.orchestra.run.smithi087.stderr:tar: /var/lib/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/crash: Cannot open: No such file or directory 2024-02-13T14:25:03.441 INFO:teuthology.orchestra.run.smithi087.stderr:tar: Error is not recoverable: exiting now 2024-02-13T14:25:03.443 DEBUG:teuthology.misc:Transferring archived files from smithi148:/var/lib/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/crash to /home/teuthworker/archive/phlogistonjohn-2024-02-13_11:41:43-orch:cephadm-wip-phlogistonjohn-testing-2024-02-12-1654-distro-default-smithi/7558012/remote/smithi148/crash 2024-02-13T14:25:03.444 DEBUG:teuthology.orchestra.run.smithi148:> sudo tar c -f - -C /var/lib/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/crash -- . 2024-02-13T14:25:03.483 INFO:teuthology.orchestra.run.smithi148.stderr:tar: /var/lib/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/crash: Cannot open: No such file or directory 2024-02-13T14:25:03.483 INFO:teuthology.orchestra.run.smithi148.stderr:tar: Error is not recoverable: exiting now 2024-02-13T14:25:03.484 INFO:tasks.cephadm:Checking cluster log for badness... 2024-02-13T14:25:03.485 DEBUG:teuthology.orchestra.run.smithi087:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'but it is still running' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(PG_' | egrep -v '\(OSD_' | egrep -v '\(OBJECT_' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | head -n 1 2024-02-13T14:25:03.517 INFO:teuthology.orchestra.run.smithi087.stdout:2024-02-13T14:20:00.000190+0000 mon.a (mon.0) 1068 : cluster 3 [WRN] POOL_APP_NOT_ENABLED: 2 pool(s) do not have an application enabled 2024-02-13T14:25:03.517 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-02-13T14:25:03.517 DEBUG:teuthology.orchestra.run.smithi087:> sudo egrep '\[SEC\]' /var/log/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'but it is still running' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(PG_' | egrep -v '\(OSD_' | egrep -v '\(OBJECT_' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | head -n 1 2024-02-13T14:25:03.588 DEBUG:teuthology.orchestra.run.smithi087:> sudo egrep '\[ERR\]' /var/log/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'but it is still running' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(PG_' | egrep -v '\(OSD_' | egrep -v '\(OBJECT_' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | head -n 1 2024-02-13T14:25:03.660 DEBUG:teuthology.orchestra.run.smithi087:> sudo egrep '\[WRN\]' /var/log/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'but it is still running' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(PG_' | egrep -v '\(OSD_' | egrep -v '\(OBJECT_' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | head -n 1 2024-02-13T14:25:03.732 INFO:teuthology.orchestra.run.smithi087.stdout:2024-02-13T14:20:00.000190+0000 mon.a (mon.0) 1068 : cluster 3 [WRN] POOL_APP_NOT_ENABLED: 2 pool(s) do not have an application enabled 2024-02-13T14:25:03.732 INFO:tasks.cephadm:Compressing logs... 2024-02-13T14:25:03.733 DEBUG:teuthology.orchestra.run.smithi087:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-02-13T14:25:03.775 DEBUG:teuthology.orchestra.run.smithi148:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-02-13T14:25:03.814 INFO:teuthology.orchestra.run.smithi148.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-02-13T14:25:03.815 INFO:teuthology.orchestra.run.smithi148.stderr:: No such file or directory 2024-02-13T14:25:03.815 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/ceph-osd.6.log 2024-02-13T14:25:03.815 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/ceph.audit.log 2024-02-13T14:25:03.815 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/cephadm.log: /var/log/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/ceph-osd.4.log 2024-02-13T14:25:03.815 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/ceph-osd.7.log 2024-02-13T14:25:03.815 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/ceph-osd.5.log 2024-02-13T14:25:03.815 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/ceph-mgr.x.log 2024-02-13T14:25:03.815 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/tcmu-runner.log 2024-02-13T14:25:03.815 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/ceph.cephadm.log 2024-02-13T14:25:03.815 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/tcmu-runner.log: 68.2% -- replaced with /var/log/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/tcmu-runner.log.gz 2024-02-13T14:25:03.815 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/ceph.log 2024-02-13T14:25:03.815 INFO:teuthology.orchestra.run.smithi148.stderr: 90.2% -- replaced with /var/log/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/ceph.audit.log.gz 2024-02-13T14:25:03.816 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/ceph-volume.log 2024-02-13T14:25:03.816 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/ceph.cephadm.log: /var/log/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/ceph.log: 81.5% -- replaced with /var/log/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/ceph.cephadm.log.gz 2024-02-13T14:25:03.816 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/ceph-mon.b.log 2024-02-13T14:25:03.816 INFO:teuthology.orchestra.run.smithi148.stderr: 85.0% -- replaced with /var/log/ceph/cephadm.log.gz 2024-02-13T14:25:03.816 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/ceph-volume.log: 86.8% -- replaced with /var/log/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/ceph.log.gz 2024-02-13T14:25:03.816 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/ceph-mon.b.log: 91.3% -- replaced with /var/log/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/ceph-mgr.x.log.gz 2024-02-13T14:25:03.817 INFO:teuthology.orchestra.run.smithi087.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-02-13T14:25:03.817 INFO:teuthology.orchestra.run.smithi087.stderr:: No such file or directory 2024-02-13T14:25:03.817 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/ceph-mon.a.log 2024-02-13T14:25:03.817 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/ceph.audit.log 2024-02-13T14:25:03.817 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/ceph-client.rgw.foo.a.log 2024-02-13T14:25:03.817 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/ceph-osd.1.log 2024-02-13T14:25:03.817 INFO:teuthology.orchestra.run.smithi087.stderr:gzip/var/log/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/ceph-client.rgw.foo.a.log: -5 --verbose -- /var/log/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/ceph-osd.3.log 2024-02-13T14:25:03.817 INFO:teuthology.orchestra.run.smithi087.stderr: 57.8% -- replaced with /var/log/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/ceph-client.rgw.foo.a.log.gz 2024-02-13T14:25:03.817 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/ceph-mgr.y.log 2024-02-13T14:25:03.817 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/ceph-osd.0.log 2024-02-13T14:25:03.817 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/ceph.cephadm.log 2024-02-13T14:25:03.817 INFO:teuthology.orchestra.run.smithi087.stderr: 91.3% -- replaced with /var/log/ceph/cephadm.log.gz 2024-02-13T14:25:03.817 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/ceph.log 2024-02-13T14:25:03.818 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/ceph.cephadm.log: 88.9% -- replaced with /var/log/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/ceph.cephadm.log.gz 2024-02-13T14:25:03.818 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/ceph-osd.2.log 2024-02-13T14:25:03.818 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/ceph.log: 94.0% -- replaced with /var/log/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/ceph.audit.log.gz 2024-02-13T14:25:03.818 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/ceph-volume.log 2024-02-13T14:25:03.818 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/ceph-osd.2.log: 92.5% -- replaced with /var/log/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/ceph.log.gz 2024-02-13T14:25:03.818 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/ceph-mon.c.log 2024-02-13T14:25:03.818 INFO:teuthology.orchestra.run.smithi148.stderr: 91.1% -- replaced with /var/log/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/ceph-volume.log.gz 2024-02-13T14:25:03.821 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/ceph-volume.log: 91.3% -- replaced with /var/log/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/ceph-volume.log.gz 2024-02-13T14:25:04.006 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/ceph-mon.c.log: 88.9% -- replaced with /var/log/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/ceph-mgr.y.log.gz 2024-02-13T14:25:04.044 INFO:teuthology.orchestra.run.smithi148.stderr: 92.1% -- replaced with /var/log/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/ceph-mon.b.log.gz 2024-02-13T14:25:04.136 INFO:teuthology.orchestra.run.smithi087.stderr: 91.9% -- replaced with /var/log/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/ceph-mon.c.log.gz 2024-02-13T14:25:04.551 INFO:teuthology.orchestra.run.smithi087.stderr: 91.2% -- replaced with /var/log/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/ceph-mon.a.log.gz 2024-02-13T14:25:05.747 INFO:teuthology.orchestra.run.smithi148.stderr: 94.0% -- replaced with /var/log/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/ceph-osd.6.log.gz 2024-02-13T14:25:05.801 INFO:teuthology.orchestra.run.smithi148.stderr: 94.0% -- replaced with /var/log/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/ceph-osd.5.log.gz 2024-02-13T14:25:05.806 INFO:teuthology.orchestra.run.smithi087.stderr: 93.9% -- replaced with /var/log/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/ceph-osd.2.log.gz 2024-02-13T14:25:05.883 INFO:teuthology.orchestra.run.smithi148.stderr: 94.0% -- replaced with /var/log/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/ceph-osd.7.log.gz 2024-02-13T14:25:05.902 INFO:teuthology.orchestra.run.smithi148.stderr: 94.1% -- replaced with /var/log/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/ceph-osd.4.log.gz 2024-02-13T14:25:05.904 INFO:teuthology.orchestra.run.smithi148.stderr: 2024-02-13T14:25:05.904 INFO:teuthology.orchestra.run.smithi148.stderr:real 0m2.115s 2024-02-13T14:25:05.904 INFO:teuthology.orchestra.run.smithi148.stderr:user 0m8.076s 2024-02-13T14:25:05.904 INFO:teuthology.orchestra.run.smithi148.stderr:sys 0m0.329s 2024-02-13T14:25:06.203 INFO:teuthology.orchestra.run.smithi087.stderr: 94.2% -- replaced with /var/log/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/ceph-osd.3.log.gz 2024-02-13T14:25:06.287 INFO:teuthology.orchestra.run.smithi087.stderr: 94.1% -- replaced with /var/log/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/ceph-osd.0.log.gz 2024-02-13T14:25:06.325 INFO:teuthology.orchestra.run.smithi087.stderr: 94.0% -- replaced with /var/log/ceph/5c04b074-ca79-11ee-95b9-87774f69a715/ceph-osd.1.log.gz 2024-02-13T14:25:06.327 INFO:teuthology.orchestra.run.smithi087.stderr: 2024-02-13T14:25:06.327 INFO:teuthology.orchestra.run.smithi087.stderr:real 0m2.541s 2024-02-13T14:25:06.327 INFO:teuthology.orchestra.run.smithi087.stderr:user 0m10.358s 2024-02-13T14:25:06.327 INFO:teuthology.orchestra.run.smithi087.stderr:sys 0m0.355s 2024-02-13T14:25:06.327 INFO:tasks.cephadm:Archiving logs... 2024-02-13T14:25:06.327 DEBUG:teuthology.misc:Transferring archived files from smithi087:/var/log/ceph to /home/teuthworker/archive/phlogistonjohn-2024-02-13_11:41:43-orch:cephadm-wip-phlogistonjohn-testing-2024-02-12-1654-distro-default-smithi/7558012/remote/smithi087/log 2024-02-13T14:25:06.329 DEBUG:teuthology.orchestra.run.smithi087:> sudo tar c -f - -C /var/log/ceph -- . 2024-02-13T14:25:07.699 DEBUG:teuthology.misc:Transferring archived files from smithi148:/var/log/ceph to /home/teuthworker/archive/phlogistonjohn-2024-02-13_11:41:43-orch:cephadm-wip-phlogistonjohn-testing-2024-02-12-1654-distro-default-smithi/7558012/remote/smithi148/log 2024-02-13T14:25:07.700 DEBUG:teuthology.orchestra.run.smithi148:> sudo tar c -f - -C /var/log/ceph -- . 2024-02-13T14:25:08.924 INFO:tasks.cephadm:Removing cluster... 2024-02-13T14:25:08.924 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 5c04b074-ca79-11ee-95b9-87774f69a715 --force 2024-02-13T14:25:09.117 INFO:teuthology.orchestra.run.smithi087.stdout:Deleting cluster with fsid: 5c04b074-ca79-11ee-95b9-87774f69a715 2024-02-13T14:25:09.381 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 5c04b074-ca79-11ee-95b9-87774f69a715 --force 2024-02-13T14:25:09.574 INFO:teuthology.orchestra.run.smithi148.stdout:Deleting cluster with fsid: 5c04b074-ca79-11ee-95b9-87774f69a715 2024-02-13T14:25:09.837 INFO:tasks.cephadm:Removing cephadm ... 2024-02-13T14:25:09.838 DEBUG:teuthology.orchestra.run.smithi087:> rm -rf /home/ubuntu/cephtest/cephadm 2024-02-13T14:25:09.854 DEBUG:teuthology.orchestra.run.smithi148:> rm -rf /home/ubuntu/cephtest/cephadm 2024-02-13T14:25:09.868 INFO:tasks.cephadm:Teardown complete 2024-02-13T14:25:09.869 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-02-13T14:25:09.878 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-02-13T14:25:09.878 DEBUG:teuthology.orchestra.run.smithi087:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-02-13T14:25:09.896 DEBUG:teuthology.orchestra.run.smithi148:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-02-13T14:25:09.980 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-02-13T14:25:09.989 INFO:teuthology.task.clock:Checking final clock skew... 2024-02-13T14:25:09.989 DEBUG:teuthology.orchestra.run.smithi087:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-13T14:25:09.992 DEBUG:teuthology.orchestra.run.smithi148:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-13T14:25:10.004 INFO:teuthology.orchestra.run.smithi087.stderr:bash: line 1: ntpq: command not found 2024-02-13T14:25:10.006 INFO:teuthology.orchestra.run.smithi148.stderr:bash: line 1: ntpq: command not found 2024-02-13T14:25:10.008 INFO:teuthology.orchestra.run.smithi087.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-13T14:25:10.008 INFO:teuthology.orchestra.run.smithi087.stdout:=============================================================================== 2024-02-13T14:25:10.009 INFO:teuthology.orchestra.run.smithi087.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 3 +685us[ +685us] +/- 92ms 2024-02-13T14:25:10.009 INFO:teuthology.orchestra.run.smithi087.stdout:^* hv02.front.sepia.ceph.com 3 6 377 6 -886us[ -861us] +/- 75ms 2024-02-13T14:25:10.009 INFO:teuthology.orchestra.run.smithi087.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 5 +498us[ +498us] +/- 89ms 2024-02-13T14:25:10.009 INFO:teuthology.orchestra.run.smithi087.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-02-13T14:25:10.010 INFO:teuthology.orchestra.run.smithi148.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-13T14:25:10.010 INFO:teuthology.orchestra.run.smithi148.stdout:=============================================================================== 2024-02-13T14:25:10.010 INFO:teuthology.orchestra.run.smithi148.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 3 +677us[ +721us] +/- 92ms 2024-02-13T14:25:10.010 INFO:teuthology.orchestra.run.smithi148.stdout:^* hv02.front.sepia.ceph.com 3 6 377 3 -911us[ -867us] +/- 75ms 2024-02-13T14:25:10.010 INFO:teuthology.orchestra.run.smithi148.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 4 +482us[ +526us] +/- 89ms 2024-02-13T14:25:10.010 INFO:teuthology.orchestra.run.smithi148.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-02-13T14:25:10.010 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-02-13T14:25:10.018 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-02-13T14:25:10.019 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-02-13T14:25:10.069 DEBUG:teuthology.orchestra.run.smithi087:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-02-13T14:25:10.147 DEBUG:teuthology.orchestra.run.smithi148:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-02-13T14:25:10.220 DEBUG:teuthology.orchestra.run.smithi087:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-13T14:25:10.250 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-13T14:25:10.251 DEBUG:teuthology.orchestra.run.smithi148:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-13T14:25:10.281 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-13T14:25:10.281 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-02-13T14:25:10.290 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-02-13T14:25:10.319 INFO:teuthology.task.internal:Duration was 1360.960395 seconds 2024-02-13T14:25:10.319 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-02-13T14:25:10.370 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-02-13T14:25:10.371 DEBUG:teuthology.orchestra.run.smithi087:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-02-13T14:25:10.373 DEBUG:teuthology.orchestra.run.smithi148:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-02-13T14:25:10.414 INFO:teuthology.orchestra.run.smithi087.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-13T14:25:10.417 INFO:teuthology.orchestra.run.smithi148.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-13T14:25:10.775 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-02-13T14:25:10.775 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi087.front.sepia.ceph.com 2024-02-13T14:25:10.776 DEBUG:teuthology.orchestra.run.smithi087:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-02-13T14:25:10.835 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi148.front.sepia.ceph.com 2024-02-13T14:25:10.835 DEBUG:teuthology.orchestra.run.smithi148:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-02-13T14:25:10.864 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-02-13T14:25:10.864 DEBUG:teuthology.orchestra.run.smithi087:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-02-13T14:25:10.878 DEBUG:teuthology.orchestra.run.smithi148:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-02-13T14:25:10.932 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-02-13T14:25:10.932 DEBUG:teuthology.orchestra.run.smithi087:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-02-13T14:25:10.945 DEBUG:teuthology.orchestra.run.smithi148:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-02-13T14:25:11.207 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-02-13T14:25:11.217 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-02-13T14:25:11.217 DEBUG:teuthology.orchestra.run.smithi087:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-02-13T14:25:11.244 DEBUG:teuthology.orchestra.run.smithi148:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-02-13T14:25:11.272 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-02-13T14:25:11.282 DEBUG:teuthology.orchestra.run.smithi087:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-02-13T14:25:11.286 DEBUG:teuthology.orchestra.run.smithi148:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-02-13T14:25:11.309 INFO:teuthology.orchestra.run.smithi087.stdout:kernel.core_pattern = core 2024-02-13T14:25:11.338 INFO:teuthology.orchestra.run.smithi148.stdout:kernel.core_pattern = core 2024-02-13T14:25:11.364 DEBUG:teuthology.orchestra.run.smithi087:> test -e /home/ubuntu/cephtest/archive/coredump 2024-02-13T14:25:11.379 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-13T14:25:11.379 DEBUG:teuthology.orchestra.run.smithi148:> test -e /home/ubuntu/cephtest/archive/coredump 2024-02-13T14:25:11.418 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-13T14:25:11.419 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-02-13T14:25:11.431 INFO:teuthology.task.internal:Transferring archived files... 2024-02-13T14:25:11.432 DEBUG:teuthology.misc:Transferring archived files from smithi087:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/phlogistonjohn-2024-02-13_11:41:43-orch:cephadm-wip-phlogistonjohn-testing-2024-02-12-1654-distro-default-smithi/7558012/remote/smithi087 2024-02-13T14:25:11.433 DEBUG:teuthology.orchestra.run.smithi087:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-02-13T14:25:11.474 DEBUG:teuthology.misc:Transferring archived files from smithi148:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/phlogistonjohn-2024-02-13_11:41:43-orch:cephadm-wip-phlogistonjohn-testing-2024-02-12-1654-distro-default-smithi/7558012/remote/smithi148 2024-02-13T14:25:11.474 DEBUG:teuthology.orchestra.run.smithi148:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-02-13T14:25:11.512 INFO:teuthology.task.internal:Removing archive directory... 2024-02-13T14:25:11.512 DEBUG:teuthology.orchestra.run.smithi087:> rm -rf -- /home/ubuntu/cephtest/archive 2024-02-13T14:25:11.514 DEBUG:teuthology.orchestra.run.smithi148:> rm -rf -- /home/ubuntu/cephtest/archive 2024-02-13T14:25:11.560 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-02-13T14:25:11.572 INFO:teuthology.task.internal:Not uploading archives. 2024-02-13T14:25:11.572 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-02-13T14:25:11.604 INFO:teuthology.task.internal:Tidying up after the test... 2024-02-13T14:25:11.605 DEBUG:teuthology.orchestra.run.smithi087:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-02-13T14:25:11.607 DEBUG:teuthology.orchestra.run.smithi148:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-02-13T14:25:11.619 INFO:teuthology.orchestra.run.smithi087.stdout: 265946 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 13 14:25 /home/ubuntu/cephtest 2024-02-13T14:25:11.621 INFO:teuthology.orchestra.run.smithi148.stdout: 265968 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 13 14:25 /home/ubuntu/cephtest 2024-02-13T14:25:11.622 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-02-13T14:25:11.653 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-02-13T14:25:11.757 INFO:teuthology.nuke:Checking targets against current locks 2024-02-13T14:25:11.782 DEBUG:teuthology.nuke:shortname: smithi087 2024-02-13T14:25:11.783 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-13T14:25:11.810 DEBUG:teuthology.nuke:shortname: smithi148 2024-02-13T14:25:11.810 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-13T14:25:11.814 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi087.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/phlogistonjohn-2024-02-13_11:41:43-orch:cephadm-wip-phlogistonjohn-testing-2024-02-12-1654-distro-default-smithi/7558012', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-13 13:54:06.895163', 'locked_by': 'scheduled_phlogistonjohn@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPYH3T79/fS8yIgsiTb6IbwUfljLcOEV5eBwlRcb8CdsxB6pcYb0BFy2gnaz4NGKpP6BOg2HxnJBSzhFcCBHbi4='} 2024-02-13T14:25:11.840 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi148.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/phlogistonjohn-2024-02-13_11:41:43-orch:cephadm-wip-phlogistonjohn-testing-2024-02-12-1654-distro-default-smithi/7558012', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-13 13:54:06.893557', 'locked_by': 'scheduled_phlogistonjohn@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFfUCk8jh0XR5KaJqXZ5E9hxJyiF/VMF7OqStLQjgWjeVtIYNFd4mR7buTxN16zH/61k1o//+oFYzJFF/LOL394='} 2024-02-13T14:25:11.870 INFO:teuthology.orchestra.console.smithi087:Power off 2024-02-13T14:25:11.870 DEBUG:teuthology.orchestra.console.smithi087:pexpect command: ipmitool -H smithi087.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-13T14:25:11.896 DEBUG:teuthology.orchestra.console.smithi087:power off output: Chassis Power Control: Down/Off 2024-02-13T14:25:11.897 DEBUG:teuthology.orchestra.console.smithi087:pexpect command: ipmitool -H smithi087.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-13T14:25:11.911 INFO:teuthology.orchestra.console.smithi148:Power off 2024-02-13T14:25:11.912 DEBUG:teuthology.orchestra.console.smithi148:pexpect command: ipmitool -H smithi148.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-13T14:25:11.922 DEBUG:teuthology.orchestra.console.smithi087:check power output: Chassis Power is on 2024-02-13T14:25:11.934 DEBUG:teuthology.orchestra.console.smithi148:power off output: Chassis Power Control: Down/Off 2024-02-13T14:25:11.935 DEBUG:teuthology.orchestra.console.smithi148:pexpect command: ipmitool -H smithi148.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-13T14:25:11.964 DEBUG:teuthology.orchestra.console.smithi148:check power output: Chassis Power is on 2024-02-13T14:25:15.923 DEBUG:teuthology.orchestra.console.smithi087:pexpect command: ipmitool -H smithi087.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-13T14:25:15.965 DEBUG:teuthology.orchestra.console.smithi148:pexpect command: ipmitool -H smithi148.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-13T14:25:16.035 DEBUG:teuthology.orchestra.console.smithi087:check power output: Chassis Power is on 2024-02-13T14:25:16.075 DEBUG:teuthology.orchestra.console.smithi148:check power output: Chassis Power is on 2024-02-13T14:25:20.040 DEBUG:teuthology.orchestra.console.smithi087:pexpect command: ipmitool -H smithi087.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-13T14:25:20.076 DEBUG:teuthology.orchestra.console.smithi148:pexpect command: ipmitool -H smithi148.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-13T14:25:20.151 DEBUG:teuthology.orchestra.console.smithi087:check power output: Chassis Power is off 2024-02-13T14:25:20.186 DEBUG:teuthology.orchestra.console.smithi148:check power output: Chassis Power is on 2024-02-13T14:25:20.252 INFO:teuthology.orchestra.console.smithi087:Power off completed 2024-02-13T14:25:24.188 DEBUG:teuthology.orchestra.console.smithi148:pexpect command: ipmitool -H smithi148.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-13T14:25:24.297 DEBUG:teuthology.orchestra.console.smithi148:check power output: Chassis Power is off 2024-02-13T14:25:24.398 INFO:teuthology.orchestra.console.smithi148:Power off completed 2024-02-13T14:25:24.502 INFO:teuthology.run:Summary data: description: orch:cephadm/with-work/{0-distro/centos_9.stream_runc fixed-2 mode/root mon_election/connectivity msgr/async start tasks/rados_python} duration: 1360.9603946208954 failure_reason: '"2024-02-13T14:20:00.000190+0000 mon.a (mon.0) 1068 : cluster 3 [WRN] POOL_APP_NOT_ENABLED: 2 pool(s) do not have an application enabled" in cluster log' flavor: default owner: scheduled_phlogistonjohn@teuthology success: false 2024-02-13T14:25:24.502 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-13T14:25:24.674 INFO:teuthology.run:FAIL