2026-02-24T18:11:19.867 INFO:root:teuthology version: 1.2.4.dev11+ge6d9099ce 2026-02-24T18:11:19.883 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com 2026-02-24T18:11:19.961 INFO:teuthology.run:Config: archive_path: /home/teuthworker/mnt/teuthology/yuriw-2026-02-24_16:22:43-rados-wip-yuri6-testing-2026-02-22-1629-squid-distro-default-trial/68639 branch: wip-yuri6-testing-2026-02-22-1629-squid description: rados/valgrind-leaks/{1-start 2-inject-leak/none centos_latest} email: null first_in_suite: false flavor: default job_id: '68639' kernel: branch: distro kdb: true sha1: distro last_in_suite: false machine_type: trial name: yuriw-2026-02-24_16:22:43-rados-wip-yuri6-testing-2026-02-22-1629-squid-distro-default-trial no_nested_subset: false openstack: - volumes: count: 2 size: 10 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-yuri6-testing-2026-02-22-1629-squid ceph: conf: global: osd heartbeat grace: 40 osd max object name len: 460 osd max object namespace len: 64 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon osd crush smoke test: false osd: debug ms: 1 debug osd: 20 osd fast shutdown: false flavor: default log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(PG_ - \(POOL_APP_NOT_ENABLED\) - OSD bench result sha1: f90f4eaebb28b619cf76e80cbbe997656da69579 stop-daemons-timeout: 1200 valgrind: mon: - --tool=memcheck - --leak-check=full - --show-reachable=yes osd: - --tool=memcheck ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: debuginfo: true flavor: default sha1: f90f4eaebb28b619cf76e80cbbe997656da69579 selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 workunit: branch: wip-yuri6-testing-2026-02-22-1629-squid sha1: f90f4eaebb28b619cf76e80cbbe997656da69579 owner: scheduled_yuriw@soko04.front.sepia.ceph.com priority: 69 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mon.b - mon.c - mgr.x - mgr.y - osd.0 - osd.1 - osd.2 - client.0 seed: 5907 sha1: f90f4eaebb28b619cf76e80cbbe997656da69579 sleep_before_teardown: 0 subset: 111/120000 suite: rados suite_branch: wip-yuri6-testing-2026-02-22-1629-squid suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_f90f4eaebb28b619cf76e80cbbe997656da69579/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: f90f4eaebb28b619cf76e80cbbe997656da69579 targets: trial038.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFs3Ils7PR9Tm15uNdKJ0W5F2g9JQYSKaSM2xRd3l2l9AKo9OmyoT62jWNp07urXvPzP8D96jrJIk/LBhfOG9Ug= tasks: - install: null - ceph: null teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: reimage-ssh-down teuthology_sha1: e6d9099ce03dc21193fda653159ec82465c3a94a timestamp: 2026-02-24_16:22:43 tube: trial user: yuriw verbose: true worker_log: /home/teuthworker/mnt/teuthology/worker_logs/dispatcher.trial.904699 2026-02-24T18:11:19.961 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_f90f4eaebb28b619cf76e80cbbe997656da69579/qa; will attempt to use it 2026-02-24T18:11:19.962 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_f90f4eaebb28b619cf76e80cbbe997656da69579/qa/tasks 2026-02-24T18:11:19.962 INFO:teuthology.run_tasks:Running task internal.check_packages... 2026-02-24T18:11:19.963 INFO:teuthology.task.internal:Checking packages... 2026-02-24T18:11:19.963 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'f90f4eaebb28b619cf76e80cbbe997656da69579' 2026-02-24T18:11:19.963 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2026-02-24T18:11:19.963 INFO:teuthology.packaging:ref: None 2026-02-24T18:11:19.963 INFO:teuthology.packaging:tag: None 2026-02-24T18:11:19.963 INFO:teuthology.packaging:branch: wip-yuri6-testing-2026-02-22-1629-squid 2026-02-24T18:11:19.963 INFO:teuthology.packaging:sha1: f90f4eaebb28b619cf76e80cbbe997656da69579 2026-02-24T18:11:19.963 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-yuri6-testing-2026-02-22-1629-squid 2026-02-24T18:11:20.081 INFO:teuthology.task.internal:Found packages for ceph version 19.2.3-689.gf90f4eae 2026-02-24T18:11:20.082 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2026-02-24T18:11:20.094 INFO:teuthology.task.internal:no buildpackages task found 2026-02-24T18:11:20.094 INFO:teuthology.run_tasks:Running task internal.save_config... 2026-02-24T18:11:20.109 INFO:teuthology.task.internal:Saving configuration 2026-02-24T18:11:20.120 INFO:teuthology.run_tasks:Running task internal.check_lock... 2026-02-24T18:11:20.132 INFO:teuthology.task.internal.check_lock:Checking locks... 2026-02-24T18:11:20.209 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'trial038.front.sepia.ceph.com', 'description': '/home/teuthworker/mnt/teuthology/yuriw-2026-02-24_16:22:43-rados-wip-yuri6-testing-2026-02-22-1629-squid-distro-default-trial/68639', 'up': True, 'machine_type': 'trial', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2026-02-24 18:07:25.949759', 'locked_by': 'scheduled_yuriw@soko04.front.sepia.ceph.com', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFs3Ils7PR9Tm15uNdKJ0W5F2g9JQYSKaSM2xRd3l2l9AKo9OmyoT62jWNp07urXvPzP8D96jrJIk/LBhfOG9Ug='} 2026-02-24T18:11:20.209 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2026-02-24T18:11:20.220 INFO:teuthology.task.internal:roles: ubuntu@trial038.front.sepia.ceph.com - ['mon.a', 'mon.b', 'mon.c', 'mgr.x', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'client.0'] 2026-02-24T18:11:20.220 INFO:teuthology.run_tasks:Running task console_log... 2026-02-24T18:11:20.307 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f4e393e7880>, signals=[15]) 2026-02-24T18:11:20.307 INFO:teuthology.run_tasks:Running task internal.connect... 2026-02-24T18:11:20.319 INFO:teuthology.task.internal:Opening connections... 2026-02-24T18:11:20.319 DEBUG:teuthology.task.internal:connecting to ubuntu@trial038.front.sepia.ceph.com 2026-02-24T18:11:20.319 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial038.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-24T18:11:20.383 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2026-02-24T18:11:20.394 DEBUG:teuthology.orchestra.run.trial038:> uname -m 2026-02-24T18:11:20.456 INFO:teuthology.orchestra.run.trial038.stdout:x86_64 2026-02-24T18:11:20.456 DEBUG:teuthology.orchestra.run.trial038:> cat /etc/os-release 2026-02-24T18:11:20.512 INFO:teuthology.orchestra.run.trial038.stdout:NAME="CentOS Stream" 2026-02-24T18:11:20.512 INFO:teuthology.orchestra.run.trial038.stdout:VERSION="9" 2026-02-24T18:11:20.512 INFO:teuthology.orchestra.run.trial038.stdout:ID="centos" 2026-02-24T18:11:20.512 INFO:teuthology.orchestra.run.trial038.stdout:ID_LIKE="rhel fedora" 2026-02-24T18:11:20.512 INFO:teuthology.orchestra.run.trial038.stdout:VERSION_ID="9" 2026-02-24T18:11:20.512 INFO:teuthology.orchestra.run.trial038.stdout:PLATFORM_ID="platform:el9" 2026-02-24T18:11:20.512 INFO:teuthology.orchestra.run.trial038.stdout:PRETTY_NAME="CentOS Stream 9" 2026-02-24T18:11:20.512 INFO:teuthology.orchestra.run.trial038.stdout:ANSI_COLOR="0;31" 2026-02-24T18:11:20.512 INFO:teuthology.orchestra.run.trial038.stdout:LOGO="fedora-logo-icon" 2026-02-24T18:11:20.512 INFO:teuthology.orchestra.run.trial038.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2026-02-24T18:11:20.512 INFO:teuthology.orchestra.run.trial038.stdout:HOME_URL="https://centos.org/" 2026-02-24T18:11:20.512 INFO:teuthology.orchestra.run.trial038.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2026-02-24T18:11:20.512 INFO:teuthology.orchestra.run.trial038.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2026-02-24T18:11:20.512 INFO:teuthology.orchestra.run.trial038.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2026-02-24T18:11:20.513 INFO:teuthology.lock.ops:Updating trial038.front.sepia.ceph.com on lock server 2026-02-24T18:11:20.571 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2026-02-24T18:11:20.583 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2026-02-24T18:11:20.596 INFO:teuthology.task.internal:Checking for old test directory... 2026-02-24T18:11:20.596 DEBUG:teuthology.orchestra.run.trial038:> test '!' -e /home/ubuntu/cephtest 2026-02-24T18:11:20.611 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2026-02-24T18:11:20.622 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2026-02-24T18:11:20.622 DEBUG:teuthology.orchestra.run.trial038:> test -z $(ls -A /var/lib/ceph) 2026-02-24T18:11:20.666 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2026-02-24T18:11:20.760 INFO:teuthology.run_tasks:Running task kernel... 2026-02-24T18:11:20.781 INFO:teuthology.task.kernel:normalize config orig: {'branch': 'distro', 'kdb': True, 'sha1': 'distro'} 2026-02-24T18:11:20.781 INFO:teuthology.task.kernel:config {'mon.a': {'branch': 'distro', 'kdb': True, 'sha1': 'distro'}}, timeout 300 2026-02-24T18:11:20.781 DEBUG:teuthology.orchestra.run.trial038:> test -f /run/.containerenv -o -f /.dockerenv 2026-02-24T18:11:20.796 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T18:11:20.796 DEBUG:teuthology.orchestra.run.trial038:> uname -r 2026-02-24T18:11:20.852 INFO:teuthology.orchestra.run.trial038.stdout:5.14.0-681.el9.x86_64 2026-02-24T18:11:20.852 INFO:teuthology.task.kernel:Running kernel on trial038: 5.14.0-681.el9.x86_64 2026-02-24T18:11:20.852 DEBUG:teuthology.orchestra.run.trial038:> sudo yum install -y kernel 2026-02-24T18:11:29.809 INFO:teuthology.orchestra.run.trial038.stdout:CentOS Stream 9 - BaseOS 1.0 MB/s | 8.9 MB 00:08 2026-02-24T18:11:32.363 INFO:teuthology.orchestra.run.trial038.stdout:CentOS Stream 9 - AppStream 13 MB/s | 27 MB 00:02 2026-02-24T18:11:36.026 INFO:teuthology.orchestra.run.trial038.stdout:CentOS Stream 9 - CRB 6.1 MB/s | 8.0 MB 00:01 2026-02-24T18:11:36.988 INFO:teuthology.orchestra.run.trial038.stdout:CentOS Stream 9 - Extras packages 92 kB/s | 20 kB 00:00 2026-02-24T18:11:37.417 INFO:teuthology.orchestra.run.trial038.stdout:Extra Packages for Enterprise Linux 56 MB/s | 20 MB 00:00 2026-02-24T18:11:40.645 INFO:teuthology.orchestra.run.trial038.stdout:lab-extras 1.4 MB/s | 50 kB 00:00 2026-02-24T18:11:41.665 INFO:teuthology.orchestra.run.trial038.stdout:Package kernel-5.14.0-645.el9.x86_64 is already installed. 2026-02-24T18:11:41.665 INFO:teuthology.orchestra.run.trial038.stdout:Package kernel-5.14.0-681.el9.x86_64 is already installed. 2026-02-24T18:11:41.685 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-24T18:11:41.686 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-24T18:11:41.686 INFO:teuthology.orchestra.run.trial038.stdout: Package Architecture Version Repository Size 2026-02-24T18:11:41.686 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-24T18:11:41.686 INFO:teuthology.orchestra.run.trial038.stdout:Installing: 2026-02-24T18:11:41.686 INFO:teuthology.orchestra.run.trial038.stdout: kernel x86_64 5.14.0-686.el9 baseos 901 k 2026-02-24T18:11:41.686 INFO:teuthology.orchestra.run.trial038.stdout:Installing dependencies: 2026-02-24T18:11:41.687 INFO:teuthology.orchestra.run.trial038.stdout: kernel-core x86_64 5.14.0-686.el9 baseos 17 M 2026-02-24T18:11:41.687 INFO:teuthology.orchestra.run.trial038.stdout: kernel-modules x86_64 5.14.0-686.el9 baseos 40 M 2026-02-24T18:11:41.687 INFO:teuthology.orchestra.run.trial038.stdout: kernel-modules-core x86_64 5.14.0-686.el9 baseos 31 M 2026-02-24T18:11:41.687 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:11:41.687 INFO:teuthology.orchestra.run.trial038.stdout:Transaction Summary 2026-02-24T18:11:41.687 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-24T18:11:41.687 INFO:teuthology.orchestra.run.trial038.stdout:Install 4 Packages 2026-02-24T18:11:41.687 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:11:41.687 INFO:teuthology.orchestra.run.trial038.stdout:Total download size: 89 M 2026-02-24T18:11:41.687 INFO:teuthology.orchestra.run.trial038.stdout:Installed size: 135 M 2026-02-24T18:11:41.688 INFO:teuthology.orchestra.run.trial038.stdout:Downloading Packages: 2026-02-24T18:11:43.278 INFO:teuthology.orchestra.run.trial038.stdout:(1/4): kernel-5.14.0-686.el9.x86_64.rpm 808 kB/s | 901 kB 00:01 2026-02-24T18:11:43.917 INFO:teuthology.orchestra.run.trial038.stdout:(2/4): kernel-core-5.14.0-686.el9.x86_64.rpm 9.8 MB/s | 17 MB 00:01 2026-02-24T18:11:44.421 INFO:teuthology.orchestra.run.trial038.stdout:(3/4): kernel-modules-core-5.14.0-686.el9.x86_6 27 MB/s | 31 MB 00:01 2026-02-24T18:11:44.635 INFO:teuthology.orchestra.run.trial038.stdout:(4/4): kernel-modules-5.14.0-686.el9.x86_64.rpm 16 MB/s | 40 MB 00:02 2026-02-24T18:11:44.637 INFO:teuthology.orchestra.run.trial038.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:11:44.637 INFO:teuthology.orchestra.run.trial038.stdout:Total 30 MB/s | 89 MB 00:02 2026-02-24T18:11:44.829 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction check 2026-02-24T18:11:45.038 INFO:teuthology.orchestra.run.trial038.stdout:Transaction check succeeded. 2026-02-24T18:11:45.038 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction test 2026-02-24T18:11:45.420 INFO:teuthology.orchestra.run.trial038.stdout:Transaction test succeeded. 2026-02-24T18:11:45.420 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction 2026-02-24T18:11:46.539 INFO:teuthology.orchestra.run.trial038.stdout: Preparing : 1/1 2026-02-24T18:11:46.832 INFO:teuthology.orchestra.run.trial038.stdout: Installing : kernel-modules-core-5.14.0-686.el9.x86_64 1/4 2026-02-24T18:11:46.854 INFO:teuthology.orchestra.run.trial038.stdout: Installing : kernel-core-5.14.0-686.el9.x86_64 2/4 2026-02-24T18:11:49.418 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: kernel-core-5.14.0-686.el9.x86_64 2/4 2026-02-24T18:11:49.451 INFO:teuthology.orchestra.run.trial038.stdout: Installing : kernel-modules-5.14.0-686.el9.x86_64 3/4 2026-02-24T18:11:52.558 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: kernel-modules-5.14.0-686.el9.x86_64 3/4 2026-02-24T18:11:52.564 INFO:teuthology.orchestra.run.trial038.stdout: Installing : kernel-5.14.0-686.el9.x86_64 4/4 2026-02-24T18:11:55.675 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: kernel-modules-core-5.14.0-686.el9.x86_64 4/4 2026-02-24T18:12:13.110 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: kernel-core-5.14.0-686.el9.x86_64 4/4 2026-02-24T18:12:13.132 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: kernel-modules-5.14.0-686.el9.x86_64 4/4 2026-02-24T18:12:13.317 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: kernel-5.14.0-686.el9.x86_64 4/4 2026-02-24T18:12:13.317 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : kernel-5.14.0-686.el9.x86_64 1/4 2026-02-24T18:12:13.317 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : kernel-core-5.14.0-686.el9.x86_64 2/4 2026-02-24T18:12:13.318 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : kernel-modules-5.14.0-686.el9.x86_64 3/4 2026-02-24T18:12:13.375 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : kernel-modules-core-5.14.0-686.el9.x86_64 4/4 2026-02-24T18:12:13.375 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:12:13.375 INFO:teuthology.orchestra.run.trial038.stdout:Installed: 2026-02-24T18:12:13.375 INFO:teuthology.orchestra.run.trial038.stdout: kernel-5.14.0-686.el9.x86_64 2026-02-24T18:12:13.375 INFO:teuthology.orchestra.run.trial038.stdout: kernel-core-5.14.0-686.el9.x86_64 2026-02-24T18:12:13.375 INFO:teuthology.orchestra.run.trial038.stdout: kernel-modules-5.14.0-686.el9.x86_64 2026-02-24T18:12:13.376 INFO:teuthology.orchestra.run.trial038.stdout: kernel-modules-core-5.14.0-686.el9.x86_64 2026-02-24T18:12:13.376 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:12:13.376 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-24T18:12:13.437 DEBUG:teuthology.orchestra.run.trial038:> rpm -q kernel | sort -rV | head -n 1 2026-02-24T18:12:13.465 INFO:teuthology.orchestra.run.trial038.stdout:kernel-5.14.0-686.el9.x86_64 2026-02-24T18:12:13.466 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-686.el9.x86_64 2026-02-24T18:12:13.467 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-681.el9.x86_64 Expected: 5.14.0-686.el9.x86_64 2026-02-24T18:12:13.467 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2026-02-24T18:12:13.467 DEBUG:teuthology.task.kernel:src is distro, skipping download 2026-02-24T18:12:13.467 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2026-02-24T18:12:13.467 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@trial038.front.sepia.ceph.com, path=None, version=distro) 2026-02-24T18:12:13.467 DEBUG:teuthology.orchestra.run.trial038:> rpm -q kernel | sort -rV | head -n 1 2026-02-24T18:12:13.533 INFO:teuthology.orchestra.run.trial038.stdout:kernel-5.14.0-686.el9.x86_64 2026-02-24T18:12:13.534 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-686.el9.x86_64 2026-02-24T18:12:13.534 DEBUG:teuthology.orchestra.run.trial038:> sudo rpm -qi grub2-tools 2026-02-24T18:12:13.606 INFO:teuthology.orchestra.run.trial038.stdout:Name : grub2-tools 2026-02-24T18:12:13.606 INFO:teuthology.orchestra.run.trial038.stdout:Epoch : 1 2026-02-24T18:12:13.606 INFO:teuthology.orchestra.run.trial038.stdout:Version : 2.06 2026-02-24T18:12:13.606 INFO:teuthology.orchestra.run.trial038.stdout:Release : 120.el9 2026-02-24T18:12:13.606 INFO:teuthology.orchestra.run.trial038.stdout:Architecture: x86_64 2026-02-24T18:12:13.607 INFO:teuthology.orchestra.run.trial038.stdout:Install Date: Wed 18 Feb 2026 05:18:42 PM UTC 2026-02-24T18:12:13.607 INFO:teuthology.orchestra.run.trial038.stdout:Group : Unspecified 2026-02-24T18:12:13.607 INFO:teuthology.orchestra.run.trial038.stdout:Size : 9108611 2026-02-24T18:12:13.607 INFO:teuthology.orchestra.run.trial038.stdout:License : GPLv3+ 2026-02-24T18:12:13.607 INFO:teuthology.orchestra.run.trial038.stdout:Signature : RSA/SHA256, Thu 04 Dec 2025 09:42:59 AM UTC, Key ID 05b555b38483c65d 2026-02-24T18:12:13.607 INFO:teuthology.orchestra.run.trial038.stdout:Source RPM : grub2-2.06-120.el9.src.rpm 2026-02-24T18:12:13.607 INFO:teuthology.orchestra.run.trial038.stdout:Build Date : Wed 03 Dec 2025 12:10:38 PM UTC 2026-02-24T18:12:13.607 INFO:teuthology.orchestra.run.trial038.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2026-02-24T18:12:13.607 INFO:teuthology.orchestra.run.trial038.stdout:Packager : builder@centos.org 2026-02-24T18:12:13.607 INFO:teuthology.orchestra.run.trial038.stdout:Vendor : CentOS 2026-02-24T18:12:13.607 INFO:teuthology.orchestra.run.trial038.stdout:URL : http://www.gnu.org/software/grub/ 2026-02-24T18:12:13.608 INFO:teuthology.orchestra.run.trial038.stdout:Summary : Support tools for GRUB. 2026-02-24T18:12:13.608 INFO:teuthology.orchestra.run.trial038.stdout:Description : 2026-02-24T18:12:13.608 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:12:13.608 INFO:teuthology.orchestra.run.trial038.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2026-02-24T18:12:13.608 INFO:teuthology.orchestra.run.trial038.stdout:customizable bootloader with modular architecture. It supports a rich 2026-02-24T18:12:13.608 INFO:teuthology.orchestra.run.trial038.stdout:variety of kernel formats, file systems, computer architectures and 2026-02-24T18:12:13.608 INFO:teuthology.orchestra.run.trial038.stdout:hardware devices. 2026-02-24T18:12:13.608 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:12:13.608 INFO:teuthology.orchestra.run.trial038.stdout:This subpackage provides tools for support of all platforms. 2026-02-24T18:12:13.609 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2026-02-24T18:12:13.610 INFO:teuthology.task.kernel:Updating grub on trial038 to boot 5.14.0-686.el9.x86_64 2026-02-24T18:12:13.610 DEBUG:teuthology.orchestra.run.trial038:> sudo find /boot/loader/entries -maxdepth 1 -name "*.conf" -type f -print -quit 2026-02-24T18:12:13.676 INFO:teuthology.orchestra.run.trial038.stdout:/boot/loader/entries/b0373be789df49cdafedf423703d2566-5.14.0-681.el9.x86_64.conf 2026-02-24T18:12:13.677 DEBUG:teuthology.orchestra.run.trial038:> sudo command -v grubby && echo yes || echo no 2026-02-24T18:12:13.743 INFO:teuthology.orchestra.run.trial038.stdout:/sbin/grubby 2026-02-24T18:12:13.744 INFO:teuthology.orchestra.run.trial038.stdout:yes 2026-02-24T18:12:13.745 DEBUG:teuthology.orchestra.run.trial038:> sudo find /boot/loader/entries -maxdepth 1 -name "*.conf" -type f | grep -F "5.14.0-686.el9.x86_64" | head -n 1 2026-02-24T18:12:13.809 INFO:teuthology.orchestra.run.trial038.stdout:/boot/loader/entries/b0373be789df49cdafedf423703d2566-5.14.0-686.el9.x86_64.conf 2026-02-24T18:12:13.810 DEBUG:teuthology.orchestra.run.trial038:> sudo grub2-set-default b0373be789df49cdafedf423703d2566-5.14.0-686.el9.x86_64 2026-02-24T18:12:13.883 DEBUG:teuthology.orchestra.run.trial038:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2026-02-24T18:12:14.090 INFO:teuthology.orchestra.run.trial038.stderr:Generating grub configuration file ... 2026-02-24T18:12:14.157 INFO:teuthology.orchestra.run.trial038.stderr:Found linux image: /boot/vmlinuz-5.14.0-686.el9.x86_64 2026-02-24T18:12:14.164 INFO:teuthology.orchestra.run.trial038.stderr:Found initrd image: /boot/initramfs-5.14.0-686.el9.x86_64.img 2026-02-24T18:12:14.414 INFO:teuthology.orchestra.run.trial038.stderr:Found linux image: /boot/vmlinuz-5.14.0-681.el9.x86_64 2026-02-24T18:12:14.421 INFO:teuthology.orchestra.run.trial038.stderr:Found initrd image: /boot/initramfs-5.14.0-681.el9.x86_64.img 2026-02-24T18:12:14.455 INFO:teuthology.orchestra.run.trial038.stderr:Found linux image: /boot/vmlinuz-5.14.0-645.el9.x86_64 2026-02-24T18:12:14.462 INFO:teuthology.orchestra.run.trial038.stderr:Found initrd image: /boot/initramfs-5.14.0-645.el9.x86_64.img 2026-02-24T18:12:14.493 INFO:teuthology.orchestra.run.trial038.stderr:Found linux image: /boot/vmlinuz-0-rescue-b0373be789df49cdafedf423703d2566 2026-02-24T18:12:14.500 INFO:teuthology.orchestra.run.trial038.stderr:Found initrd image: /boot/initramfs-0-rescue-b0373be789df49cdafedf423703d2566.img 2026-02-24T18:12:14.523 INFO:teuthology.orchestra.run.trial038.stderr:Found linux image: /boot/vmlinuz-0-rescue-2734456005d743dca7f3dc77c35a2d0d 2026-02-24T18:12:14.529 INFO:teuthology.orchestra.run.trial038.stderr:Found initrd image: /boot/initramfs-0-rescue-2734456005d743dca7f3dc77c35a2d0d.img 2026-02-24T18:12:14.618 INFO:teuthology.orchestra.run.trial038.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2026-02-24T18:12:14.626 INFO:teuthology.orchestra.run.trial038.stderr:done 2026-02-24T18:12:14.628 DEBUG:teuthology.orchestra.run.trial038:> test -d /sys/firmware/efi 2026-02-24T18:12:14.644 DEBUG:teuthology.orchestra.run.trial038:> sudo find /boot/efi/EFI -mindepth 1 -maxdepth 1 -type d || true 2026-02-24T18:12:14.709 INFO:teuthology.orchestra.run.trial038.stdout:/boot/efi/EFI/BOOT 2026-02-24T18:12:14.709 INFO:teuthology.orchestra.run.trial038.stdout:/boot/efi/EFI/centos 2026-02-24T18:12:14.710 DEBUG:teuthology.orchestra.run.trial038:> sudo test -f /boot/efi/EFI/BOOT/grub.cfg && echo yes || echo no 2026-02-24T18:12:14.777 INFO:teuthology.orchestra.run.trial038.stdout:no 2026-02-24T18:12:14.778 DEBUG:teuthology.orchestra.run.trial038:> sudo test -f /boot/efi/EFI/centos/grub.cfg && echo yes || echo no 2026-02-24T18:12:14.842 INFO:teuthology.orchestra.run.trial038.stdout:yes 2026-02-24T18:12:14.842 DEBUG:teuthology.orchestra.run.trial038:> sudo cp -f /boot/grub2/grub.cfg /boot/efi/EFI/centos/grub.cfg 2026-02-24T18:12:14.906 INFO:teuthology.orchestra.remote:Triggering safe hard reboot via SysRq on trial038 2026-02-24T18:12:14.906 DEBUG:teuthology.orchestra.run.trial038:> sudo bash -c ' 2026-02-24T18:12:14.906 DEBUG:teuthology.orchestra.run.trial038:> echo pci > /sys/kernel/reboot/type && echo 1 > /sys/kernel/reboot/force && sync && echo s > /proc/sysrq-trigger && echo u > /proc/sysrq-trigger && sleep 2 && echo b > /proc/sysrq-trigger 2026-02-24T18:12:14.906 DEBUG:teuthology.orchestra.run.trial038:> ' 2026-02-24T18:12:44.949 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2026-02-24T18:12:44.950 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial038.front.sepia.ceph.com' 2026-02-24T18:12:44.950 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial038.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-24T18:13:45.011 DEBUG:teuthology.orchestra.remote:timed out 2026-02-24T18:13:54.019 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial038.front.sepia.ceph.com' 2026-02-24T18:13:54.020 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial038.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-24T18:13:54.164 DEBUG:teuthology.orchestra.run.trial038:> true 2026-02-24T18:13:54.320 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial038.front.sepia.ceph.com' 2026-02-24T18:13:54.321 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-686.el9.x86_64"... 2026-02-24T18:13:54.321 DEBUG:teuthology.orchestra.run.trial038:> uname -r 2026-02-24T18:13:54.378 INFO:teuthology.orchestra.run.trial038.stdout:5.14.0-686.el9.x86_64 2026-02-24T18:13:54.378 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-686.el9.x86_64 vs 5.14.0-686.el9.x86_64 2026-02-24T18:13:54.378 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2026-02-24T18:13:54.379 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2026-02-24T18:13:55.380 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2026-02-24T18:13:55.380 DEBUG:teuthology.orchestra.run.trial038:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2026-02-24T18:13:55.411 INFO:teuthology.orchestra.run.trial038.stdout:ttyS1 2026-02-24T18:13:55.438 DEBUG:teuthology.parallel:result is None 2026-02-24T18:13:55.438 INFO:teuthology.run_tasks:Running task internal.base... 2026-02-24T18:13:55.450 INFO:teuthology.task.internal:Creating test directory... 2026-02-24T18:13:55.450 DEBUG:teuthology.orchestra.run.trial038:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2026-02-24T18:13:55.495 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2026-02-24T18:13:55.506 INFO:teuthology.run_tasks:Running task internal.archive... 2026-02-24T18:13:55.520 INFO:teuthology.task.internal:Creating archive directory... 2026-02-24T18:13:55.520 DEBUG:teuthology.orchestra.run.trial038:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2026-02-24T18:13:55.558 INFO:teuthology.run_tasks:Running task internal.coredump... 2026-02-24T18:13:55.569 INFO:teuthology.task.internal:Enabling coredump saving... 2026-02-24T18:13:55.569 DEBUG:teuthology.orchestra.run.trial038:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2026-02-24T18:13:55.616 INFO:teuthology.orchestra.run.trial038.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T18:13:55.625 INFO:teuthology.orchestra.run.trial038.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-24T18:13:55.627 INFO:teuthology.run_tasks:Running task internal.sudo... 2026-02-24T18:13:55.639 INFO:teuthology.task.internal:Configuring sudo... 2026-02-24T18:13:55.639 DEBUG:teuthology.orchestra.run.trial038:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2026-02-24T18:13:55.691 INFO:teuthology.run_tasks:Running task internal.syslog... 2026-02-24T18:13:55.707 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2026-02-24T18:13:55.708 DEBUG:teuthology.orchestra.run.trial038:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2026-02-24T18:13:55.747 DEBUG:teuthology.orchestra.run.trial038:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-24T18:13:55.919 DEBUG:teuthology.orchestra.run.trial038:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-24T18:13:55.944 DEBUG:teuthology.orchestra.run.trial038:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-24T18:13:56.093 DEBUG:teuthology.orchestra.run.trial038:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-24T18:13:56.118 DEBUG:teuthology.orchestra.run.trial038:> set -ex 2026-02-24T18:13:56.118 DEBUG:teuthology.orchestra.run.trial038:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2026-02-24T18:13:56.182 DEBUG:teuthology.orchestra.run.trial038:> sudo service rsyslog restart 2026-02-24T18:13:56.248 INFO:teuthology.orchestra.run.trial038.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-24T18:13:56.767 INFO:teuthology.run_tasks:Running task internal.timer... 2026-02-24T18:13:56.781 INFO:teuthology.task.internal:Starting timer... 2026-02-24T18:13:56.781 INFO:teuthology.run_tasks:Running task pcp... 2026-02-24T18:13:56.802 INFO:teuthology.run_tasks:Running task selinux... 2026-02-24T18:13:56.820 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2026-02-24T18:13:56.820 DEBUG:teuthology.orchestra.run.trial038:> sudo service auditd rotate 2026-02-24T18:13:56.849 INFO:teuthology.orchestra.run.trial038.stdout:Rotating logs: 2026-02-24T18:13:56.850 DEBUG:teuthology.task.selinux:Getting current SELinux state 2026-02-24T18:13:56.850 DEBUG:teuthology.orchestra.run.trial038:> /usr/sbin/getenforce 2026-02-24T18:13:56.907 INFO:teuthology.orchestra.run.trial038.stdout:Permissive 2026-02-24T18:13:56.907 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@trial038.front.sepia.ceph.com': 'permissive'} 2026-02-24T18:13:56.908 DEBUG:teuthology.orchestra.run.trial038:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2026-02-24T18:13:56.972 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T18:13:56.972 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2026-02-24T18:13:56.972 DEBUG:teuthology.orchestra.run.trial038:> sudo /usr/sbin/setenforce permissive 2026-02-24T18:13:57.038 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2026-02-24T18:13:57.055 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2026-02-24T18:13:57.073 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2026-02-24T18:13:57.075 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit trial038.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2026-02-24T18:16:54.925 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@trial038.front.sepia.ceph.com')] 2026-02-24T18:16:54.927 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial038.front.sepia.ceph.com' 2026-02-24T18:16:54.928 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial038.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-24T18:16:55.043 DEBUG:teuthology.orchestra.run.trial038:> true 2026-02-24T18:16:55.120 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial038.front.sepia.ceph.com' 2026-02-24T18:16:55.120 INFO:teuthology.run_tasks:Running task clock... 2026-02-24T18:16:55.134 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2026-02-24T18:16:55.134 INFO:teuthology.orchestra.run:Running command with timeout 360 2026-02-24T18:16:55.135 DEBUG:teuthology.orchestra.run.trial038:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-24T18:16:55.187 INFO:teuthology.orchestra.run.trial038.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2026-02-24T18:16:55.199 INFO:teuthology.orchestra.run.trial038.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2026-02-24T18:16:55.220 INFO:teuthology.orchestra.run.trial038.stderr:sudo: ntpd: command not found 2026-02-24T18:16:55.230 INFO:teuthology.orchestra.run.trial038.stdout:506 Cannot talk to daemon 2026-02-24T18:16:55.241 INFO:teuthology.orchestra.run.trial038.stderr:Failed to start ntp.service: Unit ntp.service not found. 2026-02-24T18:16:55.254 INFO:teuthology.orchestra.run.trial038.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2026-02-24T18:16:55.298 INFO:teuthology.orchestra.run.trial038.stderr:bash: line 1: ntpq: command not found 2026-02-24T18:16:55.300 INFO:teuthology.orchestra.run.trial038.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-24T18:16:55.300 INFO:teuthology.orchestra.run.trial038.stdout:=============================================================================== 2026-02-24T18:16:55.301 INFO:teuthology.run_tasks:Running task install... 2026-02-24T18:16:55.352 DEBUG:teuthology.task.install:project ceph 2026-02-24T18:16:55.352 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'debuginfo': True, 'flavor': 'default', 'sha1': 'f90f4eaebb28b619cf76e80cbbe997656da69579'}} 2026-02-24T18:16:55.352 DEBUG:teuthology.task.install:config {'debuginfo': True, 'flavor': 'default', 'sha1': 'f90f4eaebb28b619cf76e80cbbe997656da69579'} 2026-02-24T18:16:55.352 INFO:teuthology.task.install:Using flavor: default 2026-02-24T18:16:55.355 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse', 'ceph-common-dbg', 'ceph-fuse-dbg', 'ceph-mds-dbg', 'ceph-mgr-dbg', 'ceph-mon-dbg', 'ceph-osd-dbg', 'ceph-test-dbg', 'libcephfs2-dbg', 'librados2-dbg', 'libradosstriper1-dbg', 'librbd1-dbg', 'librgw2-dbg', 'radosgw-dbg', 'rbd-fuse-dbg', 'rbd-mirror-dbg', 'rbd-nbd-dbg'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd', 'ceph-base-debuginfo', 'ceph-common-debuginfo', 'ceph-immutable-object-cache-debuginfo', 'ceph-radosgw-debuginfo', 'ceph-test-debuginfo', 'ceph-base-debuginfo', 'ceph-mgr-debuginfo', 'ceph-mds-debuginfo', 'ceph-mon-debuginfo', 'ceph-osd-debuginfo', 'ceph-fuse-debuginfo', 'librados-devel-debuginfo', 'libcephfs2-debuginfo', 'librados2-debuginfo', 'librbd1-debuginfo', 'python3-cephfs-debuginfo', 'python3-rados-debuginfo', 'python3-rbd-debuginfo', 'python3-rgw-debuginfo', 'rbd-fuse-debuginfo', 'rbd-mirror-debuginfo', 'rbd-nbd-debuginfo']} 2026-02-24T18:16:55.355 INFO:teuthology.task.install:extra packages: [] 2026-02-24T18:16:55.355 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': True, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'f90f4eaebb28b619cf76e80cbbe997656da69579', 'tag': None, 'wait_for_package': False} 2026-02-24T18:16:55.355 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=f90f4eaebb28b619cf76e80cbbe997656da69579 2026-02-24T18:16:55.485 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/centos/9/flavors/default/ 2026-02-24T18:16:55.485 INFO:teuthology.task.install.rpm:Package version is 19.2.3-689.gf90f4eae 2026-02-24T18:16:55.613 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2026-02-24T18:16:55.613 DEBUG:teuthology.orchestra.run.trial038:> set -ex 2026-02-24T18:16:55.613 DEBUG:teuthology.orchestra.run.trial038:> sudo dd of=/etc/yum.repos.d/ceph.repo 2026-02-24T18:16:55.638 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, ceph-base-debuginfo, ceph-common-debuginfo, ceph-immutable-object-cache-debuginfo, ceph-radosgw-debuginfo, ceph-test-debuginfo, ceph-base-debuginfo, ceph-mgr-debuginfo, ceph-mds-debuginfo, ceph-mon-debuginfo, ceph-osd-debuginfo, ceph-fuse-debuginfo, librados-devel-debuginfo, libcephfs2-debuginfo, librados2-debuginfo, librbd1-debuginfo, python3-cephfs-debuginfo, python3-rados-debuginfo, python3-rbd-debuginfo, python3-rgw-debuginfo, rbd-fuse-debuginfo, rbd-mirror-debuginfo, rbd-nbd-debuginfo on remote rpm x86_64 2026-02-24T18:16:55.639 DEBUG:teuthology.orchestra.run.trial038:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/f90f4eaebb28b619cf76e80cbbe997656da69579/;g' /etc/yum.repos.d/ceph.repo ; fi 2026-02-24T18:16:55.705 DEBUG:teuthology.orchestra.run.trial038:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2026-02-24T18:16:55.778 DEBUG:teuthology.orchestra.run.trial038:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2026-02-24T18:16:55.842 INFO:teuthology.orchestra.run.trial038.stdout:check_obsoletes = 1 2026-02-24T18:16:55.843 DEBUG:teuthology.orchestra.run.trial038:> sudo yum clean all 2026-02-24T18:16:55.991 INFO:teuthology.orchestra.run.trial038.stdout:41 files removed 2026-02-24T18:16:56.006 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install ceph-radosgw 2026-02-24T18:16:56.431 INFO:teuthology.orchestra.run.trial038.stdout:ceph packages for x86_64 311 kB/s | 86 kB 00:00 2026-02-24T18:16:56.845 INFO:teuthology.orchestra.run.trial038.stdout:ceph noarch packages 31 kB/s | 12 kB 00:00 2026-02-24T18:16:56.975 INFO:teuthology.orchestra.run.trial038.stdout:ceph source packages 17 kB/s | 1.9 kB 00:00 2026-02-24T18:17:02.566 INFO:teuthology.orchestra.run.trial038.stdout:CentOS Stream 9 - BaseOS 1.6 MB/s | 8.9 MB 00:05 2026-02-24T18:17:10.675 INFO:teuthology.orchestra.run.trial038.stdout:CentOS Stream 9 - AppStream 3.6 MB/s | 27 MB 00:07 2026-02-24T18:17:14.304 INFO:teuthology.orchestra.run.trial038.stdout:CentOS Stream 9 - CRB 6.4 MB/s | 8.0 MB 00:01 2026-02-24T18:17:15.152 INFO:teuthology.orchestra.run.trial038.stdout:CentOS Stream 9 - Extras packages 180 kB/s | 20 kB 00:00 2026-02-24T18:17:16.015 INFO:teuthology.orchestra.run.trial038.stdout:Extra Packages for Enterprise Linux 25 MB/s | 20 MB 00:00 2026-02-24T18:17:19.201 INFO:teuthology.orchestra.run.trial038.stdout:lab-extras 2.7 MB/s | 50 kB 00:00 2026-02-24T18:17:20.435 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-24T18:17:20.436 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-24T18:17:20.436 INFO:teuthology.orchestra.run.trial038.stdout: Package Arch Version Repository Size 2026-02-24T18:17:20.437 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-24T18:17:20.437 INFO:teuthology.orchestra.run.trial038.stdout:Installing: 2026-02-24T18:17:20.437 INFO:teuthology.orchestra.run.trial038.stdout: ceph-radosgw x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 11 M 2026-02-24T18:17:20.437 INFO:teuthology.orchestra.run.trial038.stdout:Upgrading: 2026-02-24T18:17:20.437 INFO:teuthology.orchestra.run.trial038.stdout: librados2 x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 3.4 M 2026-02-24T18:17:20.437 INFO:teuthology.orchestra.run.trial038.stdout: librbd1 x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 3.2 M 2026-02-24T18:17:20.437 INFO:teuthology.orchestra.run.trial038.stdout:Installing dependencies: 2026-02-24T18:17:20.437 INFO:teuthology.orchestra.run.trial038.stdout: boost-program-options x86_64 1.75.0-13.el9 appstream 104 k 2026-02-24T18:17:20.437 INFO:teuthology.orchestra.run.trial038.stdout: ceph-base x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 5.5 M 2026-02-24T18:17:20.437 INFO:teuthology.orchestra.run.trial038.stdout: ceph-common x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 22 M 2026-02-24T18:17:20.437 INFO:teuthology.orchestra.run.trial038.stdout: ceph-selinux x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 25 k 2026-02-24T18:17:20.438 INFO:teuthology.orchestra.run.trial038.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2026-02-24T18:17:20.438 INFO:teuthology.orchestra.run.trial038.stdout: libarrow x86_64 9.0.0-15.el9 epel 4.4 M 2026-02-24T18:17:20.438 INFO:teuthology.orchestra.run.trial038.stdout: libarrow-doc noarch 9.0.0-15.el9 epel 25 k 2026-02-24T18:17:20.438 INFO:teuthology.orchestra.run.trial038.stdout: libcephfs2 x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 999 k 2026-02-24T18:17:20.438 INFO:teuthology.orchestra.run.trial038.stdout: libnbd x86_64 1.20.3-4.el9 appstream 164 k 2026-02-24T18:17:20.438 INFO:teuthology.orchestra.run.trial038.stdout: liboath x86_64 2.6.12-1.el9 epel 49 k 2026-02-24T18:17:20.438 INFO:teuthology.orchestra.run.trial038.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2026-02-24T18:17:20.438 INFO:teuthology.orchestra.run.trial038.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2026-02-24T18:17:20.438 INFO:teuthology.orchestra.run.trial038.stdout: libradosstriper1 x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 503 k 2026-02-24T18:17:20.438 INFO:teuthology.orchestra.run.trial038.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2026-02-24T18:17:20.438 INFO:teuthology.orchestra.run.trial038.stdout: librgw2 x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 5.4 M 2026-02-24T18:17:20.438 INFO:teuthology.orchestra.run.trial038.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2026-02-24T18:17:20.439 INFO:teuthology.orchestra.run.trial038.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2026-02-24T18:17:20.439 INFO:teuthology.orchestra.run.trial038.stdout: mailcap noarch 2.1.49-5.el9 baseos 33 k 2026-02-24T18:17:20.439 INFO:teuthology.orchestra.run.trial038.stdout: parquet-libs x86_64 9.0.0-15.el9 epel 838 k 2026-02-24T18:17:20.439 INFO:teuthology.orchestra.run.trial038.stdout: pciutils x86_64 3.7.0-7.el9 baseos 93 k 2026-02-24T18:17:20.439 INFO:teuthology.orchestra.run.trial038.stdout: python3-ceph-argparse x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 45 k 2026-02-24T18:17:20.439 INFO:teuthology.orchestra.run.trial038.stdout: python3-ceph-common x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 142 k 2026-02-24T18:17:20.439 INFO:teuthology.orchestra.run.trial038.stdout: python3-cephfs x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 165 k 2026-02-24T18:17:20.439 INFO:teuthology.orchestra.run.trial038.stdout: python3-rados x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 322 k 2026-02-24T18:17:20.439 INFO:teuthology.orchestra.run.trial038.stdout: python3-rbd x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 303 k 2026-02-24T18:17:20.439 INFO:teuthology.orchestra.run.trial038.stdout: python3-rgw x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 100 k 2026-02-24T18:17:20.439 INFO:teuthology.orchestra.run.trial038.stdout: qatlib x86_64 25.08.0-2.el9 appstream 240 k 2026-02-24T18:17:20.439 INFO:teuthology.orchestra.run.trial038.stdout: qatzip-libs x86_64 1.3.1-1.el9 appstream 66 k 2026-02-24T18:17:20.439 INFO:teuthology.orchestra.run.trial038.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2026-02-24T18:17:20.440 INFO:teuthology.orchestra.run.trial038.stdout: thrift x86_64 0.15.0-4.el9 epel 1.6 M 2026-02-24T18:17:20.440 INFO:teuthology.orchestra.run.trial038.stdout:Installing weak dependencies: 2026-02-24T18:17:20.440 INFO:teuthology.orchestra.run.trial038.stdout: nvme-cli x86_64 2.16-1.el9 baseos 1.2 M 2026-02-24T18:17:20.440 INFO:teuthology.orchestra.run.trial038.stdout: qatlib-service x86_64 25.08.0-2.el9 appstream 37 k 2026-02-24T18:17:20.440 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:17:20.440 INFO:teuthology.orchestra.run.trial038.stdout:Transaction Summary 2026-02-24T18:17:20.440 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-24T18:17:20.440 INFO:teuthology.orchestra.run.trial038.stdout:Install 33 Packages 2026-02-24T18:17:20.440 INFO:teuthology.orchestra.run.trial038.stdout:Upgrade 2 Packages 2026-02-24T18:17:20.440 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:17:20.441 INFO:teuthology.orchestra.run.trial038.stdout:Total download size: 63 M 2026-02-24T18:17:20.441 INFO:teuthology.orchestra.run.trial038.stdout:Downloading Packages: 2026-02-24T18:17:22.253 INFO:teuthology.orchestra.run.trial038.stdout:(1/35): ceph-base-19.2.3-689.gf90f4eae.el9.x86_ 4.0 MB/s | 5.5 MB 00:01 2026-02-24T18:17:22.303 INFO:teuthology.orchestra.run.trial038.stdout:(2/35): ceph-radosgw-19.2.3-689.gf90f4eae.el9.x 7.5 MB/s | 11 MB 00:01 2026-02-24T18:17:22.303 INFO:teuthology.orchestra.run.trial038.stdout:(3/35): ceph-selinux-19.2.3-689.gf90f4eae.el9.x 495 kB/s | 25 kB 00:00 2026-02-24T18:17:22.415 INFO:teuthology.orchestra.run.trial038.stdout:(4/35): libcephfs2-19.2.3-689.gf90f4eae.el9.x86 8.8 MB/s | 999 kB 00:00 2026-02-24T18:17:22.416 INFO:teuthology.orchestra.run.trial038.stdout:(5/35): libradosstriper1-19.2.3-689.gf90f4eae.e 4.4 MB/s | 503 kB 00:00 2026-02-24T18:17:22.479 INFO:teuthology.orchestra.run.trial038.stdout:(6/35): python3-ceph-argparse-19.2.3-689.gf90f4 727 kB/s | 45 kB 00:00 2026-02-24T18:17:22.541 INFO:teuthology.orchestra.run.trial038.stdout:(7/35): python3-ceph-common-19.2.3-689.gf90f4ea 2.2 MB/s | 142 kB 00:00 2026-02-24T18:17:22.604 INFO:teuthology.orchestra.run.trial038.stdout:(8/35): python3-cephfs-19.2.3-689.gf90f4eae.el9 2.5 MB/s | 165 kB 00:00 2026-02-24T18:17:22.671 INFO:teuthology.orchestra.run.trial038.stdout:(9/35): python3-rados-19.2.3-689.gf90f4eae.el9. 4.7 MB/s | 322 kB 00:00 2026-02-24T18:17:22.727 INFO:teuthology.orchestra.run.trial038.stdout:(10/35): python3-rbd-19.2.3-689.gf90f4eae.el9.x 5.3 MB/s | 303 kB 00:00 2026-02-24T18:17:22.774 INFO:teuthology.orchestra.run.trial038.stdout:(11/35): librgw2-19.2.3-689.gf90f4eae.el9.x86_6 15 MB/s | 5.4 MB 00:00 2026-02-24T18:17:22.776 INFO:teuthology.orchestra.run.trial038.stdout:(12/35): python3-rgw-19.2.3-689.gf90f4eae.el9.x 2.0 MB/s | 100 kB 00:00 2026-02-24T18:17:22.802 INFO:teuthology.orchestra.run.trial038.stdout:(13/35): mailcap-2.1.49-5.el9.noarch.rpm 1.2 MB/s | 33 kB 00:00 2026-02-24T18:17:22.825 INFO:teuthology.orchestra.run.trial038.stdout:(14/35): pciutils-3.7.0-7.el9.x86_64.rpm 3.9 MB/s | 93 kB 00:00 2026-02-24T18:17:22.848 INFO:teuthology.orchestra.run.trial038.stdout:(15/35): nvme-cli-2.16-1.el9.x86_64.rpm 16 MB/s | 1.2 MB 00:00 2026-02-24T18:17:22.959 INFO:teuthology.orchestra.run.trial038.stdout:(16/35): boost-program-options-1.75.0-13.el9.x8 780 kB/s | 104 kB 00:00 2026-02-24T18:17:22.990 INFO:teuthology.orchestra.run.trial038.stdout:(17/35): libnbd-1.20.3-4.el9.x86_64.rpm 1.1 MB/s | 164 kB 00:00 2026-02-24T18:17:23.014 INFO:teuthology.orchestra.run.trial038.stdout:(18/35): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.9 MB/s | 160 kB 00:00 2026-02-24T18:17:23.018 INFO:teuthology.orchestra.run.trial038.stdout:(19/35): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.5 MB/s | 45 kB 00:00 2026-02-24T18:17:23.074 INFO:teuthology.orchestra.run.trial038.stdout:(20/35): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.1 MB/s | 292 kB 00:00 2026-02-24T18:17:23.088 INFO:teuthology.orchestra.run.trial038.stdout:(21/35): librdkafka-1.6.1-102.el9.x86_64.rpm 8.7 MB/s | 662 kB 00:00 2026-02-24T18:17:23.107 INFO:teuthology.orchestra.run.trial038.stdout:(22/35): qatlib-25.08.0-2.el9.x86_64.rpm 7.1 MB/s | 240 kB 00:00 2026-02-24T18:17:23.114 INFO:teuthology.orchestra.run.trial038.stdout:(23/35): qatlib-service-25.08.0-2.el9.x86_64.rp 1.4 MB/s | 37 kB 00:00 2026-02-24T18:17:23.134 INFO:teuthology.orchestra.run.trial038.stdout:(24/35): qatzip-libs-1.3.1-1.el9.x86_64.rpm 2.5 MB/s | 66 kB 00:00 2026-02-24T18:17:23.564 INFO:teuthology.orchestra.run.trial038.stdout:(25/35): gperftools-libs-2.9.1-3.el9.x86_64.rpm 683 kB/s | 308 kB 00:00 2026-02-24T18:17:23.634 INFO:teuthology.orchestra.run.trial038.stdout:(26/35): libarrow-doc-9.0.0-15.el9.noarch.rpm 354 kB/s | 25 kB 00:00 2026-02-24T18:17:23.708 INFO:teuthology.orchestra.run.trial038.stdout:(27/35): liboath-2.6.12-1.el9.x86_64.rpm 667 kB/s | 49 kB 00:00 2026-02-24T18:17:23.784 INFO:teuthology.orchestra.run.trial038.stdout:(28/35): libunwind-1.6.2-1.el9.x86_64.rpm 885 kB/s | 67 kB 00:00 2026-02-24T18:17:23.866 INFO:teuthology.orchestra.run.trial038.stdout:(29/35): libarrow-9.0.0-15.el9.x86_64.rpm 6.0 MB/s | 4.4 MB 00:00 2026-02-24T18:17:23.938 INFO:teuthology.orchestra.run.trial038.stdout:(30/35): re2-20211101-20.el9.x86_64.rpm 2.6 MB/s | 191 kB 00:00 2026-02-24T18:17:23.977 INFO:teuthology.orchestra.run.trial038.stdout:(31/35): parquet-libs-9.0.0-15.el9.x86_64.rpm 4.3 MB/s | 838 kB 00:00 2026-02-24T18:17:24.026 INFO:teuthology.orchestra.run.trial038.stdout:(32/35): thrift-0.15.0-4.el9.x86_64.rpm 18 MB/s | 1.6 MB 00:00 2026-02-24T18:17:24.715 INFO:teuthology.orchestra.run.trial038.stdout:(33/35): ceph-common-19.2.3-689.gf90f4eae.el9.x 5.7 MB/s | 22 MB 00:03 2026-02-24T18:17:24.920 INFO:teuthology.orchestra.run.trial038.stdout:(34/35): librbd1-19.2.3-689.gf90f4eae.el9.x86_6 3.5 MB/s | 3.2 MB 00:00 2026-02-24T18:17:24.988 INFO:teuthology.orchestra.run.trial038.stdout:(35/35): librados2-19.2.3-689.gf90f4eae.el9.x86 3.4 MB/s | 3.4 MB 00:01 2026-02-24T18:17:24.991 INFO:teuthology.orchestra.run.trial038.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:17:24.991 INFO:teuthology.orchestra.run.trial038.stdout:Total 14 MB/s | 63 MB 00:04 2026-02-24T18:17:25.096 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction check 2026-02-24T18:17:25.114 INFO:teuthology.orchestra.run.trial038.stdout:Transaction check succeeded. 2026-02-24T18:17:25.114 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction test 2026-02-24T18:17:25.364 INFO:teuthology.orchestra.run.trial038.stdout:Transaction test succeeded. 2026-02-24T18:17:25.364 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction 2026-02-24T18:17:25.656 INFO:teuthology.orchestra.run.trial038.stdout: Preparing : 1/1 2026-02-24T18:17:25.664 INFO:teuthology.orchestra.run.trial038.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/37 2026-02-24T18:17:25.666 INFO:teuthology.orchestra.run.trial038.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/37 2026-02-24T18:17:25.678 INFO:teuthology.orchestra.run.trial038.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/37 2026-02-24T18:17:25.828 INFO:teuthology.orchestra.run.trial038.stdout: Installing : thrift-0.15.0-4.el9.x86_64 4/37 2026-02-24T18:17:25.832 INFO:teuthology.orchestra.run.trial038.stdout: Upgrading : librados2-2:19.2.3-689.gf90f4eae.el9.x86_64 5/37 2026-02-24T18:17:25.862 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: librados2-2:19.2.3-689.gf90f4eae.el9.x86_64 5/37 2026-02-24T18:17:25.905 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-rados-2:19.2.3-689.gf90f4eae.el9.x86_64 6/37 2026-02-24T18:17:25.905 INFO:teuthology.orchestra.run.trial038.stdout: Installing : libcephfs2-2:19.2.3-689.gf90f4eae.el9.x86_64 7/37 2026-02-24T18:17:25.919 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: libcephfs2-2:19.2.3-689.gf90f4eae.el9.x86_64 7/37 2026-02-24T18:17:25.923 INFO:teuthology.orchestra.run.trial038.stdout: Installing : liboath-2.6.12-1.el9.x86_64 8/37 2026-02-24T18:17:25.933 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-ceph-argparse-2:19.2.3-689.gf90f4eae.el9.x 9/37 2026-02-24T18:17:25.956 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-cephfs-2:19.2.3-689.gf90f4eae.el9.x86_64 10/37 2026-02-24T18:17:25.957 INFO:teuthology.orchestra.run.trial038.stdout: Installing : libradosstriper1-2:19.2.3-689.gf90f4eae.el9.x86_64 11/37 2026-02-24T18:17:25.970 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: libradosstriper1-2:19.2.3-689.gf90f4eae.el9.x86_64 11/37 2026-02-24T18:17:25.999 INFO:teuthology.orchestra.run.trial038.stdout: Installing : re2-1:20211101-20.el9.x86_64 12/37 2026-02-24T18:17:26.004 INFO:teuthology.orchestra.run.trial038.stdout: Installing : libarrow-9.0.0-15.el9.x86_64 13/37 2026-02-24T18:17:26.011 INFO:teuthology.orchestra.run.trial038.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/37 2026-02-24T18:17:26.013 INFO:teuthology.orchestra.run.trial038.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 15/37 2026-02-24T18:17:26.017 INFO:teuthology.orchestra.run.trial038.stdout: Installing : libarrow-doc-9.0.0-15.el9.noarch 16/37 2026-02-24T18:17:26.021 INFO:teuthology.orchestra.run.trial038.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 17/37 2026-02-24T18:17:26.160 INFO:teuthology.orchestra.run.trial038.stdout: Installing : libnbd-1.20.3-4.el9.x86_64 18/37 2026-02-24T18:17:26.161 INFO:teuthology.orchestra.run.trial038.stdout: Upgrading : librbd1-2:19.2.3-689.gf90f4eae.el9.x86_64 19/37 2026-02-24T18:17:26.186 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: librbd1-2:19.2.3-689.gf90f4eae.el9.x86_64 19/37 2026-02-24T18:17:26.188 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-rbd-2:19.2.3-689.gf90f4eae.el9.x86_64 20/37 2026-02-24T18:17:26.194 INFO:teuthology.orchestra.run.trial038.stdout: Installing : boost-program-options-1.75.0-13.el9.x86_64 21/37 2026-02-24T18:17:26.417 INFO:teuthology.orchestra.run.trial038.stdout: Installing : parquet-libs-9.0.0-15.el9.x86_64 22/37 2026-02-24T18:17:26.419 INFO:teuthology.orchestra.run.trial038.stdout: Installing : librgw2-2:19.2.3-689.gf90f4eae.el9.x86_64 23/37 2026-02-24T18:17:26.435 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: librgw2-2:19.2.3-689.gf90f4eae.el9.x86_64 23/37 2026-02-24T18:17:26.439 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-rgw-2:19.2.3-689.gf90f4eae.el9.x86_64 24/37 2026-02-24T18:17:26.443 INFO:teuthology.orchestra.run.trial038.stdout: Installing : pciutils-3.7.0-7.el9.x86_64 25/37 2026-02-24T18:17:26.446 INFO:teuthology.orchestra.run.trial038.stdout: Installing : qatlib-25.08.0-2.el9.x86_64 26/37 2026-02-24T18:17:26.447 INFO:teuthology.orchestra.run.trial038.stdout: Installing : qatlib-service-25.08.0-2.el9.x86_64 27/37 2026-02-24T18:17:26.465 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: qatlib-service-25.08.0-2.el9.x86_64 27/37 2026-02-24T18:17:26.582 INFO:teuthology.orchestra.run.trial038.stdout: Installing : qatzip-libs-1.3.1-1.el9.x86_64 28/37 2026-02-24T18:17:26.584 INFO:teuthology.orchestra.run.trial038.stdout: Installing : nvme-cli-2.16-1.el9.x86_64 29/37 2026-02-24T18:17:26.865 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: nvme-cli-2.16-1.el9.x86_64 29/37 2026-02-24T18:17:26.865 INFO:teuthology.orchestra.run.trial038.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service → /usr/lib/systemd/system/nvmefc-boot-connections.service. 2026-02-24T18:17:26.865 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:17:26.903 INFO:teuthology.orchestra.run.trial038.stdout: Installing : mailcap-2.1.49-5.el9.noarch 30/37 2026-02-24T18:17:26.904 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-ceph-common-2:19.2.3-689.gf90f4eae.el9.x86 31/37 2026-02-24T18:17:27.880 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: ceph-common-2:19.2.3-689.gf90f4eae.el9.x86_64 32/37 2026-02-24T18:17:27.882 INFO:teuthology.orchestra.run.trial038.stdout: Installing : ceph-common-2:19.2.3-689.gf90f4eae.el9.x86_64 32/37 2026-02-24T18:17:28.169 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: ceph-common-2:19.2.3-689.gf90f4eae.el9.x86_64 32/37 2026-02-24T18:17:28.172 INFO:teuthology.orchestra.run.trial038.stdout: Installing : ceph-base-2:19.2.3-689.gf90f4eae.el9.x86_64 33/37 2026-02-24T18:17:28.214 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: ceph-base-2:19.2.3-689.gf90f4eae.el9.x86_64 33/37 2026-02-24T18:17:28.214 INFO:teuthology.orchestra.run.trial038.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2026-02-24T18:17:28.214 INFO:teuthology.orchestra.run.trial038.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2026-02-24T18:17:28.215 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:17:28.218 INFO:teuthology.orchestra.run.trial038.stdout: Installing : ceph-selinux-2:19.2.3-689.gf90f4eae.el9.x86_64 34/37 2026-02-24T18:17:32.701 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: ceph-selinux-2:19.2.3-689.gf90f4eae.el9.x86_64 34/37 2026-02-24T18:17:32.702 INFO:teuthology.orchestra.run.trial038.stdout:skipping the directory /sys 2026-02-24T18:17:32.702 INFO:teuthology.orchestra.run.trial038.stdout:skipping the directory /proc 2026-02-24T18:17:32.702 INFO:teuthology.orchestra.run.trial038.stdout:skipping the directory /mnt 2026-02-24T18:17:32.702 INFO:teuthology.orchestra.run.trial038.stdout:skipping the directory /var/tmp 2026-02-24T18:17:32.702 INFO:teuthology.orchestra.run.trial038.stdout:skipping the directory /home 2026-02-24T18:17:32.702 INFO:teuthology.orchestra.run.trial038.stdout:skipping the directory /root 2026-02-24T18:17:32.702 INFO:teuthology.orchestra.run.trial038.stdout:skipping the directory /tmp 2026-02-24T18:17:32.702 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:17:33.148 INFO:teuthology.orchestra.run.trial038.stdout: Installing : ceph-radosgw-2:19.2.3-689.gf90f4eae.el9.x86_64 35/37 2026-02-24T18:17:33.173 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: ceph-radosgw-2:19.2.3-689.gf90f4eae.el9.x86_64 35/37 2026-02-24T18:17:33.173 INFO:teuthology.orchestra.run.trial038.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-24T18:17:33.173 INFO:teuthology.orchestra.run.trial038.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2026-02-24T18:17:33.173 INFO:teuthology.orchestra.run.trial038.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-24T18:17:33.173 INFO:teuthology.orchestra.run.trial038.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-24T18:17:33.173 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:17:33.175 INFO:teuthology.orchestra.run.trial038.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 36/37 2026-02-24T18:17:33.187 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 36/37 2026-02-24T18:17:33.188 INFO:teuthology.orchestra.run.trial038.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 37/37 2026-02-24T18:17:33.244 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 37/37 2026-02-24T18:17:33.244 INFO:teuthology.orchestra.run.trial038.stdout:Creating group 'qat' with GID 992. 2026-02-24T18:17:33.245 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:17:33.525 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : ceph-base-2:19.2.3-689.gf90f4eae.el9.x86_64 1/37 2026-02-24T18:17:33.525 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : ceph-common-2:19.2.3-689.gf90f4eae.el9.x86_64 2/37 2026-02-24T18:17:33.525 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : ceph-radosgw-2:19.2.3-689.gf90f4eae.el9.x86_64 3/37 2026-02-24T18:17:33.525 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : ceph-selinux-2:19.2.3-689.gf90f4eae.el9.x86_64 4/37 2026-02-24T18:17:33.525 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : libcephfs2-2:19.2.3-689.gf90f4eae.el9.x86_64 5/37 2026-02-24T18:17:33.525 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : libradosstriper1-2:19.2.3-689.gf90f4eae.el9.x86_64 6/37 2026-02-24T18:17:33.525 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : librgw2-2:19.2.3-689.gf90f4eae.el9.x86_64 7/37 2026-02-24T18:17:33.525 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-ceph-argparse-2:19.2.3-689.gf90f4eae.el9.x 8/37 2026-02-24T18:17:33.526 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-ceph-common-2:19.2.3-689.gf90f4eae.el9.x86 9/37 2026-02-24T18:17:33.526 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-cephfs-2:19.2.3-689.gf90f4eae.el9.x86_64 10/37 2026-02-24T18:17:33.526 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-rados-2:19.2.3-689.gf90f4eae.el9.x86_64 11/37 2026-02-24T18:17:33.526 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-rbd-2:19.2.3-689.gf90f4eae.el9.x86_64 12/37 2026-02-24T18:17:33.526 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-rgw-2:19.2.3-689.gf90f4eae.el9.x86_64 13/37 2026-02-24T18:17:33.526 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : mailcap-2.1.49-5.el9.noarch 14/37 2026-02-24T18:17:33.526 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : nvme-cli-2.16-1.el9.x86_64 15/37 2026-02-24T18:17:33.526 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : pciutils-3.7.0-7.el9.x86_64 16/37 2026-02-24T18:17:33.526 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : boost-program-options-1.75.0-13.el9.x86_64 17/37 2026-02-24T18:17:33.526 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : libnbd-1.20.3-4.el9.x86_64 18/37 2026-02-24T18:17:33.527 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 19/37 2026-02-24T18:17:33.527 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 20/37 2026-02-24T18:17:33.527 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 21/37 2026-02-24T18:17:33.527 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 22/37 2026-02-24T18:17:33.527 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : qatlib-25.08.0-2.el9.x86_64 23/37 2026-02-24T18:17:33.527 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : qatlib-service-25.08.0-2.el9.x86_64 24/37 2026-02-24T18:17:33.527 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : qatzip-libs-1.3.1-1.el9.x86_64 25/37 2026-02-24T18:17:33.527 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 26/37 2026-02-24T18:17:33.527 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : libarrow-9.0.0-15.el9.x86_64 27/37 2026-02-24T18:17:33.527 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : libarrow-doc-9.0.0-15.el9.noarch 28/37 2026-02-24T18:17:33.527 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : liboath-2.6.12-1.el9.x86_64 29/37 2026-02-24T18:17:33.528 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 30/37 2026-02-24T18:17:33.528 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : parquet-libs-9.0.0-15.el9.x86_64 31/37 2026-02-24T18:17:33.528 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : re2-1:20211101-20.el9.x86_64 32/37 2026-02-24T18:17:33.528 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : thrift-0.15.0-4.el9.x86_64 33/37 2026-02-24T18:17:33.528 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : librados2-2:19.2.3-689.gf90f4eae.el9.x86_64 34/37 2026-02-24T18:17:33.528 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 35/37 2026-02-24T18:17:33.528 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : librbd1-2:19.2.3-689.gf90f4eae.el9.x86_64 36/37 2026-02-24T18:17:33.590 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 37/37 2026-02-24T18:17:33.590 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:17:33.591 INFO:teuthology.orchestra.run.trial038.stdout:Upgraded: 2026-02-24T18:17:33.591 INFO:teuthology.orchestra.run.trial038.stdout: librados2-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:17:33.591 INFO:teuthology.orchestra.run.trial038.stdout: librbd1-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:17:33.591 INFO:teuthology.orchestra.run.trial038.stdout:Installed: 2026-02-24T18:17:33.591 INFO:teuthology.orchestra.run.trial038.stdout: boost-program-options-1.75.0-13.el9.x86_64 2026-02-24T18:17:33.591 INFO:teuthology.orchestra.run.trial038.stdout: ceph-base-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:17:33.591 INFO:teuthology.orchestra.run.trial038.stdout: ceph-common-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:17:33.591 INFO:teuthology.orchestra.run.trial038.stdout: ceph-radosgw-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:17:33.591 INFO:teuthology.orchestra.run.trial038.stdout: ceph-selinux-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:17:33.591 INFO:teuthology.orchestra.run.trial038.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2026-02-24T18:17:33.591 INFO:teuthology.orchestra.run.trial038.stdout: libarrow-9.0.0-15.el9.x86_64 2026-02-24T18:17:33.592 INFO:teuthology.orchestra.run.trial038.stdout: libarrow-doc-9.0.0-15.el9.noarch 2026-02-24T18:17:33.592 INFO:teuthology.orchestra.run.trial038.stdout: libcephfs2-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:17:33.592 INFO:teuthology.orchestra.run.trial038.stdout: libnbd-1.20.3-4.el9.x86_64 2026-02-24T18:17:33.592 INFO:teuthology.orchestra.run.trial038.stdout: liboath-2.6.12-1.el9.x86_64 2026-02-24T18:17:33.592 INFO:teuthology.orchestra.run.trial038.stdout: libpmemobj-1.12.1-1.el9.x86_64 2026-02-24T18:17:33.592 INFO:teuthology.orchestra.run.trial038.stdout: librabbitmq-0.11.0-7.el9.x86_64 2026-02-24T18:17:33.592 INFO:teuthology.orchestra.run.trial038.stdout: libradosstriper1-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:17:33.592 INFO:teuthology.orchestra.run.trial038.stdout: librdkafka-1.6.1-102.el9.x86_64 2026-02-24T18:17:33.592 INFO:teuthology.orchestra.run.trial038.stdout: librgw2-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:17:33.592 INFO:teuthology.orchestra.run.trial038.stdout: libunwind-1.6.2-1.el9.x86_64 2026-02-24T18:17:33.592 INFO:teuthology.orchestra.run.trial038.stdout: lttng-ust-2.12.0-6.el9.x86_64 2026-02-24T18:17:33.592 INFO:teuthology.orchestra.run.trial038.stdout: mailcap-2.1.49-5.el9.noarch 2026-02-24T18:17:33.592 INFO:teuthology.orchestra.run.trial038.stdout: nvme-cli-2.16-1.el9.x86_64 2026-02-24T18:17:33.593 INFO:teuthology.orchestra.run.trial038.stdout: parquet-libs-9.0.0-15.el9.x86_64 2026-02-24T18:17:33.593 INFO:teuthology.orchestra.run.trial038.stdout: pciutils-3.7.0-7.el9.x86_64 2026-02-24T18:17:33.593 INFO:teuthology.orchestra.run.trial038.stdout: python3-ceph-argparse-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:17:33.593 INFO:teuthology.orchestra.run.trial038.stdout: python3-ceph-common-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:17:33.593 INFO:teuthology.orchestra.run.trial038.stdout: python3-cephfs-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:17:33.593 INFO:teuthology.orchestra.run.trial038.stdout: python3-rados-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:17:33.593 INFO:teuthology.orchestra.run.trial038.stdout: python3-rbd-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:17:33.593 INFO:teuthology.orchestra.run.trial038.stdout: python3-rgw-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:17:33.593 INFO:teuthology.orchestra.run.trial038.stdout: qatlib-25.08.0-2.el9.x86_64 2026-02-24T18:17:33.593 INFO:teuthology.orchestra.run.trial038.stdout: qatlib-service-25.08.0-2.el9.x86_64 2026-02-24T18:17:33.593 INFO:teuthology.orchestra.run.trial038.stdout: qatzip-libs-1.3.1-1.el9.x86_64 2026-02-24T18:17:33.593 INFO:teuthology.orchestra.run.trial038.stdout: re2-1:20211101-20.el9.x86_64 2026-02-24T18:17:33.593 INFO:teuthology.orchestra.run.trial038.stdout: thrift-0.15.0-4.el9.x86_64 2026-02-24T18:17:33.593 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:17:33.594 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-24T18:17:33.632 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install ceph-test 2026-02-24T18:17:33.948 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:00:14 ago on Tue 24 Feb 2026 06:17:19 PM UTC. 2026-02-24T18:17:34.031 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-24T18:17:34.031 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-24T18:17:34.031 INFO:teuthology.orchestra.run.trial038.stdout: Package Arch Version Repository Size 2026-02-24T18:17:34.031 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-24T18:17:34.031 INFO:teuthology.orchestra.run.trial038.stdout:Installing: 2026-02-24T18:17:34.031 INFO:teuthology.orchestra.run.trial038.stdout: ceph-test x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 50 M 2026-02-24T18:17:34.031 INFO:teuthology.orchestra.run.trial038.stdout:Installing dependencies: 2026-02-24T18:17:34.031 INFO:teuthology.orchestra.run.trial038.stdout: libcephsqlite x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 163 k 2026-02-24T18:17:34.031 INFO:teuthology.orchestra.run.trial038.stdout: libxslt x86_64 1.1.34-12.el9 appstream 233 k 2026-02-24T18:17:34.031 INFO:teuthology.orchestra.run.trial038.stdout: socat x86_64 1.7.4.1-8.el9 appstream 303 k 2026-02-24T18:17:34.031 INFO:teuthology.orchestra.run.trial038.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2026-02-24T18:17:34.031 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:17:34.031 INFO:teuthology.orchestra.run.trial038.stdout:Transaction Summary 2026-02-24T18:17:34.032 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-24T18:17:34.032 INFO:teuthology.orchestra.run.trial038.stdout:Install 5 Packages 2026-02-24T18:17:34.032 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:17:34.032 INFO:teuthology.orchestra.run.trial038.stdout:Total download size: 50 M 2026-02-24T18:17:34.032 INFO:teuthology.orchestra.run.trial038.stdout:Installed size: 212 M 2026-02-24T18:17:34.032 INFO:teuthology.orchestra.run.trial038.stdout:Downloading Packages: 2026-02-24T18:17:34.537 INFO:teuthology.orchestra.run.trial038.stdout:(1/5): libcephsqlite-19.2.3-689.gf90f4eae.el9.x 389 kB/s | 163 kB 00:00 2026-02-24T18:17:34.612 INFO:teuthology.orchestra.run.trial038.stdout:(2/5): libxslt-1.1.34-12.el9.x86_64.rpm 471 kB/s | 233 kB 00:00 2026-02-24T18:17:34.706 INFO:teuthology.orchestra.run.trial038.stdout:(3/5): xmlstarlet-1.6.1-20.el9.x86_64.rpm 681 kB/s | 64 kB 00:00 2026-02-24T18:17:35.056 INFO:teuthology.orchestra.run.trial038.stdout:(4/5): socat-1.7.4.1-8.el9.x86_64.rpm 585 kB/s | 303 kB 00:00 2026-02-24T18:18:00.831 INFO:teuthology.orchestra.run.trial038.stdout:(5/5): ceph-test-19.2.3-689.gf90f4eae.el9.x86_6 1.9 MB/s | 50 MB 00:26 2026-02-24T18:18:00.832 INFO:teuthology.orchestra.run.trial038.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:18:00.833 INFO:teuthology.orchestra.run.trial038.stdout:Total 1.9 MB/s | 50 MB 00:26 2026-02-24T18:18:00.857 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction check 2026-02-24T18:18:00.867 INFO:teuthology.orchestra.run.trial038.stdout:Transaction check succeeded. 2026-02-24T18:18:00.867 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction test 2026-02-24T18:18:01.057 INFO:teuthology.orchestra.run.trial038.stdout:Transaction test succeeded. 2026-02-24T18:18:01.057 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction 2026-02-24T18:18:01.272 INFO:teuthology.orchestra.run.trial038.stdout: Preparing : 1/1 2026-02-24T18:18:01.278 INFO:teuthology.orchestra.run.trial038.stdout: Installing : socat-1.7.4.1-8.el9.x86_64 1/5 2026-02-24T18:18:01.282 INFO:teuthology.orchestra.run.trial038.stdout: Installing : libxslt-1.1.34-12.el9.x86_64 2/5 2026-02-24T18:18:01.290 INFO:teuthology.orchestra.run.trial038.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 3/5 2026-02-24T18:18:01.290 INFO:teuthology.orchestra.run.trial038.stdout: Installing : libcephsqlite-2:19.2.3-689.gf90f4eae.el9.x86_64 4/5 2026-02-24T18:18:03.541 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: libcephsqlite-2:19.2.3-689.gf90f4eae.el9.x86_64 4/5 2026-02-24T18:18:03.560 INFO:teuthology.orchestra.run.trial038.stdout: Installing : ceph-test-2:19.2.3-689.gf90f4eae.el9.x86_64 5/5 2026-02-24T18:18:03.805 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: ceph-test-2:19.2.3-689.gf90f4eae.el9.x86_64 5/5 2026-02-24T18:18:03.805 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : ceph-test-2:19.2.3-689.gf90f4eae.el9.x86_64 1/5 2026-02-24T18:18:03.806 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : libcephsqlite-2:19.2.3-689.gf90f4eae.el9.x86_64 2/5 2026-02-24T18:18:03.806 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : libxslt-1.1.34-12.el9.x86_64 3/5 2026-02-24T18:18:03.806 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : socat-1.7.4.1-8.el9.x86_64 4/5 2026-02-24T18:18:03.863 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 5/5 2026-02-24T18:18:03.864 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:18:03.864 INFO:teuthology.orchestra.run.trial038.stdout:Installed: 2026-02-24T18:18:03.864 INFO:teuthology.orchestra.run.trial038.stdout: ceph-test-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:18:03.864 INFO:teuthology.orchestra.run.trial038.stdout: libcephsqlite-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:18:03.864 INFO:teuthology.orchestra.run.trial038.stdout: libxslt-1.1.34-12.el9.x86_64 2026-02-24T18:18:03.864 INFO:teuthology.orchestra.run.trial038.stdout: socat-1.7.4.1-8.el9.x86_64 2026-02-24T18:18:03.864 INFO:teuthology.orchestra.run.trial038.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2026-02-24T18:18:03.865 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:18:03.865 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-24T18:18:03.896 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install ceph 2026-02-24T18:18:04.209 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:00:45 ago on Tue 24 Feb 2026 06:17:19 PM UTC. 2026-02-24T18:18:04.297 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-24T18:18:04.300 INFO:teuthology.orchestra.run.trial038.stdout:====================================================================================== 2026-02-24T18:18:04.300 INFO:teuthology.orchestra.run.trial038.stdout: Package Arch Version Repository Size 2026-02-24T18:18:04.300 INFO:teuthology.orchestra.run.trial038.stdout:====================================================================================== 2026-02-24T18:18:04.300 INFO:teuthology.orchestra.run.trial038.stdout:Installing: 2026-02-24T18:18:04.300 INFO:teuthology.orchestra.run.trial038.stdout: ceph x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 6.5 k 2026-02-24T18:18:04.300 INFO:teuthology.orchestra.run.trial038.stdout:Installing dependencies: 2026-02-24T18:18:04.300 INFO:teuthology.orchestra.run.trial038.stdout: abseil-cpp x86_64 20211102.0-4.el9 epel 551 k 2026-02-24T18:18:04.300 INFO:teuthology.orchestra.run.trial038.stdout: ceph-grafana-dashboards noarch 2:19.2.3-689.gf90f4eae.el9 ceph-noarch 31 k 2026-02-24T18:18:04.300 INFO:teuthology.orchestra.run.trial038.stdout: ceph-mds x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 2.4 M 2026-02-24T18:18:04.300 INFO:teuthology.orchestra.run.trial038.stdout: ceph-mgr x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 1.1 M 2026-02-24T18:18:04.300 INFO:teuthology.orchestra.run.trial038.stdout: ceph-mgr-modules-core noarch 2:19.2.3-689.gf90f4eae.el9 ceph-noarch 253 k 2026-02-24T18:18:04.301 INFO:teuthology.orchestra.run.trial038.stdout: ceph-mon x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 4.7 M 2026-02-24T18:18:04.301 INFO:teuthology.orchestra.run.trial038.stdout: ceph-osd x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 17 M 2026-02-24T18:18:04.301 INFO:teuthology.orchestra.run.trial038.stdout: ceph-prometheus-alerts noarch 2:19.2.3-689.gf90f4eae.el9 ceph-noarch 17 k 2026-02-24T18:18:04.301 INFO:teuthology.orchestra.run.trial038.stdout: cephadm noarch 2:19.2.3-689.gf90f4eae.el9 ceph-noarch 769 k 2026-02-24T18:18:04.301 INFO:teuthology.orchestra.run.trial038.stdout: cryptsetup x86_64 2.8.1-3.el9 baseos 351 k 2026-02-24T18:18:04.301 INFO:teuthology.orchestra.run.trial038.stdout: flexiblas x86_64 3.0.4-9.el9 appstream 30 k 2026-02-24T18:18:04.301 INFO:teuthology.orchestra.run.trial038.stdout: flexiblas-netlib x86_64 3.0.4-9.el9 appstream 3.0 M 2026-02-24T18:18:04.301 INFO:teuthology.orchestra.run.trial038.stdout: flexiblas-openblas-openmp x86_64 3.0.4-9.el9 appstream 15 k 2026-02-24T18:18:04.301 INFO:teuthology.orchestra.run.trial038.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2026-02-24T18:18:04.301 INFO:teuthology.orchestra.run.trial038.stdout: ledmon-libs x86_64 1.1.0-3.el9 baseos 40 k 2026-02-24T18:18:04.301 INFO:teuthology.orchestra.run.trial038.stdout: libconfig x86_64 1.7.2-9.el9 baseos 72 k 2026-02-24T18:18:04.301 INFO:teuthology.orchestra.run.trial038.stdout: libgfortran x86_64 11.5.0-14.el9 baseos 794 k 2026-02-24T18:18:04.301 INFO:teuthology.orchestra.run.trial038.stdout: libquadmath x86_64 11.5.0-14.el9 baseos 184 k 2026-02-24T18:18:04.301 INFO:teuthology.orchestra.run.trial038.stdout: libstoragemgmt x86_64 1.10.1-1.el9 appstream 246 k 2026-02-24T18:18:04.302 INFO:teuthology.orchestra.run.trial038.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2026-02-24T18:18:04.302 INFO:teuthology.orchestra.run.trial038.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2026-02-24T18:18:04.302 INFO:teuthology.orchestra.run.trial038.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2026-02-24T18:18:04.302 INFO:teuthology.orchestra.run.trial038.stdout: openblas x86_64 0.3.29-1.el9 appstream 42 k 2026-02-24T18:18:04.302 INFO:teuthology.orchestra.run.trial038.stdout: openblas-openmp x86_64 0.3.29-1.el9 appstream 5.3 M 2026-02-24T18:18:04.302 INFO:teuthology.orchestra.run.trial038.stdout: protobuf x86_64 3.14.0-17.el9 appstream 1.0 M 2026-02-24T18:18:04.302 INFO:teuthology.orchestra.run.trial038.stdout: protobuf-compiler x86_64 3.14.0-17.el9 crb 862 k 2026-02-24T18:18:04.302 INFO:teuthology.orchestra.run.trial038.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2026-02-24T18:18:04.302 INFO:teuthology.orchestra.run.trial038.stdout: python3-autocommand noarch 2.2.2-8.el9 epel 29 k 2026-02-24T18:18:04.302 INFO:teuthology.orchestra.run.trial038.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2026-02-24T18:18:04.302 INFO:teuthology.orchestra.run.trial038.stdout: python3-backports-tarfile noarch 1.2.0-1.el9 epel 60 k 2026-02-24T18:18:04.302 INFO:teuthology.orchestra.run.trial038.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2026-02-24T18:18:04.302 INFO:teuthology.orchestra.run.trial038.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2026-02-24T18:18:04.302 INFO:teuthology.orchestra.run.trial038.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2026-02-24T18:18:04.302 INFO:teuthology.orchestra.run.trial038.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2026-02-24T18:18:04.302 INFO:teuthology.orchestra.run.trial038.stdout: python3-cheroot noarch 10.0.1-4.el9 epel 173 k 2026-02-24T18:18:04.303 INFO:teuthology.orchestra.run.trial038.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2026-02-24T18:18:04.303 INFO:teuthology.orchestra.run.trial038.stdout: python3-cryptography x86_64 36.0.1-5.el9 baseos 1.2 M 2026-02-24T18:18:04.303 INFO:teuthology.orchestra.run.trial038.stdout: python3-devel x86_64 3.9.25-3.el9 appstream 244 k 2026-02-24T18:18:04.303 INFO:teuthology.orchestra.run.trial038.stdout: python3-google-auth noarch 1:2.45.0-1.el9 epel 254 k 2026-02-24T18:18:04.303 INFO:teuthology.orchestra.run.trial038.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2026-02-24T18:18:04.303 INFO:teuthology.orchestra.run.trial038.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2026-02-24T18:18:04.303 INFO:teuthology.orchestra.run.trial038.stdout: python3-isodate noarch 0.6.1-3.el9 epel 56 k 2026-02-24T18:18:04.303 INFO:teuthology.orchestra.run.trial038.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2026-02-24T18:18:04.303 INFO:teuthology.orchestra.run.trial038.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2026-02-24T18:18:04.303 INFO:teuthology.orchestra.run.trial038.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2026-02-24T18:18:04.303 INFO:teuthology.orchestra.run.trial038.stdout: python3-jaraco-context noarch 6.0.1-3.el9 epel 20 k 2026-02-24T18:18:04.303 INFO:teuthology.orchestra.run.trial038.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2026-02-24T18:18:04.303 INFO:teuthology.orchestra.run.trial038.stdout: python3-jaraco-text noarch 4.0.0-2.el9 epel 26 k 2026-02-24T18:18:04.303 INFO:teuthology.orchestra.run.trial038.stdout: python3-jinja2 noarch 2.11.3-8.el9 appstream 249 k 2026-02-24T18:18:04.303 INFO:teuthology.orchestra.run.trial038.stdout: python3-kubernetes noarch 1:26.1.0-3.el9 epel 1.0 M 2026-02-24T18:18:04.304 INFO:teuthology.orchestra.run.trial038.stdout: python3-libstoragemgmt x86_64 1.10.1-1.el9 appstream 177 k 2026-02-24T18:18:04.304 INFO:teuthology.orchestra.run.trial038.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2026-02-24T18:18:04.304 INFO:teuthology.orchestra.run.trial038.stdout: python3-lxml x86_64 4.6.5-3.el9 appstream 1.2 M 2026-02-24T18:18:04.304 INFO:teuthology.orchestra.run.trial038.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2026-02-24T18:18:04.304 INFO:teuthology.orchestra.run.trial038.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2026-02-24T18:18:04.304 INFO:teuthology.orchestra.run.trial038.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2026-02-24T18:18:04.304 INFO:teuthology.orchestra.run.trial038.stdout: python3-msgpack x86_64 1.0.3-2.el9 epel 86 k 2026-02-24T18:18:04.304 INFO:teuthology.orchestra.run.trial038.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2026-02-24T18:18:04.304 INFO:teuthology.orchestra.run.trial038.stdout: python3-numpy x86_64 1:1.23.5-2.el9 appstream 6.1 M 2026-02-24T18:18:04.304 INFO:teuthology.orchestra.run.trial038.stdout: python3-numpy-f2py x86_64 1:1.23.5-2.el9 appstream 442 k 2026-02-24T18:18:04.304 INFO:teuthology.orchestra.run.trial038.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2026-02-24T18:18:04.304 INFO:teuthology.orchestra.run.trial038.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2026-02-24T18:18:04.304 INFO:teuthology.orchestra.run.trial038.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2026-02-24T18:18:04.304 INFO:teuthology.orchestra.run.trial038.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2026-02-24T18:18:04.304 INFO:teuthology.orchestra.run.trial038.stdout: python3-protobuf noarch 3.14.0-17.el9 appstream 267 k 2026-02-24T18:18:04.305 INFO:teuthology.orchestra.run.trial038.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2026-02-24T18:18:04.305 INFO:teuthology.orchestra.run.trial038.stdout: python3-pyasn1 noarch 0.4.8-7.el9 appstream 157 k 2026-02-24T18:18:04.305 INFO:teuthology.orchestra.run.trial038.stdout: python3-pyasn1-modules noarch 0.4.8-7.el9 appstream 277 k 2026-02-24T18:18:04.305 INFO:teuthology.orchestra.run.trial038.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2026-02-24T18:18:04.305 INFO:teuthology.orchestra.run.trial038.stdout: python3-pyparsing noarch 2.4.7-9.el9 baseos 150 k 2026-02-24T18:18:04.305 INFO:teuthology.orchestra.run.trial038.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2026-02-24T18:18:04.305 INFO:teuthology.orchestra.run.trial038.stdout: python3-requests noarch 2.25.1-10.el9 baseos 126 k 2026-02-24T18:18:04.305 INFO:teuthology.orchestra.run.trial038.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2026-02-24T18:18:04.305 INFO:teuthology.orchestra.run.trial038.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2026-02-24T18:18:04.305 INFO:teuthology.orchestra.run.trial038.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2026-02-24T18:18:04.305 INFO:teuthology.orchestra.run.trial038.stdout: python3-scipy x86_64 1.9.3-2.el9 appstream 19 M 2026-02-24T18:18:04.305 INFO:teuthology.orchestra.run.trial038.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2026-02-24T18:18:04.305 INFO:teuthology.orchestra.run.trial038.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2026-02-24T18:18:04.305 INFO:teuthology.orchestra.run.trial038.stdout: python3-typing-extensions noarch 4.15.0-1.el9 epel 86 k 2026-02-24T18:18:04.305 INFO:teuthology.orchestra.run.trial038.stdout: python3-urllib3 noarch 1.26.5-7.el9 baseos 218 k 2026-02-24T18:18:04.305 INFO:teuthology.orchestra.run.trial038.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2026-02-24T18:18:04.306 INFO:teuthology.orchestra.run.trial038.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2026-02-24T18:18:04.306 INFO:teuthology.orchestra.run.trial038.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2026-02-24T18:18:04.306 INFO:teuthology.orchestra.run.trial038.stdout: python3-xmlsec x86_64 1.3.13-1.el9 epel 48 k 2026-02-24T18:18:04.306 INFO:teuthology.orchestra.run.trial038.stdout: python3-xmltodict noarch 0.12.0-15.el9 epel 22 k 2026-02-24T18:18:04.306 INFO:teuthology.orchestra.run.trial038.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2026-02-24T18:18:04.306 INFO:teuthology.orchestra.run.trial038.stdout: unzip x86_64 6.0-59.el9 baseos 182 k 2026-02-24T18:18:04.306 INFO:teuthology.orchestra.run.trial038.stdout: xmlsec1 x86_64 1.2.29-13.el9 appstream 189 k 2026-02-24T18:18:04.306 INFO:teuthology.orchestra.run.trial038.stdout: xmlsec1-openssl x86_64 1.2.29-13.el9 appstream 90 k 2026-02-24T18:18:04.306 INFO:teuthology.orchestra.run.trial038.stdout: zip x86_64 3.0-35.el9 baseos 266 k 2026-02-24T18:18:04.306 INFO:teuthology.orchestra.run.trial038.stdout:Installing weak dependencies: 2026-02-24T18:18:04.306 INFO:teuthology.orchestra.run.trial038.stdout: ceph-mgr-cephadm noarch 2:19.2.3-689.gf90f4eae.el9 ceph-noarch 150 k 2026-02-24T18:18:04.306 INFO:teuthology.orchestra.run.trial038.stdout: ceph-mgr-dashboard noarch 2:19.2.3-689.gf90f4eae.el9 ceph-noarch 3.8 M 2026-02-24T18:18:04.306 INFO:teuthology.orchestra.run.trial038.stdout: ceph-mgr-diskprediction-local noarch 2:19.2.3-689.gf90f4eae.el9 ceph-noarch 7.4 M 2026-02-24T18:18:04.306 INFO:teuthology.orchestra.run.trial038.stdout: ceph-mgr-k8sevents noarch 2:19.2.3-689.gf90f4eae.el9 ceph-noarch 22 k 2026-02-24T18:18:04.306 INFO:teuthology.orchestra.run.trial038.stdout: ceph-mgr-rook noarch 2:19.2.3-689.gf90f4eae.el9 ceph-noarch 49 k 2026-02-24T18:18:04.307 INFO:teuthology.orchestra.run.trial038.stdout: ceph-volume noarch 2:19.2.3-689.gf90f4eae.el9 ceph-noarch 299 k 2026-02-24T18:18:04.307 INFO:teuthology.orchestra.run.trial038.stdout: python3-influxdb noarch 5.3.1-1.el9 epel 139 k 2026-02-24T18:18:04.307 INFO:teuthology.orchestra.run.trial038.stdout: python3-saml noarch 1.16.0-1.el9 epel 125 k 2026-02-24T18:18:04.307 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:18:04.307 INFO:teuthology.orchestra.run.trial038.stdout:Transaction Summary 2026-02-24T18:18:04.307 INFO:teuthology.orchestra.run.trial038.stdout:====================================================================================== 2026-02-24T18:18:04.307 INFO:teuthology.orchestra.run.trial038.stdout:Install 99 Packages 2026-02-24T18:18:04.307 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:18:04.308 INFO:teuthology.orchestra.run.trial038.stdout:Total download size: 95 M 2026-02-24T18:18:04.308 INFO:teuthology.orchestra.run.trial038.stdout:Installed size: 520 M 2026-02-24T18:18:04.308 INFO:teuthology.orchestra.run.trial038.stdout:Downloading Packages: 2026-02-24T18:18:05.074 INFO:teuthology.orchestra.run.trial038.stdout:(1/99): ceph-19.2.3-689.gf90f4eae.el9.x86_64.rp 43 kB/s | 6.5 kB 00:00 2026-02-24T18:18:05.385 INFO:teuthology.orchestra.run.trial038.stdout:(2/99): ceph-mgr-19.2.3-689.gf90f4eae.el9.x86_6 2.3 MB/s | 1.1 MB 00:00 2026-02-24T18:18:05.577 INFO:teuthology.orchestra.run.trial038.stdout:(3/99): ceph-mds-19.2.3-689.gf90f4eae.el9.x86_6 3.7 MB/s | 2.4 MB 00:00 2026-02-24T18:18:05.624 INFO:teuthology.orchestra.run.trial038.stdout:(4/99): ceph-grafana-dashboards-19.2.3-689.gf90 671 kB/s | 31 kB 00:00 2026-02-24T18:18:05.668 INFO:teuthology.orchestra.run.trial038.stdout:(5/99): ceph-mgr-cephadm-19.2.3-689.gf90f4eae.e 3.3 MB/s | 150 kB 00:00 2026-02-24T18:18:05.983 INFO:teuthology.orchestra.run.trial038.stdout:(6/99): ceph-mgr-dashboard-19.2.3-689.gf90f4eae 12 MB/s | 3.8 MB 00:00 2026-02-24T18:18:06.085 INFO:teuthology.orchestra.run.trial038.stdout:(7/99): ceph-mon-19.2.3-689.gf90f4eae.el9.x86_6 4.7 MB/s | 4.7 MB 00:01 2026-02-24T18:18:06.123 INFO:teuthology.orchestra.run.trial038.stdout:(8/99): ceph-mgr-k8sevents-19.2.3-689.gf90f4eae 591 kB/s | 22 kB 00:00 2026-02-24T18:18:06.166 INFO:teuthology.orchestra.run.trial038.stdout:(9/99): ceph-mgr-modules-core-19.2.3-689.gf90f4 5.8 MB/s | 253 kB 00:00 2026-02-24T18:18:06.206 INFO:teuthology.orchestra.run.trial038.stdout:(10/99): ceph-mgr-rook-19.2.3-689.gf90f4eae.el9 1.2 MB/s | 49 kB 00:00 2026-02-24T18:18:06.247 INFO:teuthology.orchestra.run.trial038.stdout:(11/99): ceph-prometheus-alerts-19.2.3-689.gf90 408 kB/s | 17 kB 00:00 2026-02-24T18:18:06.295 INFO:teuthology.orchestra.run.trial038.stdout:(12/99): ceph-volume-19.2.3-689.gf90f4eae.el9.n 6.0 MB/s | 299 kB 00:00 2026-02-24T18:18:06.444 INFO:teuthology.orchestra.run.trial038.stdout:(13/99): cephadm-19.2.3-689.gf90f4eae.el9.noarc 5.1 MB/s | 769 kB 00:00 2026-02-24T18:18:06.467 INFO:teuthology.orchestra.run.trial038.stdout:(14/99): ceph-mgr-diskprediction-local-19.2.3-6 15 MB/s | 7.4 MB 00:00 2026-02-24T18:18:06.552 INFO:teuthology.orchestra.run.trial038.stdout:(15/99): ceph-osd-19.2.3-689.gf90f4eae.el9.x86_ 15 MB/s | 17 MB 00:01 2026-02-24T18:18:06.945 INFO:teuthology.orchestra.run.trial038.stdout:(16/99): ledmon-libs-1.1.0-3.el9.x86_64.rpm 85 kB/s | 40 kB 00:00 2026-02-24T18:18:07.130 INFO:teuthology.orchestra.run.trial038.stdout:(17/99): libconfig-1.7.2-9.el9.x86_64.rpm 125 kB/s | 72 kB 00:00 2026-02-24T18:18:07.357 INFO:teuthology.orchestra.run.trial038.stdout:(18/99): libgfortran-11.5.0-14.el9.x86_64.rpm 1.9 MB/s | 794 kB 00:00 2026-02-24T18:18:07.387 INFO:teuthology.orchestra.run.trial038.stdout:(19/99): libquadmath-11.5.0-14.el9.x86_64.rpm 718 kB/s | 184 kB 00:00 2026-02-24T18:18:07.458 INFO:teuthology.orchestra.run.trial038.stdout:(20/99): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.5 MB/s | 253 kB 00:00 2026-02-24T18:18:07.558 INFO:teuthology.orchestra.run.trial038.stdout:(21/99): python3-ply-3.11-14.el9.noarch.rpm 1.0 MB/s | 106 kB 00:00 2026-02-24T18:18:07.658 INFO:teuthology.orchestra.run.trial038.stdout:(22/99): python3-pycparser-2.20-6.el9.noarch.rp 1.3 MB/s | 135 kB 00:00 2026-02-24T18:18:07.726 INFO:teuthology.orchestra.run.trial038.stdout:(23/99): python3-cryptography-36.0.1-5.el9.x86_ 3.7 MB/s | 1.2 MB 00:00 2026-02-24T18:18:07.758 INFO:teuthology.orchestra.run.trial038.stdout:(24/99): python3-pyparsing-2.4.7-9.el9.noarch.r 1.5 MB/s | 150 kB 00:00 2026-02-24T18:18:07.827 INFO:teuthology.orchestra.run.trial038.stdout:(25/99): python3-requests-2.25.1-10.el9.noarch. 1.2 MB/s | 126 kB 00:00 2026-02-24T18:18:07.859 INFO:teuthology.orchestra.run.trial038.stdout:(26/99): python3-urllib3-1.26.5-7.el9.noarch.rp 2.1 MB/s | 218 kB 00:00 2026-02-24T18:18:07.905 INFO:teuthology.orchestra.run.trial038.stdout:(27/99): cryptsetup-2.8.1-3.el9.x86_64.rpm 240 kB/s | 351 kB 00:01 2026-02-24T18:18:07.916 INFO:teuthology.orchestra.run.trial038.stdout:(28/99): flexiblas-3.0.4-9.el9.x86_64.rpm 2.7 MB/s | 30 kB 00:00 2026-02-24T18:18:07.927 INFO:teuthology.orchestra.run.trial038.stdout:(29/99): unzip-6.0-59.el9.x86_64.rpm 1.8 MB/s | 182 kB 00:00 2026-02-24T18:18:07.939 INFO:teuthology.orchestra.run.trial038.stdout:(30/99): flexiblas-openblas-openmp-3.0.4-9.el9. 1.2 MB/s | 15 kB 00:00 2026-02-24T18:18:07.944 INFO:teuthology.orchestra.run.trial038.stdout:(31/99): flexiblas-netlib-3.0.4-9.el9.x86_64.rp 105 MB/s | 3.0 MB 00:00 2026-02-24T18:18:07.950 INFO:teuthology.orchestra.run.trial038.stdout:(32/99): lua-5.4.4-4.el9.x86_64.rpm 31 MB/s | 188 kB 00:00 2026-02-24T18:18:07.955 INFO:teuthology.orchestra.run.trial038.stdout:(33/99): openblas-0.3.29-1.el9.x86_64.rpm 9.6 MB/s | 42 kB 00:00 2026-02-24T18:18:07.955 INFO:teuthology.orchestra.run.trial038.stdout:(34/99): libstoragemgmt-1.10.1-1.el9.x86_64.rpm 14 MB/s | 246 kB 00:00 2026-02-24T18:18:07.958 INFO:teuthology.orchestra.run.trial038.stdout:(35/99): zip-3.0-35.el9.x86_64.rpm 2.6 MB/s | 266 kB 00:00 2026-02-24T18:18:07.976 INFO:teuthology.orchestra.run.trial038.stdout:(36/99): openblas-openmp-0.3.29-1.el9.x86_64.rp 250 MB/s | 5.3 MB 00:00 2026-02-24T18:18:07.983 INFO:teuthology.orchestra.run.trial038.stdout:(37/99): python3-devel-3.9.25-3.el9.x86_64.rpm 33 MB/s | 244 kB 00:00 2026-02-24T18:18:07.988 INFO:teuthology.orchestra.run.trial038.stdout:(38/99): protobuf-3.14.0-17.el9.x86_64.rpm 31 MB/s | 1.0 MB 00:00 2026-02-24T18:18:07.990 INFO:teuthology.orchestra.run.trial038.stdout:(39/99): python3-jinja2-2.11.3-8.el9.noarch.rpm 35 MB/s | 249 kB 00:00 2026-02-24T18:18:07.995 INFO:teuthology.orchestra.run.trial038.stdout:(40/99): python3-libstoragemgmt-1.10.1-1.el9.x8 25 MB/s | 177 kB 00:00 2026-02-24T18:18:08.000 INFO:teuthology.orchestra.run.trial038.stdout:(41/99): python3-lxml-4.6.5-3.el9.x86_64.rpm 128 MB/s | 1.2 MB 00:00 2026-02-24T18:18:08.003 INFO:teuthology.orchestra.run.trial038.stdout:(42/99): python3-mako-1.1.4-6.el9.noarch.rpm 21 MB/s | 172 kB 00:00 2026-02-24T18:18:08.005 INFO:teuthology.orchestra.run.trial038.stdout:(43/99): python3-markupsafe-1.1.1-12.el9.x86_64 6.6 MB/s | 35 kB 00:00 2026-02-24T18:18:08.033 INFO:teuthology.orchestra.run.trial038.stdout:(44/99): python3-numpy-f2py-1.23.5-2.el9.x86_64 16 MB/s | 442 kB 00:00 2026-02-24T18:18:08.039 INFO:teuthology.orchestra.run.trial038.stdout:(45/99): python3-packaging-20.9-5.el9.noarch.rp 12 MB/s | 77 kB 00:00 2026-02-24T18:18:08.046 INFO:teuthology.orchestra.run.trial038.stdout:(46/99): python3-babel-2.9.1-2.el9.noarch.rpm 68 MB/s | 6.0 MB 00:00 2026-02-24T18:18:08.059 INFO:teuthology.orchestra.run.trial038.stdout:(47/99): python3-pyasn1-0.4.8-7.el9.noarch.rpm 12 MB/s | 157 kB 00:00 2026-02-24T18:18:08.060 INFO:teuthology.orchestra.run.trial038.stdout:(48/99): python3-protobuf-3.14.0-17.el9.noarch. 12 MB/s | 267 kB 00:00 2026-02-24T18:18:08.065 INFO:teuthology.orchestra.run.trial038.stdout:(49/99): python3-requests-oauthlib-1.3.0-12.el9 9.6 MB/s | 54 kB 00:00 2026-02-24T18:18:08.086 INFO:teuthology.orchestra.run.trial038.stdout:(50/99): python3-pyasn1-modules-0.4.8-7.el9.noa 10 MB/s | 277 kB 00:00 2026-02-24T18:18:08.090 INFO:teuthology.orchestra.run.trial038.stdout:(51/99): python3-toml-0.10.2-6.el9.noarch.rpm 12 MB/s | 42 kB 00:00 2026-02-24T18:18:08.097 INFO:teuthology.orchestra.run.trial038.stdout:(52/99): python3-numpy-1.23.5-2.el9.x86_64.rpm 65 MB/s | 6.1 MB 00:00 2026-02-24T18:18:08.104 INFO:teuthology.orchestra.run.trial038.stdout:(53/99): xmlsec1-openssl-1.2.29-13.el9.x86_64.r 13 MB/s | 90 kB 00:00 2026-02-24T18:18:08.113 INFO:teuthology.orchestra.run.trial038.stdout:(54/99): xmlsec1-1.2.29-13.el9.x86_64.rpm 8.1 MB/s | 189 kB 00:00 2026-02-24T18:18:08.298 INFO:teuthology.orchestra.run.trial038.stdout:(55/99): python3-scipy-1.9.3-2.el9.x86_64.rpm 83 MB/s | 19 MB 00:00 2026-02-24T18:18:08.567 INFO:teuthology.orchestra.run.trial038.stdout:(56/99): abseil-cpp-20211102.0-4.el9.x86_64.rpm 2.0 MB/s | 551 kB 00:00 2026-02-24T18:18:08.616 INFO:teuthology.orchestra.run.trial038.stdout:(57/99): grpc-data-1.46.7-10.el9.noarch.rpm 393 kB/s | 19 kB 00:00 2026-02-24T18:18:08.649 INFO:teuthology.orchestra.run.trial038.stdout:(58/99): lua-devel-5.4.4-4.el9.x86_64.rpm 41 kB/s | 22 kB 00:00 2026-02-24T18:18:08.671 INFO:teuthology.orchestra.run.trial038.stdout:(59/99): luarocks-3.9.2-5.el9.noarch.rpm 2.7 MB/s | 151 kB 00:00 2026-02-24T18:18:08.706 INFO:teuthology.orchestra.run.trial038.stdout:(60/99): python3-asyncssh-2.13.2-5.el9.noarch.r 9.4 MB/s | 548 kB 00:00 2026-02-24T18:18:08.708 INFO:teuthology.orchestra.run.trial038.stdout:(61/99): python3-autocommand-2.2.2-8.el9.noarch 785 kB/s | 29 kB 00:00 2026-02-24T18:18:08.755 INFO:teuthology.orchestra.run.trial038.stdout:(62/99): python3-backports-tarfile-1.2.0-1.el9. 1.2 MB/s | 60 kB 00:00 2026-02-24T18:18:08.757 INFO:teuthology.orchestra.run.trial038.stdout:(63/99): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 897 kB/s | 43 kB 00:00 2026-02-24T18:18:08.816 INFO:teuthology.orchestra.run.trial038.stdout:(64/99): python3-cachetools-4.2.4-1.el9.noarch. 531 kB/s | 32 kB 00:00 2026-02-24T18:18:08.817 INFO:teuthology.orchestra.run.trial038.stdout:(65/99): python3-certifi-2023.05.07-4.el9.noarc 236 kB/s | 14 kB 00:00 2026-02-24T18:18:08.855 INFO:teuthology.orchestra.run.trial038.stdout:(66/99): python3-cheroot-10.0.1-4.el9.noarch.rp 4.3 MB/s | 173 kB 00:00 2026-02-24T18:18:08.877 INFO:teuthology.orchestra.run.trial038.stdout:(67/99): python3-cherrypy-18.6.1-2.el9.noarch.r 5.9 MB/s | 358 kB 00:00 2026-02-24T18:18:08.902 INFO:teuthology.orchestra.run.trial038.stdout:(68/99): python3-google-auth-2.45.0-1.el9.noarc 5.3 MB/s | 254 kB 00:00 2026-02-24T18:18:09.016 INFO:teuthology.orchestra.run.trial038.stdout:(69/99): python3-grpcio-1.46.7-10.el9.x86_64.rp 15 MB/s | 2.0 MB 00:00 2026-02-24T18:18:09.018 INFO:teuthology.orchestra.run.trial038.stdout:(70/99): protobuf-compiler-3.14.0-17.el9.x86_64 952 kB/s | 862 kB 00:00 2026-02-24T18:18:09.040 INFO:teuthology.orchestra.run.trial038.stdout:(71/99): python3-grpcio-tools-1.46.7-10.el9.x86 1.0 MB/s | 144 kB 00:00 2026-02-24T18:18:09.055 INFO:teuthology.orchestra.run.trial038.stdout:(72/99): python3-influxdb-5.3.1-1.el9.noarch.rp 3.5 MB/s | 139 kB 00:00 2026-02-24T18:18:09.058 INFO:teuthology.orchestra.run.trial038.stdout:(73/99): python3-isodate-0.6.1-3.el9.noarch.rpm 1.4 MB/s | 56 kB 00:00 2026-02-24T18:18:09.071 INFO:teuthology.orchestra.run.trial038.stdout:(74/99): python3-jaraco-8.2.1-3.el9.noarch.rpm 343 kB/s | 11 kB 00:00 2026-02-24T18:18:09.087 INFO:teuthology.orchestra.run.trial038.stdout:(75/99): python3-jaraco-classes-3.2.1-5.el9.noa 558 kB/s | 18 kB 00:00 2026-02-24T18:18:09.091 INFO:teuthology.orchestra.run.trial038.stdout:(76/99): python3-jaraco-collections-3.0.0-8.el9 717 kB/s | 23 kB 00:00 2026-02-24T18:18:09.102 INFO:teuthology.orchestra.run.trial038.stdout:(77/99): python3-jaraco-context-6.0.1-3.el9.noa 623 kB/s | 20 kB 00:00 2026-02-24T18:18:09.119 INFO:teuthology.orchestra.run.trial038.stdout:(78/99): python3-jaraco-functools-3.5.0-2.el9.n 618 kB/s | 19 kB 00:00 2026-02-24T18:18:09.126 INFO:teuthology.orchestra.run.trial038.stdout:(79/99): python3-jaraco-text-4.0.0-2.el9.noarch 747 kB/s | 26 kB 00:00 2026-02-24T18:18:09.184 INFO:teuthology.orchestra.run.trial038.stdout:(80/99): python3-kubernetes-26.1.0-3.el9.noarch 13 MB/s | 1.0 MB 00:00 2026-02-24T18:18:09.185 INFO:teuthology.orchestra.run.trial038.stdout:(81/99): python3-logutils-0.3.5-21.el9.noarch.r 697 kB/s | 46 kB 00:00 2026-02-24T18:18:09.189 INFO:teuthology.orchestra.run.trial038.stdout:(82/99): python3-more-itertools-8.12.0-2.el9.no 1.2 MB/s | 79 kB 00:00 2026-02-24T18:18:09.219 INFO:teuthology.orchestra.run.trial038.stdout:(83/99): python3-msgpack-1.0.3-2.el9.x86_64.rpm 2.4 MB/s | 86 kB 00:00 2026-02-24T18:18:09.222 INFO:teuthology.orchestra.run.trial038.stdout:(84/99): python3-natsort-7.1.1-5.el9.noarch.rpm 1.5 MB/s | 58 kB 00:00 2026-02-24T18:18:09.237 INFO:teuthology.orchestra.run.trial038.stdout:(85/99): python3-pecan-1.4.2-3.el9.noarch.rpm 5.6 MB/s | 272 kB 00:00 2026-02-24T18:18:09.250 INFO:teuthology.orchestra.run.trial038.stdout:(86/99): python3-portend-3.1.0-2.el9.noarch.rpm 525 kB/s | 16 kB 00:00 2026-02-24T18:18:09.258 INFO:teuthology.orchestra.run.trial038.stdout:(87/99): python3-pyOpenSSL-21.0.0-1.el9.noarch. 2.5 MB/s | 90 kB 00:00 2026-02-24T18:18:09.269 INFO:teuthology.orchestra.run.trial038.stdout:(88/99): python3-repoze-lru-0.7-16.el9.noarch.r 953 kB/s | 31 kB 00:00 2026-02-24T18:18:09.293 INFO:teuthology.orchestra.run.trial038.stdout:(89/99): python3-routes-2.5.1-5.el9.noarch.rpm 4.3 MB/s | 188 kB 00:00 2026-02-24T18:18:09.296 INFO:teuthology.orchestra.run.trial038.stdout:(90/99): python3-rsa-4.9-2.el9.noarch.rpm 1.5 MB/s | 59 kB 00:00 2026-02-24T18:18:09.307 INFO:teuthology.orchestra.run.trial038.stdout:(91/99): python3-saml-1.16.0-1.el9.noarch.rpm 3.2 MB/s | 125 kB 00:00 2026-02-24T18:18:09.326 INFO:teuthology.orchestra.run.trial038.stdout:(92/99): python3-tempora-5.0.0-2.el9.noarch.rpm 1.1 MB/s | 36 kB 00:00 2026-02-24T18:18:09.332 INFO:teuthology.orchestra.run.trial038.stdout:(93/99): python3-typing-extensions-4.15.0-1.el9 2.4 MB/s | 86 kB 00:00 2026-02-24T18:18:09.350 INFO:teuthology.orchestra.run.trial038.stdout:(94/99): python3-webob-1.8.8-2.el9.noarch.rpm 5.3 MB/s | 230 kB 00:00 2026-02-24T18:18:09.361 INFO:teuthology.orchestra.run.trial038.stdout:(95/99): python3-websocket-client-1.2.3-2.el9.n 2.5 MB/s | 90 kB 00:00 2026-02-24T18:18:09.382 INFO:teuthology.orchestra.run.trial038.stdout:(96/99): python3-werkzeug-2.0.3-3.el9.1.noarch. 8.2 MB/s | 427 kB 00:00 2026-02-24T18:18:09.385 INFO:teuthology.orchestra.run.trial038.stdout:(97/99): python3-xmlsec-1.3.13-1.el9.x86_64.rpm 1.4 MB/s | 48 kB 00:00 2026-02-24T18:18:09.393 INFO:teuthology.orchestra.run.trial038.stdout:(98/99): python3-xmltodict-0.12.0-15.el9.noarch 696 kB/s | 22 kB 00:00 2026-02-24T18:18:09.415 INFO:teuthology.orchestra.run.trial038.stdout:(99/99): python3-zc-lockfile-2.0-10.el9.noarch. 620 kB/s | 20 kB 00:00 2026-02-24T18:18:09.418 INFO:teuthology.orchestra.run.trial038.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:18:09.418 INFO:teuthology.orchestra.run.trial038.stdout:Total 19 MB/s | 95 MB 00:05 2026-02-24T18:18:09.868 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction check 2026-02-24T18:18:09.906 INFO:teuthology.orchestra.run.trial038.stdout:Transaction check succeeded. 2026-02-24T18:18:09.906 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction test 2026-02-24T18:18:10.325 INFO:teuthology.orchestra.run.trial038.stdout:Transaction test succeeded. 2026-02-24T18:18:10.325 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction 2026-02-24T18:18:10.783 INFO:teuthology.orchestra.run.trial038.stdout: Preparing : 1/1 2026-02-24T18:18:10.785 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/99 2026-02-24T18:18:10.816 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/99 2026-02-24T18:18:10.821 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 3/99 2026-02-24T18:18:10.843 INFO:teuthology.orchestra.run.trial038.stdout: Installing : xmlsec1-1.2.29-13.el9.x86_64 4/99 2026-02-24T18:18:10.849 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-pyasn1-0.4.8-7.el9.noarch 5/99 2026-02-24T18:18:10.882 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 6/99 2026-02-24T18:18:10.891 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-lxml-4.6.5-3.el9.x86_64 7/99 2026-02-24T18:18:10.897 INFO:teuthology.orchestra.run.trial038.stdout: Installing : protobuf-3.14.0-17.el9.x86_64 8/99 2026-02-24T18:18:10.899 INFO:teuthology.orchestra.run.trial038.stdout: Installing : lua-5.4.4-4.el9.x86_64 9/99 2026-02-24T18:18:10.904 INFO:teuthology.orchestra.run.trial038.stdout: Installing : flexiblas-3.0.4-9.el9.x86_64 10/99 2026-02-24T18:18:10.927 INFO:teuthology.orchestra.run.trial038.stdout: Installing : unzip-6.0-59.el9.x86_64 11/99 2026-02-24T18:18:10.939 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-urllib3-1.26.5-7.el9.noarch 12/99 2026-02-24T18:18:10.942 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-requests-2.25.1-10.el9.noarch 13/99 2026-02-24T18:18:10.949 INFO:teuthology.orchestra.run.trial038.stdout: Installing : libquadmath-11.5.0-14.el9.x86_64 14/99 2026-02-24T18:18:10.952 INFO:teuthology.orchestra.run.trial038.stdout: Installing : libgfortran-11.5.0-14.el9.x86_64 15/99 2026-02-24T18:18:10.964 INFO:teuthology.orchestra.run.trial038.stdout: Installing : ledmon-libs-1.1.0-3.el9.x86_64 16/99 2026-02-24T18:18:10.969 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 17/99 2026-02-24T18:18:10.994 INFO:teuthology.orchestra.run.trial038.stdout: Installing : zip-3.0-35.el9.x86_64 18/99 2026-02-24T18:18:10.997 INFO:teuthology.orchestra.run.trial038.stdout: Installing : luarocks-3.9.2-5.el9.noarch 19/99 2026-02-24T18:18:11.006 INFO:teuthology.orchestra.run.trial038.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 20/99 2026-02-24T18:18:11.031 INFO:teuthology.orchestra.run.trial038.stdout: Installing : protobuf-compiler-3.14.0-17.el9.x86_64 21/99 2026-02-24T18:18:11.082 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 22/99 2026-02-24T18:18:11.095 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-pyasn1-modules-0.4.8-7.el9.noarch 23/99 2026-02-24T18:18:11.099 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-rsa-4.9-2.el9.noarch 24/99 2026-02-24T18:18:11.104 INFO:teuthology.orchestra.run.trial038.stdout: Installing : xmlsec1-openssl-1.2.29-13.el9.x86_64 25/99 2026-02-24T18:18:11.109 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-xmlsec-1.3.13-1.el9.x86_64 26/99 2026-02-24T18:18:11.114 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 27/99 2026-02-24T18:18:11.117 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 28/99 2026-02-24T18:18:11.132 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-xmltodict-0.12.0-15.el9.noarch 29/99 2026-02-24T18:18:11.153 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 30/99 2026-02-24T18:18:11.157 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 31/99 2026-02-24T18:18:11.162 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-typing-extensions-4.15.0-1.el9.noarch 32/99 2026-02-24T18:18:11.174 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 33/99 2026-02-24T18:18:11.183 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 34/99 2026-02-24T18:18:11.189 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 35/99 2026-02-24T18:18:11.209 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-msgpack-1.0.3-2.el9.x86_64 36/99 2026-02-24T18:18:11.218 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-influxdb-5.3.1-1.el9.noarch 37/99 2026-02-24T18:18:11.273 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 38/99 2026-02-24T18:18:11.285 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 39/99 2026-02-24T18:18:11.301 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-isodate-0.6.1-3.el9.noarch 40/99 2026-02-24T18:18:11.306 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-saml-1.16.0-1.el9.noarch 41/99 2026-02-24T18:18:11.313 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 42/99 2026-02-24T18:18:11.354 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 43/99 2026-02-24T18:18:11.683 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-google-auth-1:2.45.0-1.el9.noarch 44/99 2026-02-24T18:18:11.693 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-kubernetes-1:26.1.0-3.el9.noarch 45/99 2026-02-24T18:18:11.697 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-backports-tarfile-1.2.0-1.el9.noarch 46/99 2026-02-24T18:18:11.703 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-jaraco-context-6.0.1-3.el9.noarch 47/99 2026-02-24T18:18:11.705 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-autocommand-2.2.2-8.el9.noarch 48/99 2026-02-24T18:18:11.733 INFO:teuthology.orchestra.run.trial038.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 49/99 2026-02-24T18:18:11.777 INFO:teuthology.orchestra.run.trial038.stdout: Installing : abseil-cpp-20211102.0-4.el9.x86_64 50/99 2026-02-24T18:18:11.784 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 51/99 2026-02-24T18:18:11.787 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 52/99 2026-02-24T18:18:11.793 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 53/99 2026-02-24T18:18:11.797 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-jaraco-text-4.0.0-2.el9.noarch 54/99 2026-02-24T18:18:11.804 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 55/99 2026-02-24T18:18:11.808 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 56/99 2026-02-24T18:18:11.836 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 57/99 2026-02-24T18:18:11.847 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-protobuf-3.14.0-17.el9.noarch 58/99 2026-02-24T18:18:11.883 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 59/99 2026-02-24T18:18:12.109 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-devel-3.9.25-3.el9.x86_64 60/99 2026-02-24T18:18:12.133 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 61/99 2026-02-24T18:18:12.137 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-jinja2-2.11.3-8.el9.noarch 62/99 2026-02-24T18:18:12.196 INFO:teuthology.orchestra.run.trial038.stdout: Installing : openblas-0.3.29-1.el9.x86_64 63/99 2026-02-24T18:18:12.198 INFO:teuthology.orchestra.run.trial038.stdout: Installing : openblas-openmp-0.3.29-1.el9.x86_64 64/99 2026-02-24T18:18:12.219 INFO:teuthology.orchestra.run.trial038.stdout: Installing : flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 65/99 2026-02-24T18:18:12.549 INFO:teuthology.orchestra.run.trial038.stdout: Installing : flexiblas-netlib-3.0.4-9.el9.x86_64 66/99 2026-02-24T18:18:12.622 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-numpy-1:1.23.5-2.el9.x86_64 67/99 2026-02-24T18:18:13.302 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 68/99 2026-02-24T18:18:13.318 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-scipy-1.9.3-2.el9.x86_64 69/99 2026-02-24T18:18:13.328 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-pyparsing-2.4.7-9.el9.noarch 70/99 2026-02-24T18:18:13.335 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-packaging-20.9-5.el9.noarch 71/99 2026-02-24T18:18:13.348 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-ply-3.11-14.el9.noarch 72/99 2026-02-24T18:18:13.364 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 73/99 2026-02-24T18:18:13.441 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 74/99 2026-02-24T18:18:13.450 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-cryptography-36.0.1-5.el9.x86_64 75/99 2026-02-24T18:18:13.473 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 76/99 2026-02-24T18:18:13.503 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-cheroot-10.0.1-4.el9.noarch 77/99 2026-02-24T18:18:13.554 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 78/99 2026-02-24T18:18:13.560 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 79/99 2026-02-24T18:18:13.566 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 80/99 2026-02-24T18:18:13.567 INFO:teuthology.orchestra.run.trial038.stdout: Installing : libconfig-1.7.2-9.el9.x86_64 81/99 2026-02-24T18:18:13.586 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 82/99 2026-02-24T18:18:13.586 INFO:teuthology.orchestra.run.trial038.stdout:Creating group 'libstoragemgmt' with GID 991. 2026-02-24T18:18:13.586 INFO:teuthology.orchestra.run.trial038.stdout:Creating user 'libstoragemgmt' (daemon account for libstoragemgmt) with UID 991 and GID 991. 2026-02-24T18:18:13.586 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:18:13.596 INFO:teuthology.orchestra.run.trial038.stdout: Installing : libstoragemgmt-1.10.1-1.el9.x86_64 82/99 2026-02-24T18:18:13.622 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 82/99 2026-02-24T18:18:13.622 INFO:teuthology.orchestra.run.trial038.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/libstoragemgmt.service → /usr/lib/systemd/system/libstoragemgmt.service. 2026-02-24T18:18:13.622 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:18:13.656 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-libstoragemgmt-1.10.1-1.el9.x86_64 83/99 2026-02-24T18:18:13.720 INFO:teuthology.orchestra.run.trial038.stdout: Installing : cryptsetup-2.8.1-3.el9.x86_64 84/99 2026-02-24T18:18:13.722 INFO:teuthology.orchestra.run.trial038.stdout: Installing : ceph-volume-2:19.2.3-689.gf90f4eae.el9.noarch 85/99 2026-02-24T18:18:13.736 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: ceph-volume-2:19.2.3-689.gf90f4eae.el9.noarch 85/99 2026-02-24T18:18:13.736 INFO:teuthology.orchestra.run.trial038.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-24T18:18:13.736 INFO:teuthology.orchestra.run.trial038.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2026-02-24T18:18:13.736 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:18:14.433 INFO:teuthology.orchestra.run.trial038.stdout: Installing : ceph-osd-2:19.2.3-689.gf90f4eae.el9.x86_64 86/99 2026-02-24T18:18:14.459 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: ceph-osd-2:19.2.3-689.gf90f4eae.el9.x86_64 86/99 2026-02-24T18:18:14.459 INFO:teuthology.orchestra.run.trial038.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-24T18:18:14.459 INFO:teuthology.orchestra.run.trial038.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2026-02-24T18:18:14.459 INFO:teuthology.orchestra.run.trial038.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-24T18:18:14.459 INFO:teuthology.orchestra.run.trial038.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-24T18:18:14.459 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:18:14.516 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: cephadm-2:19.2.3-689.gf90f4eae.el9.noarch 87/99 2026-02-24T18:18:14.518 INFO:teuthology.orchestra.run.trial038.stdout: Installing : cephadm-2:19.2.3-689.gf90f4eae.el9.noarch 87/99 2026-02-24T18:18:14.523 INFO:teuthology.orchestra.run.trial038.stdout: Installing : ceph-prometheus-alerts-2:19.2.3-689.gf90f4eae.el9. 88/99 2026-02-24T18:18:14.543 INFO:teuthology.orchestra.run.trial038.stdout: Installing : ceph-grafana-dashboards-2:19.2.3-689.gf90f4eae.el9 89/99 2026-02-24T18:18:14.544 INFO:teuthology.orchestra.run.trial038.stdout: Installing : ceph-mgr-cephadm-2:19.2.3-689.gf90f4eae.el9.noarch 90/99 2026-02-24T18:18:15.038 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.2.3-689.gf90f4eae.el9.noarch 90/99 2026-02-24T18:18:15.042 INFO:teuthology.orchestra.run.trial038.stdout: Installing : ceph-mgr-dashboard-2:19.2.3-689.gf90f4eae.el9.noar 91/99 2026-02-24T18:18:15.495 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.2.3-689.gf90f4eae.el9.noar 91/99 2026-02-24T18:18:15.496 INFO:teuthology.orchestra.run.trial038.stdout: Installing : ceph-mgr-diskprediction-local-2:19.2.3-689.gf90f4e 92/99 2026-02-24T18:18:15.510 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.2.3-689.gf90f4e 92/99 2026-02-24T18:18:15.510 INFO:teuthology.orchestra.run.trial038.stdout: Installing : ceph-mgr-k8sevents-2:19.2.3-689.gf90f4eae.el9.noar 93/99 2026-02-24T18:18:15.566 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: ceph-mgr-k8sevents-2:19.2.3-689.gf90f4eae.el9.noar 93/99 2026-02-24T18:18:15.615 INFO:teuthology.orchestra.run.trial038.stdout: Installing : ceph-mgr-modules-core-2:19.2.3-689.gf90f4eae.el9.n 94/99 2026-02-24T18:18:15.616 INFO:teuthology.orchestra.run.trial038.stdout: Installing : ceph-mgr-2:19.2.3-689.gf90f4eae.el9.x86_64 95/99 2026-02-24T18:18:15.636 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: ceph-mgr-2:19.2.3-689.gf90f4eae.el9.x86_64 95/99 2026-02-24T18:18:15.636 INFO:teuthology.orchestra.run.trial038.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-24T18:18:15.637 INFO:teuthology.orchestra.run.trial038.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2026-02-24T18:18:15.637 INFO:teuthology.orchestra.run.trial038.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-24T18:18:15.637 INFO:teuthology.orchestra.run.trial038.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-24T18:18:15.637 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:18:15.648 INFO:teuthology.orchestra.run.trial038.stdout: Installing : ceph-mgr-rook-2:19.2.3-689.gf90f4eae.el9.noarch 96/99 2026-02-24T18:18:15.864 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: ceph-mgr-rook-2:19.2.3-689.gf90f4eae.el9.noarch 96/99 2026-02-24T18:18:15.866 INFO:teuthology.orchestra.run.trial038.stdout: Installing : ceph-mon-2:19.2.3-689.gf90f4eae.el9.x86_64 97/99 2026-02-24T18:18:15.887 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: ceph-mon-2:19.2.3-689.gf90f4eae.el9.x86_64 97/99 2026-02-24T18:18:15.887 INFO:teuthology.orchestra.run.trial038.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-24T18:18:15.887 INFO:teuthology.orchestra.run.trial038.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2026-02-24T18:18:15.888 INFO:teuthology.orchestra.run.trial038.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-24T18:18:15.888 INFO:teuthology.orchestra.run.trial038.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-24T18:18:15.888 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:18:15.987 INFO:teuthology.orchestra.run.trial038.stdout: Installing : ceph-mds-2:19.2.3-689.gf90f4eae.el9.x86_64 98/99 2026-02-24T18:18:16.008 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: ceph-mds-2:19.2.3-689.gf90f4eae.el9.x86_64 98/99 2026-02-24T18:18:16.008 INFO:teuthology.orchestra.run.trial038.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-24T18:18:16.008 INFO:teuthology.orchestra.run.trial038.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2026-02-24T18:18:16.008 INFO:teuthology.orchestra.run.trial038.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-24T18:18:16.008 INFO:teuthology.orchestra.run.trial038.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-24T18:18:16.008 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:18:16.027 INFO:teuthology.orchestra.run.trial038.stdout: Installing : ceph-2:19.2.3-689.gf90f4eae.el9.x86_64 99/99 2026-02-24T18:18:16.702 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: ceph-2:19.2.3-689.gf90f4eae.el9.x86_64 99/99 2026-02-24T18:18:16.702 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : ceph-2:19.2.3-689.gf90f4eae.el9.x86_64 1/99 2026-02-24T18:18:16.702 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : ceph-mds-2:19.2.3-689.gf90f4eae.el9.x86_64 2/99 2026-02-24T18:18:16.702 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : ceph-mgr-2:19.2.3-689.gf90f4eae.el9.x86_64 3/99 2026-02-24T18:18:16.702 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : ceph-mon-2:19.2.3-689.gf90f4eae.el9.x86_64 4/99 2026-02-24T18:18:16.702 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : ceph-osd-2:19.2.3-689.gf90f4eae.el9.x86_64 5/99 2026-02-24T18:18:16.702 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : ceph-grafana-dashboards-2:19.2.3-689.gf90f4eae.el9 6/99 2026-02-24T18:18:16.702 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : ceph-mgr-cephadm-2:19.2.3-689.gf90f4eae.el9.noarch 7/99 2026-02-24T18:18:16.702 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : ceph-mgr-dashboard-2:19.2.3-689.gf90f4eae.el9.noar 8/99 2026-02-24T18:18:16.703 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.2.3-689.gf90f4e 9/99 2026-02-24T18:18:16.703 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : ceph-mgr-k8sevents-2:19.2.3-689.gf90f4eae.el9.noar 10/99 2026-02-24T18:18:16.703 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : ceph-mgr-modules-core-2:19.2.3-689.gf90f4eae.el9.n 11/99 2026-02-24T18:18:16.703 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : ceph-mgr-rook-2:19.2.3-689.gf90f4eae.el9.noarch 12/99 2026-02-24T18:18:16.703 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : ceph-prometheus-alerts-2:19.2.3-689.gf90f4eae.el9. 13/99 2026-02-24T18:18:16.703 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : ceph-volume-2:19.2.3-689.gf90f4eae.el9.noarch 14/99 2026-02-24T18:18:16.703 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : cephadm-2:19.2.3-689.gf90f4eae.el9.noarch 15/99 2026-02-24T18:18:16.703 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : cryptsetup-2.8.1-3.el9.x86_64 16/99 2026-02-24T18:18:16.703 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : ledmon-libs-1.1.0-3.el9.x86_64 17/99 2026-02-24T18:18:16.703 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : libconfig-1.7.2-9.el9.x86_64 18/99 2026-02-24T18:18:16.703 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : libgfortran-11.5.0-14.el9.x86_64 19/99 2026-02-24T18:18:16.703 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : libquadmath-11.5.0-14.el9.x86_64 20/99 2026-02-24T18:18:16.703 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 21/99 2026-02-24T18:18:16.703 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-cryptography-36.0.1-5.el9.x86_64 22/99 2026-02-24T18:18:16.703 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-ply-3.11-14.el9.noarch 23/99 2026-02-24T18:18:16.703 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 24/99 2026-02-24T18:18:16.703 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-pyparsing-2.4.7-9.el9.noarch 25/99 2026-02-24T18:18:16.703 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-requests-2.25.1-10.el9.noarch 26/99 2026-02-24T18:18:16.703 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-urllib3-1.26.5-7.el9.noarch 27/99 2026-02-24T18:18:16.703 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : unzip-6.0-59.el9.x86_64 28/99 2026-02-24T18:18:16.703 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : zip-3.0-35.el9.x86_64 29/99 2026-02-24T18:18:16.703 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : flexiblas-3.0.4-9.el9.x86_64 30/99 2026-02-24T18:18:16.703 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : flexiblas-netlib-3.0.4-9.el9.x86_64 31/99 2026-02-24T18:18:16.703 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 32/99 2026-02-24T18:18:16.703 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : libstoragemgmt-1.10.1-1.el9.x86_64 33/99 2026-02-24T18:18:16.704 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : lua-5.4.4-4.el9.x86_64 34/99 2026-02-24T18:18:16.704 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : openblas-0.3.29-1.el9.x86_64 35/99 2026-02-24T18:18:16.704 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : openblas-openmp-0.3.29-1.el9.x86_64 36/99 2026-02-24T18:18:16.704 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : protobuf-3.14.0-17.el9.x86_64 37/99 2026-02-24T18:18:16.704 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 38/99 2026-02-24T18:18:16.704 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-devel-3.9.25-3.el9.x86_64 39/99 2026-02-24T18:18:16.704 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-jinja2-2.11.3-8.el9.noarch 40/99 2026-02-24T18:18:16.704 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-libstoragemgmt-1.10.1-1.el9.x86_64 41/99 2026-02-24T18:18:16.704 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-lxml-4.6.5-3.el9.x86_64 42/99 2026-02-24T18:18:16.704 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 43/99 2026-02-24T18:18:16.704 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 44/99 2026-02-24T18:18:16.704 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-numpy-1:1.23.5-2.el9.x86_64 45/99 2026-02-24T18:18:16.704 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 46/99 2026-02-24T18:18:16.704 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 47/99 2026-02-24T18:18:16.704 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-protobuf-3.14.0-17.el9.noarch 48/99 2026-02-24T18:18:16.704 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-pyasn1-0.4.8-7.el9.noarch 49/99 2026-02-24T18:18:16.704 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-pyasn1-modules-0.4.8-7.el9.noarch 50/99 2026-02-24T18:18:16.704 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 51/99 2026-02-24T18:18:16.704 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-scipy-1.9.3-2.el9.x86_64 52/99 2026-02-24T18:18:16.704 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 53/99 2026-02-24T18:18:16.704 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : xmlsec1-1.2.29-13.el9.x86_64 54/99 2026-02-24T18:18:16.704 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : xmlsec1-openssl-1.2.29-13.el9.x86_64 55/99 2026-02-24T18:18:16.704 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 56/99 2026-02-24T18:18:16.704 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : protobuf-compiler-3.14.0-17.el9.x86_64 57/99 2026-02-24T18:18:16.704 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : abseil-cpp-20211102.0-4.el9.x86_64 58/99 2026-02-24T18:18:16.705 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 59/99 2026-02-24T18:18:16.705 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 60/99 2026-02-24T18:18:16.705 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 61/99 2026-02-24T18:18:16.705 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-autocommand-2.2.2-8.el9.noarch 62/99 2026-02-24T18:18:16.705 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-backports-tarfile-1.2.0-1.el9.noarch 63/99 2026-02-24T18:18:16.705 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 64/99 2026-02-24T18:18:16.705 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 65/99 2026-02-24T18:18:16.705 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 66/99 2026-02-24T18:18:16.705 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-cheroot-10.0.1-4.el9.noarch 67/99 2026-02-24T18:18:16.705 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 68/99 2026-02-24T18:18:16.705 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-google-auth-1:2.45.0-1.el9.noarch 69/99 2026-02-24T18:18:16.705 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 70/99 2026-02-24T18:18:16.705 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 71/99 2026-02-24T18:18:16.705 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-influxdb-5.3.1-1.el9.noarch 72/99 2026-02-24T18:18:16.705 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-isodate-0.6.1-3.el9.noarch 73/99 2026-02-24T18:18:16.705 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 74/99 2026-02-24T18:18:16.705 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 75/99 2026-02-24T18:18:16.705 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 76/99 2026-02-24T18:18:16.705 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-jaraco-context-6.0.1-3.el9.noarch 77/99 2026-02-24T18:18:16.705 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 78/99 2026-02-24T18:18:16.705 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-jaraco-text-4.0.0-2.el9.noarch 79/99 2026-02-24T18:18:16.705 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-kubernetes-1:26.1.0-3.el9.noarch 80/99 2026-02-24T18:18:16.705 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 81/99 2026-02-24T18:18:16.705 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 82/99 2026-02-24T18:18:16.706 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-msgpack-1.0.3-2.el9.x86_64 83/99 2026-02-24T18:18:16.706 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 84/99 2026-02-24T18:18:16.706 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 85/99 2026-02-24T18:18:16.706 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 86/99 2026-02-24T18:18:16.706 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 87/99 2026-02-24T18:18:16.706 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 88/99 2026-02-24T18:18:16.706 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 89/99 2026-02-24T18:18:16.706 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 90/99 2026-02-24T18:18:16.706 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-saml-1.16.0-1.el9.noarch 91/99 2026-02-24T18:18:16.706 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 92/99 2026-02-24T18:18:16.706 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-typing-extensions-4.15.0-1.el9.noarch 93/99 2026-02-24T18:18:16.706 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 94/99 2026-02-24T18:18:16.706 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 95/99 2026-02-24T18:18:16.706 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 96/99 2026-02-24T18:18:16.706 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-xmlsec-1.3.13-1.el9.x86_64 97/99 2026-02-24T18:18:16.706 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-xmltodict-0.12.0-15.el9.noarch 98/99 2026-02-24T18:18:16.776 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 99/99 2026-02-24T18:18:16.776 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:18:16.776 INFO:teuthology.orchestra.run.trial038.stdout:Installed: 2026-02-24T18:18:16.776 INFO:teuthology.orchestra.run.trial038.stdout: abseil-cpp-20211102.0-4.el9.x86_64 2026-02-24T18:18:16.776 INFO:teuthology.orchestra.run.trial038.stdout: ceph-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:18:16.776 INFO:teuthology.orchestra.run.trial038.stdout: ceph-grafana-dashboards-2:19.2.3-689.gf90f4eae.el9.noarch 2026-02-24T18:18:16.777 INFO:teuthology.orchestra.run.trial038.stdout: ceph-mds-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:18:16.777 INFO:teuthology.orchestra.run.trial038.stdout: ceph-mgr-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:18:16.777 INFO:teuthology.orchestra.run.trial038.stdout: ceph-mgr-cephadm-2:19.2.3-689.gf90f4eae.el9.noarch 2026-02-24T18:18:16.777 INFO:teuthology.orchestra.run.trial038.stdout: ceph-mgr-dashboard-2:19.2.3-689.gf90f4eae.el9.noarch 2026-02-24T18:18:16.777 INFO:teuthology.orchestra.run.trial038.stdout: ceph-mgr-diskprediction-local-2:19.2.3-689.gf90f4eae.el9.noarch 2026-02-24T18:18:16.777 INFO:teuthology.orchestra.run.trial038.stdout: ceph-mgr-k8sevents-2:19.2.3-689.gf90f4eae.el9.noarch 2026-02-24T18:18:16.777 INFO:teuthology.orchestra.run.trial038.stdout: ceph-mgr-modules-core-2:19.2.3-689.gf90f4eae.el9.noarch 2026-02-24T18:18:16.777 INFO:teuthology.orchestra.run.trial038.stdout: ceph-mgr-rook-2:19.2.3-689.gf90f4eae.el9.noarch 2026-02-24T18:18:16.777 INFO:teuthology.orchestra.run.trial038.stdout: ceph-mon-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:18:16.777 INFO:teuthology.orchestra.run.trial038.stdout: ceph-osd-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:18:16.777 INFO:teuthology.orchestra.run.trial038.stdout: ceph-prometheus-alerts-2:19.2.3-689.gf90f4eae.el9.noarch 2026-02-24T18:18:16.777 INFO:teuthology.orchestra.run.trial038.stdout: ceph-volume-2:19.2.3-689.gf90f4eae.el9.noarch 2026-02-24T18:18:16.777 INFO:teuthology.orchestra.run.trial038.stdout: cephadm-2:19.2.3-689.gf90f4eae.el9.noarch 2026-02-24T18:18:16.777 INFO:teuthology.orchestra.run.trial038.stdout: cryptsetup-2.8.1-3.el9.x86_64 2026-02-24T18:18:16.777 INFO:teuthology.orchestra.run.trial038.stdout: flexiblas-3.0.4-9.el9.x86_64 2026-02-24T18:18:16.777 INFO:teuthology.orchestra.run.trial038.stdout: flexiblas-netlib-3.0.4-9.el9.x86_64 2026-02-24T18:18:16.777 INFO:teuthology.orchestra.run.trial038.stdout: flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 2026-02-24T18:18:16.777 INFO:teuthology.orchestra.run.trial038.stdout: grpc-data-1.46.7-10.el9.noarch 2026-02-24T18:18:16.777 INFO:teuthology.orchestra.run.trial038.stdout: ledmon-libs-1.1.0-3.el9.x86_64 2026-02-24T18:18:16.777 INFO:teuthology.orchestra.run.trial038.stdout: libconfig-1.7.2-9.el9.x86_64 2026-02-24T18:18:16.777 INFO:teuthology.orchestra.run.trial038.stdout: libgfortran-11.5.0-14.el9.x86_64 2026-02-24T18:18:16.777 INFO:teuthology.orchestra.run.trial038.stdout: libquadmath-11.5.0-14.el9.x86_64 2026-02-24T18:18:16.777 INFO:teuthology.orchestra.run.trial038.stdout: libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-24T18:18:16.777 INFO:teuthology.orchestra.run.trial038.stdout: lua-5.4.4-4.el9.x86_64 2026-02-24T18:18:16.778 INFO:teuthology.orchestra.run.trial038.stdout: lua-devel-5.4.4-4.el9.x86_64 2026-02-24T18:18:16.778 INFO:teuthology.orchestra.run.trial038.stdout: luarocks-3.9.2-5.el9.noarch 2026-02-24T18:18:16.778 INFO:teuthology.orchestra.run.trial038.stdout: openblas-0.3.29-1.el9.x86_64 2026-02-24T18:18:16.778 INFO:teuthology.orchestra.run.trial038.stdout: openblas-openmp-0.3.29-1.el9.x86_64 2026-02-24T18:18:16.778 INFO:teuthology.orchestra.run.trial038.stdout: protobuf-3.14.0-17.el9.x86_64 2026-02-24T18:18:16.778 INFO:teuthology.orchestra.run.trial038.stdout: protobuf-compiler-3.14.0-17.el9.x86_64 2026-02-24T18:18:16.778 INFO:teuthology.orchestra.run.trial038.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2026-02-24T18:18:16.778 INFO:teuthology.orchestra.run.trial038.stdout: python3-autocommand-2.2.2-8.el9.noarch 2026-02-24T18:18:16.778 INFO:teuthology.orchestra.run.trial038.stdout: python3-babel-2.9.1-2.el9.noarch 2026-02-24T18:18:16.778 INFO:teuthology.orchestra.run.trial038.stdout: python3-backports-tarfile-1.2.0-1.el9.noarch 2026-02-24T18:18:16.778 INFO:teuthology.orchestra.run.trial038.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2026-02-24T18:18:16.778 INFO:teuthology.orchestra.run.trial038.stdout: python3-cachetools-4.2.4-1.el9.noarch 2026-02-24T18:18:16.778 INFO:teuthology.orchestra.run.trial038.stdout: python3-certifi-2023.05.07-4.el9.noarch 2026-02-24T18:18:16.778 INFO:teuthology.orchestra.run.trial038.stdout: python3-cffi-1.14.5-5.el9.x86_64 2026-02-24T18:18:16.778 INFO:teuthology.orchestra.run.trial038.stdout: python3-cheroot-10.0.1-4.el9.noarch 2026-02-24T18:18:16.778 INFO:teuthology.orchestra.run.trial038.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2026-02-24T18:18:16.778 INFO:teuthology.orchestra.run.trial038.stdout: python3-cryptography-36.0.1-5.el9.x86_64 2026-02-24T18:18:16.778 INFO:teuthology.orchestra.run.trial038.stdout: python3-devel-3.9.25-3.el9.x86_64 2026-02-24T18:18:16.778 INFO:teuthology.orchestra.run.trial038.stdout: python3-google-auth-1:2.45.0-1.el9.noarch 2026-02-24T18:18:16.778 INFO:teuthology.orchestra.run.trial038.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2026-02-24T18:18:16.778 INFO:teuthology.orchestra.run.trial038.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2026-02-24T18:18:16.778 INFO:teuthology.orchestra.run.trial038.stdout: python3-influxdb-5.3.1-1.el9.noarch 2026-02-24T18:18:16.778 INFO:teuthology.orchestra.run.trial038.stdout: python3-isodate-0.6.1-3.el9.noarch 2026-02-24T18:18:16.778 INFO:teuthology.orchestra.run.trial038.stdout: python3-jaraco-8.2.1-3.el9.noarch 2026-02-24T18:18:16.779 INFO:teuthology.orchestra.run.trial038.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2026-02-24T18:18:16.779 INFO:teuthology.orchestra.run.trial038.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2026-02-24T18:18:16.779 INFO:teuthology.orchestra.run.trial038.stdout: python3-jaraco-context-6.0.1-3.el9.noarch 2026-02-24T18:18:16.779 INFO:teuthology.orchestra.run.trial038.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2026-02-24T18:18:16.779 INFO:teuthology.orchestra.run.trial038.stdout: python3-jaraco-text-4.0.0-2.el9.noarch 2026-02-24T18:18:16.779 INFO:teuthology.orchestra.run.trial038.stdout: python3-jinja2-2.11.3-8.el9.noarch 2026-02-24T18:18:16.779 INFO:teuthology.orchestra.run.trial038.stdout: python3-kubernetes-1:26.1.0-3.el9.noarch 2026-02-24T18:18:16.779 INFO:teuthology.orchestra.run.trial038.stdout: python3-libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-24T18:18:16.779 INFO:teuthology.orchestra.run.trial038.stdout: python3-logutils-0.3.5-21.el9.noarch 2026-02-24T18:18:16.779 INFO:teuthology.orchestra.run.trial038.stdout: python3-lxml-4.6.5-3.el9.x86_64 2026-02-24T18:18:16.779 INFO:teuthology.orchestra.run.trial038.stdout: python3-mako-1.1.4-6.el9.noarch 2026-02-24T18:18:16.779 INFO:teuthology.orchestra.run.trial038.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2026-02-24T18:18:16.779 INFO:teuthology.orchestra.run.trial038.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2026-02-24T18:18:16.779 INFO:teuthology.orchestra.run.trial038.stdout: python3-msgpack-1.0.3-2.el9.x86_64 2026-02-24T18:18:16.779 INFO:teuthology.orchestra.run.trial038.stdout: python3-natsort-7.1.1-5.el9.noarch 2026-02-24T18:18:16.779 INFO:teuthology.orchestra.run.trial038.stdout: python3-numpy-1:1.23.5-2.el9.x86_64 2026-02-24T18:18:16.779 INFO:teuthology.orchestra.run.trial038.stdout: python3-numpy-f2py-1:1.23.5-2.el9.x86_64 2026-02-24T18:18:16.779 INFO:teuthology.orchestra.run.trial038.stdout: python3-packaging-20.9-5.el9.noarch 2026-02-24T18:18:16.779 INFO:teuthology.orchestra.run.trial038.stdout: python3-pecan-1.4.2-3.el9.noarch 2026-02-24T18:18:16.779 INFO:teuthology.orchestra.run.trial038.stdout: python3-ply-3.11-14.el9.noarch 2026-02-24T18:18:16.779 INFO:teuthology.orchestra.run.trial038.stdout: python3-portend-3.1.0-2.el9.noarch 2026-02-24T18:18:16.779 INFO:teuthology.orchestra.run.trial038.stdout: python3-protobuf-3.14.0-17.el9.noarch 2026-02-24T18:18:16.779 INFO:teuthology.orchestra.run.trial038.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2026-02-24T18:18:16.779 INFO:teuthology.orchestra.run.trial038.stdout: python3-pyasn1-0.4.8-7.el9.noarch 2026-02-24T18:18:16.780 INFO:teuthology.orchestra.run.trial038.stdout: python3-pyasn1-modules-0.4.8-7.el9.noarch 2026-02-24T18:18:16.780 INFO:teuthology.orchestra.run.trial038.stdout: python3-pycparser-2.20-6.el9.noarch 2026-02-24T18:18:16.780 INFO:teuthology.orchestra.run.trial038.stdout: python3-pyparsing-2.4.7-9.el9.noarch 2026-02-24T18:18:16.780 INFO:teuthology.orchestra.run.trial038.stdout: python3-repoze-lru-0.7-16.el9.noarch 2026-02-24T18:18:16.780 INFO:teuthology.orchestra.run.trial038.stdout: python3-requests-2.25.1-10.el9.noarch 2026-02-24T18:18:16.780 INFO:teuthology.orchestra.run.trial038.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2026-02-24T18:18:16.780 INFO:teuthology.orchestra.run.trial038.stdout: python3-routes-2.5.1-5.el9.noarch 2026-02-24T18:18:16.780 INFO:teuthology.orchestra.run.trial038.stdout: python3-rsa-4.9-2.el9.noarch 2026-02-24T18:18:16.780 INFO:teuthology.orchestra.run.trial038.stdout: python3-saml-1.16.0-1.el9.noarch 2026-02-24T18:18:16.780 INFO:teuthology.orchestra.run.trial038.stdout: python3-scipy-1.9.3-2.el9.x86_64 2026-02-24T18:18:16.780 INFO:teuthology.orchestra.run.trial038.stdout: python3-tempora-5.0.0-2.el9.noarch 2026-02-24T18:18:16.780 INFO:teuthology.orchestra.run.trial038.stdout: python3-toml-0.10.2-6.el9.noarch 2026-02-24T18:18:16.780 INFO:teuthology.orchestra.run.trial038.stdout: python3-typing-extensions-4.15.0-1.el9.noarch 2026-02-24T18:18:16.780 INFO:teuthology.orchestra.run.trial038.stdout: python3-urllib3-1.26.5-7.el9.noarch 2026-02-24T18:18:16.780 INFO:teuthology.orchestra.run.trial038.stdout: python3-webob-1.8.8-2.el9.noarch 2026-02-24T18:18:16.780 INFO:teuthology.orchestra.run.trial038.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2026-02-24T18:18:16.780 INFO:teuthology.orchestra.run.trial038.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2026-02-24T18:18:16.780 INFO:teuthology.orchestra.run.trial038.stdout: python3-xmlsec-1.3.13-1.el9.x86_64 2026-02-24T18:18:16.780 INFO:teuthology.orchestra.run.trial038.stdout: python3-xmltodict-0.12.0-15.el9.noarch 2026-02-24T18:18:16.780 INFO:teuthology.orchestra.run.trial038.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2026-02-24T18:18:16.780 INFO:teuthology.orchestra.run.trial038.stdout: unzip-6.0-59.el9.x86_64 2026-02-24T18:18:16.780 INFO:teuthology.orchestra.run.trial038.stdout: xmlsec1-1.2.29-13.el9.x86_64 2026-02-24T18:18:16.780 INFO:teuthology.orchestra.run.trial038.stdout: xmlsec1-openssl-1.2.29-13.el9.x86_64 2026-02-24T18:18:16.780 INFO:teuthology.orchestra.run.trial038.stdout: zip-3.0-35.el9.x86_64 2026-02-24T18:18:16.780 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:18:16.781 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-24T18:18:16.834 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install ceph-base 2026-02-24T18:18:17.155 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:00:58 ago on Tue 24 Feb 2026 06:17:19 PM UTC. 2026-02-24T18:18:17.220 INFO:teuthology.orchestra.run.trial038.stdout:Package ceph-base-2:19.2.3-689.gf90f4eae.el9.x86_64 is already installed. 2026-02-24T18:18:17.239 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-24T18:18:17.240 INFO:teuthology.orchestra.run.trial038.stdout:Nothing to do. 2026-02-24T18:18:17.240 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-24T18:18:17.261 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install cephadm 2026-02-24T18:18:17.576 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:00:58 ago on Tue 24 Feb 2026 06:17:19 PM UTC. 2026-02-24T18:18:17.641 INFO:teuthology.orchestra.run.trial038.stdout:Package cephadm-2:19.2.3-689.gf90f4eae.el9.noarch is already installed. 2026-02-24T18:18:17.659 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-24T18:18:17.660 INFO:teuthology.orchestra.run.trial038.stdout:Nothing to do. 2026-02-24T18:18:17.660 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-24T18:18:17.681 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install ceph-immutable-object-cache 2026-02-24T18:18:18.018 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:00:59 ago on Tue 24 Feb 2026 06:17:19 PM UTC. 2026-02-24T18:18:18.103 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-24T18:18:18.103 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-24T18:18:18.104 INFO:teuthology.orchestra.run.trial038.stdout: Package Arch Version Repo Size 2026-02-24T18:18:18.104 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-24T18:18:18.104 INFO:teuthology.orchestra.run.trial038.stdout:Installing: 2026-02-24T18:18:18.104 INFO:teuthology.orchestra.run.trial038.stdout: ceph-immutable-object-cache x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 145 k 2026-02-24T18:18:18.104 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:18:18.104 INFO:teuthology.orchestra.run.trial038.stdout:Transaction Summary 2026-02-24T18:18:18.105 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-24T18:18:18.105 INFO:teuthology.orchestra.run.trial038.stdout:Install 1 Package 2026-02-24T18:18:18.105 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:18:18.105 INFO:teuthology.orchestra.run.trial038.stdout:Total download size: 145 k 2026-02-24T18:18:18.105 INFO:teuthology.orchestra.run.trial038.stdout:Installed size: 431 k 2026-02-24T18:18:18.105 INFO:teuthology.orchestra.run.trial038.stdout:Downloading Packages: 2026-02-24T18:18:18.583 INFO:teuthology.orchestra.run.trial038.stdout:ceph-immutable-object-cache-19.2.3-689.gf90f4ea 303 kB/s | 145 kB 00:00 2026-02-24T18:18:18.584 INFO:teuthology.orchestra.run.trial038.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:18:18.584 INFO:teuthology.orchestra.run.trial038.stdout:Total 302 kB/s | 145 kB 00:00 2026-02-24T18:18:18.584 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction check 2026-02-24T18:18:18.588 INFO:teuthology.orchestra.run.trial038.stdout:Transaction check succeeded. 2026-02-24T18:18:18.588 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction test 2026-02-24T18:18:18.620 INFO:teuthology.orchestra.run.trial038.stdout:Transaction test succeeded. 2026-02-24T18:18:18.620 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction 2026-02-24T18:18:18.676 INFO:teuthology.orchestra.run.trial038.stdout: Preparing : 1/1 2026-02-24T18:18:18.677 INFO:teuthology.orchestra.run.trial038.stdout: Installing : ceph-immutable-object-cache-2:19.2.3-689.gf90f4eae.e 1/1 2026-02-24T18:18:18.699 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.2.3-689.gf90f4eae.e 1/1 2026-02-24T18:18:18.699 INFO:teuthology.orchestra.run.trial038.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-24T18:18:18.700 INFO:teuthology.orchestra.run.trial038.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2026-02-24T18:18:18.700 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:18:19.029 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : ceph-immutable-object-cache-2:19.2.3-689.gf90f4eae.e 1/1 2026-02-24T18:18:19.029 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:18:19.029 INFO:teuthology.orchestra.run.trial038.stdout:Installed: 2026-02-24T18:18:19.029 INFO:teuthology.orchestra.run.trial038.stdout: ceph-immutable-object-cache-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:18:19.029 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:18:19.029 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-24T18:18:19.057 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install ceph-mgr 2026-02-24T18:18:19.375 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:01:00 ago on Tue 24 Feb 2026 06:17:19 PM UTC. 2026-02-24T18:18:19.440 INFO:teuthology.orchestra.run.trial038.stdout:Package ceph-mgr-2:19.2.3-689.gf90f4eae.el9.x86_64 is already installed. 2026-02-24T18:18:19.459 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-24T18:18:19.460 INFO:teuthology.orchestra.run.trial038.stdout:Nothing to do. 2026-02-24T18:18:19.460 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-24T18:18:19.481 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install ceph-mgr-dashboard 2026-02-24T18:18:19.811 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:01:00 ago on Tue 24 Feb 2026 06:17:19 PM UTC. 2026-02-24T18:18:19.876 INFO:teuthology.orchestra.run.trial038.stdout:Package ceph-mgr-dashboard-2:19.2.3-689.gf90f4eae.el9.noarch is already installed. 2026-02-24T18:18:19.895 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-24T18:18:19.895 INFO:teuthology.orchestra.run.trial038.stdout:Nothing to do. 2026-02-24T18:18:19.895 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-24T18:18:19.917 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install ceph-mgr-diskprediction-local 2026-02-24T18:18:20.226 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:01:01 ago on Tue 24 Feb 2026 06:17:19 PM UTC. 2026-02-24T18:18:20.292 INFO:teuthology.orchestra.run.trial038.stdout:Package ceph-mgr-diskprediction-local-2:19.2.3-689.gf90f4eae.el9.noarch is already installed. 2026-02-24T18:18:20.310 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-24T18:18:20.311 INFO:teuthology.orchestra.run.trial038.stdout:Nothing to do. 2026-02-24T18:18:20.311 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-24T18:18:20.331 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install ceph-mgr-rook 2026-02-24T18:18:20.648 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:01:01 ago on Tue 24 Feb 2026 06:17:19 PM UTC. 2026-02-24T18:18:20.716 INFO:teuthology.orchestra.run.trial038.stdout:Package ceph-mgr-rook-2:19.2.3-689.gf90f4eae.el9.noarch is already installed. 2026-02-24T18:18:20.735 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-24T18:18:20.735 INFO:teuthology.orchestra.run.trial038.stdout:Nothing to do. 2026-02-24T18:18:20.736 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-24T18:18:20.757 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install ceph-mgr-cephadm 2026-02-24T18:18:21.079 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:01:02 ago on Tue 24 Feb 2026 06:17:19 PM UTC. 2026-02-24T18:18:21.145 INFO:teuthology.orchestra.run.trial038.stdout:Package ceph-mgr-cephadm-2:19.2.3-689.gf90f4eae.el9.noarch is already installed. 2026-02-24T18:18:21.163 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-24T18:18:21.164 INFO:teuthology.orchestra.run.trial038.stdout:Nothing to do. 2026-02-24T18:18:21.164 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-24T18:18:21.185 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install ceph-fuse 2026-02-24T18:18:21.505 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:01:02 ago on Tue 24 Feb 2026 06:17:19 PM UTC. 2026-02-24T18:18:21.591 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-24T18:18:21.592 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-24T18:18:21.592 INFO:teuthology.orchestra.run.trial038.stdout: Package Architecture Version Repository Size 2026-02-24T18:18:21.592 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-24T18:18:21.592 INFO:teuthology.orchestra.run.trial038.stdout:Installing: 2026-02-24T18:18:21.592 INFO:teuthology.orchestra.run.trial038.stdout: ceph-fuse x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 1.2 M 2026-02-24T18:18:21.592 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:18:21.592 INFO:teuthology.orchestra.run.trial038.stdout:Transaction Summary 2026-02-24T18:18:21.592 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-24T18:18:21.592 INFO:teuthology.orchestra.run.trial038.stdout:Install 1 Package 2026-02-24T18:18:21.592 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:18:21.592 INFO:teuthology.orchestra.run.trial038.stdout:Total download size: 1.2 M 2026-02-24T18:18:21.592 INFO:teuthology.orchestra.run.trial038.stdout:Installed size: 3.6 M 2026-02-24T18:18:21.592 INFO:teuthology.orchestra.run.trial038.stdout:Downloading Packages: 2026-02-24T18:18:22.643 INFO:teuthology.orchestra.run.trial038.stdout:ceph-fuse-19.2.3-689.gf90f4eae.el9.x86_64.rpm 1.1 MB/s | 1.2 MB 00:01 2026-02-24T18:18:22.643 INFO:teuthology.orchestra.run.trial038.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:18:22.643 INFO:teuthology.orchestra.run.trial038.stdout:Total 1.1 MB/s | 1.2 MB 00:01 2026-02-24T18:18:22.643 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction check 2026-02-24T18:18:22.648 INFO:teuthology.orchestra.run.trial038.stdout:Transaction check succeeded. 2026-02-24T18:18:22.648 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction test 2026-02-24T18:18:22.685 INFO:teuthology.orchestra.run.trial038.stdout:Transaction test succeeded. 2026-02-24T18:18:22.685 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction 2026-02-24T18:18:22.789 INFO:teuthology.orchestra.run.trial038.stdout: Preparing : 1/1 2026-02-24T18:18:22.807 INFO:teuthology.orchestra.run.trial038.stdout: Installing : ceph-fuse-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:18:23.047 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: ceph-fuse-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:18:23.111 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : ceph-fuse-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:18:23.111 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:18:23.111 INFO:teuthology.orchestra.run.trial038.stdout:Installed: 2026-02-24T18:18:23.111 INFO:teuthology.orchestra.run.trial038.stdout: ceph-fuse-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:18:23.111 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:18:23.111 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-24T18:18:23.139 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install ceph-volume 2026-02-24T18:18:23.452 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:01:04 ago on Tue 24 Feb 2026 06:17:19 PM UTC. 2026-02-24T18:18:23.517 INFO:teuthology.orchestra.run.trial038.stdout:Package ceph-volume-2:19.2.3-689.gf90f4eae.el9.noarch is already installed. 2026-02-24T18:18:23.535 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-24T18:18:23.536 INFO:teuthology.orchestra.run.trial038.stdout:Nothing to do. 2026-02-24T18:18:23.536 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-24T18:18:23.557 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install librados-devel 2026-02-24T18:18:23.870 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:01:04 ago on Tue 24 Feb 2026 06:17:19 PM UTC. 2026-02-24T18:18:23.955 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-24T18:18:23.955 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-24T18:18:23.955 INFO:teuthology.orchestra.run.trial038.stdout: Package Arch Version Repo Size 2026-02-24T18:18:23.955 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-24T18:18:23.955 INFO:teuthology.orchestra.run.trial038.stdout:Installing: 2026-02-24T18:18:23.956 INFO:teuthology.orchestra.run.trial038.stdout: librados-devel x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 126 k 2026-02-24T18:18:23.956 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:18:23.956 INFO:teuthology.orchestra.run.trial038.stdout:Transaction Summary 2026-02-24T18:18:23.956 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-24T18:18:23.956 INFO:teuthology.orchestra.run.trial038.stdout:Install 1 Package 2026-02-24T18:18:23.956 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:18:23.956 INFO:teuthology.orchestra.run.trial038.stdout:Total download size: 126 k 2026-02-24T18:18:23.956 INFO:teuthology.orchestra.run.trial038.stdout:Installed size: 456 k 2026-02-24T18:18:23.956 INFO:teuthology.orchestra.run.trial038.stdout:Downloading Packages: 2026-02-24T18:18:24.442 INFO:teuthology.orchestra.run.trial038.stdout:librados-devel-19.2.3-689.gf90f4eae.el9.x86_64. 260 kB/s | 126 kB 00:00 2026-02-24T18:18:24.442 INFO:teuthology.orchestra.run.trial038.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:18:24.443 INFO:teuthology.orchestra.run.trial038.stdout:Total 259 kB/s | 126 kB 00:00 2026-02-24T18:18:24.443 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction check 2026-02-24T18:18:24.445 INFO:teuthology.orchestra.run.trial038.stdout:Transaction check succeeded. 2026-02-24T18:18:24.445 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction test 2026-02-24T18:18:24.477 INFO:teuthology.orchestra.run.trial038.stdout:Transaction test succeeded. 2026-02-24T18:18:24.477 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction 2026-02-24T18:18:24.532 INFO:teuthology.orchestra.run.trial038.stdout: Preparing : 1/1 2026-02-24T18:18:24.550 INFO:teuthology.orchestra.run.trial038.stdout: Installing : librados-devel-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:18:24.640 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: librados-devel-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:18:24.703 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : librados-devel-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:18:24.703 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:18:24.703 INFO:teuthology.orchestra.run.trial038.stdout:Installed: 2026-02-24T18:18:24.703 INFO:teuthology.orchestra.run.trial038.stdout: librados-devel-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:18:24.703 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:18:24.703 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-24T18:18:24.731 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install libcephfs2 2026-02-24T18:18:25.053 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:01:06 ago on Tue 24 Feb 2026 06:17:19 PM UTC. 2026-02-24T18:18:25.118 INFO:teuthology.orchestra.run.trial038.stdout:Package libcephfs2-2:19.2.3-689.gf90f4eae.el9.x86_64 is already installed. 2026-02-24T18:18:25.136 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-24T18:18:25.137 INFO:teuthology.orchestra.run.trial038.stdout:Nothing to do. 2026-02-24T18:18:25.137 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-24T18:18:25.158 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install libcephfs-devel 2026-02-24T18:18:25.474 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:01:06 ago on Tue 24 Feb 2026 06:17:19 PM UTC. 2026-02-24T18:18:25.559 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-24T18:18:25.559 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-24T18:18:25.560 INFO:teuthology.orchestra.run.trial038.stdout: Package Arch Version Repo Size 2026-02-24T18:18:25.560 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-24T18:18:25.560 INFO:teuthology.orchestra.run.trial038.stdout:Installing: 2026-02-24T18:18:25.560 INFO:teuthology.orchestra.run.trial038.stdout: libcephfs-devel x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 34 k 2026-02-24T18:18:25.560 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:18:25.560 INFO:teuthology.orchestra.run.trial038.stdout:Transaction Summary 2026-02-24T18:18:25.561 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-24T18:18:25.561 INFO:teuthology.orchestra.run.trial038.stdout:Install 1 Package 2026-02-24T18:18:25.561 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:18:25.561 INFO:teuthology.orchestra.run.trial038.stdout:Total download size: 34 k 2026-02-24T18:18:25.561 INFO:teuthology.orchestra.run.trial038.stdout:Installed size: 153 k 2026-02-24T18:18:25.561 INFO:teuthology.orchestra.run.trial038.stdout:Downloading Packages: 2026-02-24T18:18:25.855 INFO:teuthology.orchestra.run.trial038.stdout:libcephfs-devel-19.2.3-689.gf90f4eae.el9.x86_64 113 kB/s | 34 kB 00:00 2026-02-24T18:18:25.856 INFO:teuthology.orchestra.run.trial038.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:18:25.856 INFO:teuthology.orchestra.run.trial038.stdout:Total 113 kB/s | 34 kB 00:00 2026-02-24T18:18:25.856 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction check 2026-02-24T18:18:25.858 INFO:teuthology.orchestra.run.trial038.stdout:Transaction check succeeded. 2026-02-24T18:18:25.858 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction test 2026-02-24T18:18:25.861 INFO:teuthology.orchestra.run.trial038.stdout:Transaction test succeeded. 2026-02-24T18:18:25.861 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction 2026-02-24T18:18:25.883 INFO:teuthology.orchestra.run.trial038.stdout: Preparing : 1/1 2026-02-24T18:18:25.900 INFO:teuthology.orchestra.run.trial038.stdout: Installing : libcephfs-devel-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:18:25.962 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: libcephfs-devel-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:18:26.028 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : libcephfs-devel-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:18:26.028 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:18:26.028 INFO:teuthology.orchestra.run.trial038.stdout:Installed: 2026-02-24T18:18:26.028 INFO:teuthology.orchestra.run.trial038.stdout: libcephfs-devel-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:18:26.028 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:18:26.028 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-24T18:18:26.054 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install librados2 2026-02-24T18:18:26.369 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:01:07 ago on Tue 24 Feb 2026 06:17:19 PM UTC. 2026-02-24T18:18:26.434 INFO:teuthology.orchestra.run.trial038.stdout:Package librados2-2:19.2.3-689.gf90f4eae.el9.x86_64 is already installed. 2026-02-24T18:18:26.453 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-24T18:18:26.454 INFO:teuthology.orchestra.run.trial038.stdout:Nothing to do. 2026-02-24T18:18:26.454 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-24T18:18:26.474 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install librbd1 2026-02-24T18:18:26.786 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:01:07 ago on Tue 24 Feb 2026 06:17:19 PM UTC. 2026-02-24T18:18:26.852 INFO:teuthology.orchestra.run.trial038.stdout:Package librbd1-2:19.2.3-689.gf90f4eae.el9.x86_64 is already installed. 2026-02-24T18:18:26.871 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-24T18:18:26.871 INFO:teuthology.orchestra.run.trial038.stdout:Nothing to do. 2026-02-24T18:18:26.871 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-24T18:18:26.892 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install python3-rados 2026-02-24T18:18:27.206 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:01:08 ago on Tue 24 Feb 2026 06:17:19 PM UTC. 2026-02-24T18:18:27.271 INFO:teuthology.orchestra.run.trial038.stdout:Package python3-rados-2:19.2.3-689.gf90f4eae.el9.x86_64 is already installed. 2026-02-24T18:18:27.290 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-24T18:18:27.290 INFO:teuthology.orchestra.run.trial038.stdout:Nothing to do. 2026-02-24T18:18:27.290 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-24T18:18:27.312 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install python3-rgw 2026-02-24T18:18:27.626 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:01:08 ago on Tue 24 Feb 2026 06:17:19 PM UTC. 2026-02-24T18:18:27.691 INFO:teuthology.orchestra.run.trial038.stdout:Package python3-rgw-2:19.2.3-689.gf90f4eae.el9.x86_64 is already installed. 2026-02-24T18:18:27.709 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-24T18:18:27.710 INFO:teuthology.orchestra.run.trial038.stdout:Nothing to do. 2026-02-24T18:18:27.710 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-24T18:18:27.732 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install python3-cephfs 2026-02-24T18:18:28.053 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:01:09 ago on Tue 24 Feb 2026 06:17:19 PM UTC. 2026-02-24T18:18:28.119 INFO:teuthology.orchestra.run.trial038.stdout:Package python3-cephfs-2:19.2.3-689.gf90f4eae.el9.x86_64 is already installed. 2026-02-24T18:18:28.138 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-24T18:18:28.139 INFO:teuthology.orchestra.run.trial038.stdout:Nothing to do. 2026-02-24T18:18:28.139 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-24T18:18:28.160 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install python3-rbd 2026-02-24T18:18:28.478 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:01:09 ago on Tue 24 Feb 2026 06:17:19 PM UTC. 2026-02-24T18:18:28.543 INFO:teuthology.orchestra.run.trial038.stdout:Package python3-rbd-2:19.2.3-689.gf90f4eae.el9.x86_64 is already installed. 2026-02-24T18:18:28.562 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-24T18:18:28.562 INFO:teuthology.orchestra.run.trial038.stdout:Nothing to do. 2026-02-24T18:18:28.562 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-24T18:18:28.584 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install rbd-fuse 2026-02-24T18:18:28.899 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:01:09 ago on Tue 24 Feb 2026 06:17:19 PM UTC. 2026-02-24T18:18:28.984 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-24T18:18:28.984 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-24T18:18:28.984 INFO:teuthology.orchestra.run.trial038.stdout: Package Architecture Version Repository Size 2026-02-24T18:18:28.984 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-24T18:18:28.984 INFO:teuthology.orchestra.run.trial038.stdout:Installing: 2026-02-24T18:18:28.985 INFO:teuthology.orchestra.run.trial038.stdout: rbd-fuse x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 85 k 2026-02-24T18:18:28.985 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:18:28.985 INFO:teuthology.orchestra.run.trial038.stdout:Transaction Summary 2026-02-24T18:18:28.985 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-24T18:18:28.985 INFO:teuthology.orchestra.run.trial038.stdout:Install 1 Package 2026-02-24T18:18:28.985 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:18:28.985 INFO:teuthology.orchestra.run.trial038.stdout:Total download size: 85 k 2026-02-24T18:18:28.985 INFO:teuthology.orchestra.run.trial038.stdout:Installed size: 227 k 2026-02-24T18:18:28.985 INFO:teuthology.orchestra.run.trial038.stdout:Downloading Packages: 2026-02-24T18:18:29.312 INFO:teuthology.orchestra.run.trial038.stdout:rbd-fuse-19.2.3-689.gf90f4eae.el9.x86_64.rpm 260 kB/s | 85 kB 00:00 2026-02-24T18:18:29.312 INFO:teuthology.orchestra.run.trial038.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:18:29.312 INFO:teuthology.orchestra.run.trial038.stdout:Total 260 kB/s | 85 kB 00:00 2026-02-24T18:18:29.313 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction check 2026-02-24T18:18:29.316 INFO:teuthology.orchestra.run.trial038.stdout:Transaction check succeeded. 2026-02-24T18:18:29.316 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction test 2026-02-24T18:18:29.347 INFO:teuthology.orchestra.run.trial038.stdout:Transaction test succeeded. 2026-02-24T18:18:29.347 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction 2026-02-24T18:18:29.398 INFO:teuthology.orchestra.run.trial038.stdout: Preparing : 1/1 2026-02-24T18:18:29.416 INFO:teuthology.orchestra.run.trial038.stdout: Installing : rbd-fuse-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:18:29.504 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: rbd-fuse-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:18:29.566 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : rbd-fuse-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:18:29.566 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:18:29.566 INFO:teuthology.orchestra.run.trial038.stdout:Installed: 2026-02-24T18:18:29.566 INFO:teuthology.orchestra.run.trial038.stdout: rbd-fuse-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:18:29.566 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:18:29.566 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-24T18:18:29.595 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install rbd-mirror 2026-02-24T18:18:29.912 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:01:10 ago on Tue 24 Feb 2026 06:17:19 PM UTC. 2026-02-24T18:18:29.997 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-24T18:18:29.997 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-24T18:18:29.997 INFO:teuthology.orchestra.run.trial038.stdout: Package Arch Version Repo Size 2026-02-24T18:18:29.998 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-24T18:18:29.998 INFO:teuthology.orchestra.run.trial038.stdout:Installing: 2026-02-24T18:18:29.998 INFO:teuthology.orchestra.run.trial038.stdout: rbd-mirror x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 3.1 M 2026-02-24T18:18:29.998 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:18:29.998 INFO:teuthology.orchestra.run.trial038.stdout:Transaction Summary 2026-02-24T18:18:29.998 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-24T18:18:29.998 INFO:teuthology.orchestra.run.trial038.stdout:Install 1 Package 2026-02-24T18:18:29.998 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:18:29.998 INFO:teuthology.orchestra.run.trial038.stdout:Total download size: 3.1 M 2026-02-24T18:18:29.998 INFO:teuthology.orchestra.run.trial038.stdout:Installed size: 13 M 2026-02-24T18:18:29.999 INFO:teuthology.orchestra.run.trial038.stdout:Downloading Packages: 2026-02-24T18:18:30.569 INFO:teuthology.orchestra.run.trial038.stdout:rbd-mirror-19.2.3-689.gf90f4eae.el9.x86_64.rpm 5.5 MB/s | 3.1 MB 00:00 2026-02-24T18:18:30.571 INFO:teuthology.orchestra.run.trial038.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:18:30.572 INFO:teuthology.orchestra.run.trial038.stdout:Total 5.4 MB/s | 3.1 MB 00:00 2026-02-24T18:18:30.572 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction check 2026-02-24T18:18:30.574 INFO:teuthology.orchestra.run.trial038.stdout:Transaction check succeeded. 2026-02-24T18:18:30.575 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction test 2026-02-24T18:18:30.614 INFO:teuthology.orchestra.run.trial038.stdout:Transaction test succeeded. 2026-02-24T18:18:30.614 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction 2026-02-24T18:18:30.807 INFO:teuthology.orchestra.run.trial038.stdout: Preparing : 1/1 2026-02-24T18:18:30.849 INFO:teuthology.orchestra.run.trial038.stdout: Installing : rbd-mirror-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:18:30.849 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: rbd-mirror-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:18:30.849 INFO:teuthology.orchestra.run.trial038.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-24T18:18:30.849 INFO:teuthology.orchestra.run.trial038.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2026-02-24T18:18:30.849 INFO:teuthology.orchestra.run.trial038.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-24T18:18:30.849 INFO:teuthology.orchestra.run.trial038.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-24T18:18:30.849 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:18:31.141 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : rbd-mirror-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:18:31.142 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:18:31.142 INFO:teuthology.orchestra.run.trial038.stdout:Installed: 2026-02-24T18:18:31.142 INFO:teuthology.orchestra.run.trial038.stdout: rbd-mirror-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:18:31.142 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:18:31.142 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-24T18:18:31.167 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install rbd-nbd 2026-02-24T18:18:31.482 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:01:12 ago on Tue 24 Feb 2026 06:17:19 PM UTC. 2026-02-24T18:18:31.565 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-24T18:18:31.566 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-24T18:18:31.566 INFO:teuthology.orchestra.run.trial038.stdout: Package Architecture Version Repository Size 2026-02-24T18:18:31.566 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-24T18:18:31.566 INFO:teuthology.orchestra.run.trial038.stdout:Installing: 2026-02-24T18:18:31.566 INFO:teuthology.orchestra.run.trial038.stdout: rbd-nbd x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 170 k 2026-02-24T18:18:31.566 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:18:31.566 INFO:teuthology.orchestra.run.trial038.stdout:Transaction Summary 2026-02-24T18:18:31.566 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-24T18:18:31.566 INFO:teuthology.orchestra.run.trial038.stdout:Install 1 Package 2026-02-24T18:18:31.566 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:18:31.566 INFO:teuthology.orchestra.run.trial038.stdout:Total download size: 170 k 2026-02-24T18:18:31.566 INFO:teuthology.orchestra.run.trial038.stdout:Installed size: 490 k 2026-02-24T18:18:31.566 INFO:teuthology.orchestra.run.trial038.stdout:Downloading Packages: 2026-02-24T18:18:31.881 INFO:teuthology.orchestra.run.trial038.stdout:rbd-nbd-19.2.3-689.gf90f4eae.el9.x86_64.rpm 543 kB/s | 170 kB 00:00 2026-02-24T18:18:31.881 INFO:teuthology.orchestra.run.trial038.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:18:31.881 INFO:teuthology.orchestra.run.trial038.stdout:Total 542 kB/s | 170 kB 00:00 2026-02-24T18:18:31.881 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction check 2026-02-24T18:18:31.885 INFO:teuthology.orchestra.run.trial038.stdout:Transaction check succeeded. 2026-02-24T18:18:31.886 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction test 2026-02-24T18:18:31.917 INFO:teuthology.orchestra.run.trial038.stdout:Transaction test succeeded. 2026-02-24T18:18:31.917 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction 2026-02-24T18:18:31.973 INFO:teuthology.orchestra.run.trial038.stdout: Preparing : 1/1 2026-02-24T18:18:31.991 INFO:teuthology.orchestra.run.trial038.stdout: Installing : rbd-nbd-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:18:32.078 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: rbd-nbd-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:18:32.141 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : rbd-nbd-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:18:32.141 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:18:32.141 INFO:teuthology.orchestra.run.trial038.stdout:Installed: 2026-02-24T18:18:32.141 INFO:teuthology.orchestra.run.trial038.stdout: rbd-nbd-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:18:32.141 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:18:32.141 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-24T18:18:32.170 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install ceph-base-debuginfo 2026-02-24T18:18:32.491 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:01:13 ago on Tue 24 Feb 2026 06:17:19 PM UTC. 2026-02-24T18:18:32.574 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-24T18:18:32.575 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-24T18:18:32.576 INFO:teuthology.orchestra.run.trial038.stdout: Package Arch Version Repo Size 2026-02-24T18:18:32.576 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-24T18:18:32.576 INFO:teuthology.orchestra.run.trial038.stdout:Installing: 2026-02-24T18:18:32.576 INFO:teuthology.orchestra.run.trial038.stdout: ceph-base-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 72 M 2026-02-24T18:18:32.576 INFO:teuthology.orchestra.run.trial038.stdout:Installing dependencies: 2026-02-24T18:18:32.576 INFO:teuthology.orchestra.run.trial038.stdout: ceph-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 24 M 2026-02-24T18:18:32.576 INFO:teuthology.orchestra.run.trial038.stdout:Installing weak dependencies: 2026-02-24T18:18:32.576 INFO:teuthology.orchestra.run.trial038.stdout: ceph-debugsource x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 11 M 2026-02-24T18:18:32.576 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:18:32.576 INFO:teuthology.orchestra.run.trial038.stdout:Transaction Summary 2026-02-24T18:18:32.577 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-24T18:18:32.577 INFO:teuthology.orchestra.run.trial038.stdout:Install 3 Packages 2026-02-24T18:18:32.577 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:18:32.577 INFO:teuthology.orchestra.run.trial038.stdout:Total download size: 107 M 2026-02-24T18:18:32.577 INFO:teuthology.orchestra.run.trial038.stdout:Installed size: 654 M 2026-02-24T18:18:32.577 INFO:teuthology.orchestra.run.trial038.stdout:Downloading Packages: 2026-02-24T18:18:33.782 INFO:teuthology.orchestra.run.trial038.stdout:(1/3): ceph-debugsource-19.2.3-689.gf90f4eae.el 9.0 MB/s | 11 MB 00:01 2026-02-24T18:18:34.724 INFO:teuthology.orchestra.run.trial038.stdout:(2/3): ceph-debuginfo-19.2.3-689.gf90f4eae.el9. 11 MB/s | 24 MB 00:02 2026-02-24T18:18:35.858 INFO:teuthology.orchestra.run.trial038.stdout:(3/3): ceph-base-debuginfo-19.2.3-689.gf90f4eae 22 MB/s | 72 MB 00:03 2026-02-24T18:18:35.860 INFO:teuthology.orchestra.run.trial038.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:18:35.860 INFO:teuthology.orchestra.run.trial038.stdout:Total 33 MB/s | 107 MB 00:03 2026-02-24T18:18:35.860 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction check 2026-02-24T18:18:35.865 INFO:teuthology.orchestra.run.trial038.stdout:Transaction check succeeded. 2026-02-24T18:18:35.865 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction test 2026-02-24T18:18:36.244 INFO:teuthology.orchestra.run.trial038.stdout:Transaction test succeeded. 2026-02-24T18:18:36.244 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction 2026-02-24T18:18:38.690 INFO:teuthology.orchestra.run.trial038.stdout: Preparing : 1/1 2026-02-24T18:18:39.983 INFO:teuthology.orchestra.run.trial038.stdout: Installing : ceph-debugsource-2:19.2.3-689.gf90f4eae.el9.x86_64 1/3 2026-02-24T18:18:43.736 INFO:teuthology.orchestra.run.trial038.stdout: Installing : ceph-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2/3 2026-02-24T18:18:43.756 INFO:teuthology.orchestra.run.trial038.stdout: Installing : ceph-base-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_6 3/3 2026-02-24T18:18:44.266 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: ceph-base-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_6 3/3 2026-02-24T18:18:44.266 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : ceph-base-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_6 1/3 2026-02-24T18:18:44.266 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : ceph-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2/3 2026-02-24T18:18:44.329 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : ceph-debugsource-2:19.2.3-689.gf90f4eae.el9.x86_64 3/3 2026-02-24T18:18:44.329 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:18:44.329 INFO:teuthology.orchestra.run.trial038.stdout:Installed: 2026-02-24T18:18:44.329 INFO:teuthology.orchestra.run.trial038.stdout: ceph-base-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:18:44.329 INFO:teuthology.orchestra.run.trial038.stdout: ceph-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:18:44.329 INFO:teuthology.orchestra.run.trial038.stdout: ceph-debugsource-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:18:44.329 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:18:44.329 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-24T18:18:44.370 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install ceph-common-debuginfo 2026-02-24T18:18:44.691 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:01:25 ago on Tue 24 Feb 2026 06:17:19 PM UTC. 2026-02-24T18:18:44.776 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-24T18:18:44.776 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-24T18:18:44.776 INFO:teuthology.orchestra.run.trial038.stdout: Package Arch Version Repo Size 2026-02-24T18:18:44.776 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-24T18:18:44.776 INFO:teuthology.orchestra.run.trial038.stdout:Installing: 2026-02-24T18:18:44.776 INFO:teuthology.orchestra.run.trial038.stdout: ceph-common-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 474 M 2026-02-24T18:18:44.777 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:18:44.777 INFO:teuthology.orchestra.run.trial038.stdout:Transaction Summary 2026-02-24T18:18:44.777 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-24T18:18:44.777 INFO:teuthology.orchestra.run.trial038.stdout:Install 1 Package 2026-02-24T18:18:44.777 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:18:44.777 INFO:teuthology.orchestra.run.trial038.stdout:Total download size: 474 M 2026-02-24T18:18:44.777 INFO:teuthology.orchestra.run.trial038.stdout:Installed size: 2.0 G 2026-02-24T18:18:44.777 INFO:teuthology.orchestra.run.trial038.stdout:Downloading Packages: 2026-02-24T18:20:23.319 INFO:teuthology.orchestra.run.trial038.stdout:ceph-common-debuginfo-19.2.3-689.gf90f4eae.el9. 4.8 MB/s | 474 MB 01:38 2026-02-24T18:20:23.320 INFO:teuthology.orchestra.run.trial038.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:20:23.320 INFO:teuthology.orchestra.run.trial038.stdout:Total 4.8 MB/s | 474 MB 01:38 2026-02-24T18:20:23.320 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction check 2026-02-24T18:20:23.322 INFO:teuthology.orchestra.run.trial038.stdout:Transaction check succeeded. 2026-02-24T18:20:23.322 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction test 2026-02-24T18:20:24.733 INFO:teuthology.orchestra.run.trial038.stdout:Transaction test succeeded. 2026-02-24T18:20:24.733 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction 2026-02-24T18:20:48.694 INFO:teuthology.orchestra.run.trial038.stdout: Preparing : 1/1 2026-02-24T18:20:48.713 INFO:teuthology.orchestra.run.trial038.stdout: Installing : ceph-common-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86 1/1 2026-02-24T18:20:50.227 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: ceph-common-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86 1/1 2026-02-24T18:20:50.291 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : ceph-common-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86 1/1 2026-02-24T18:20:50.291 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:20:50.291 INFO:teuthology.orchestra.run.trial038.stdout:Installed: 2026-02-24T18:20:50.291 INFO:teuthology.orchestra.run.trial038.stdout: ceph-common-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:20:50.291 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:20:50.291 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-24T18:20:50.360 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install ceph-immutable-object-cache-debuginfo 2026-02-24T18:20:50.682 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:03:31 ago on Tue 24 Feb 2026 06:17:19 PM UTC. 2026-02-24T18:20:50.765 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-24T18:20:50.766 INFO:teuthology.orchestra.run.trial038.stdout:======================================================================================= 2026-02-24T18:20:50.766 INFO:teuthology.orchestra.run.trial038.stdout: Package Arch Version Repo Size 2026-02-24T18:20:50.766 INFO:teuthology.orchestra.run.trial038.stdout:======================================================================================= 2026-02-24T18:20:50.766 INFO:teuthology.orchestra.run.trial038.stdout:Installing: 2026-02-24T18:20:50.766 INFO:teuthology.orchestra.run.trial038.stdout: ceph-immutable-object-cache-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 1.5 M 2026-02-24T18:20:50.767 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:20:50.767 INFO:teuthology.orchestra.run.trial038.stdout:Transaction Summary 2026-02-24T18:20:50.767 INFO:teuthology.orchestra.run.trial038.stdout:======================================================================================= 2026-02-24T18:20:50.767 INFO:teuthology.orchestra.run.trial038.stdout:Install 1 Package 2026-02-24T18:20:50.767 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:20:50.767 INFO:teuthology.orchestra.run.trial038.stdout:Total download size: 1.5 M 2026-02-24T18:20:50.767 INFO:teuthology.orchestra.run.trial038.stdout:Installed size: 7.6 M 2026-02-24T18:20:50.767 INFO:teuthology.orchestra.run.trial038.stdout:Downloading Packages: 2026-02-24T18:20:51.654 INFO:teuthology.orchestra.run.trial038.stdout:ceph-immutable-object-cache-debuginfo-19.2.3-68 1.7 MB/s | 1.5 MB 00:00 2026-02-24T18:20:51.654 INFO:teuthology.orchestra.run.trial038.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:20:51.654 INFO:teuthology.orchestra.run.trial038.stdout:Total 1.7 MB/s | 1.5 MB 00:00 2026-02-24T18:20:51.655 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction check 2026-02-24T18:20:51.656 INFO:teuthology.orchestra.run.trial038.stdout:Transaction check succeeded. 2026-02-24T18:20:51.656 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction test 2026-02-24T18:20:51.679 INFO:teuthology.orchestra.run.trial038.stdout:Transaction test succeeded. 2026-02-24T18:20:51.679 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction 2026-02-24T18:20:51.794 INFO:teuthology.orchestra.run.trial038.stdout: Preparing : 1/1 2026-02-24T18:20:51.811 INFO:teuthology.orchestra.run.trial038.stdout: Installing : ceph-immutable-object-cache-debuginfo-2:19.2.3-689.g 1/1 2026-02-24T18:20:51.876 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: ceph-immutable-object-cache-debuginfo-2:19.2.3-689.g 1/1 2026-02-24T18:20:51.940 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : ceph-immutable-object-cache-debuginfo-2:19.2.3-689.g 1/1 2026-02-24T18:20:51.940 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:20:51.940 INFO:teuthology.orchestra.run.trial038.stdout:Installed: 2026-02-24T18:20:51.940 INFO:teuthology.orchestra.run.trial038.stdout: ceph-immutable-object-cache-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:20:51.940 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:20:51.941 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-24T18:20:51.968 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install ceph-radosgw-debuginfo 2026-02-24T18:20:52.286 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:03:33 ago on Tue 24 Feb 2026 06:17:19 PM UTC. 2026-02-24T18:20:52.369 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-24T18:20:52.370 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-24T18:20:52.370 INFO:teuthology.orchestra.run.trial038.stdout: Package Arch Version Repo Size 2026-02-24T18:20:52.371 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-24T18:20:52.371 INFO:teuthology.orchestra.run.trial038.stdout:Installing: 2026-02-24T18:20:52.371 INFO:teuthology.orchestra.run.trial038.stdout: ceph-radosgw-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 200 M 2026-02-24T18:20:52.371 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:20:52.371 INFO:teuthology.orchestra.run.trial038.stdout:Transaction Summary 2026-02-24T18:20:52.371 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-24T18:20:52.371 INFO:teuthology.orchestra.run.trial038.stdout:Install 1 Package 2026-02-24T18:20:52.371 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:20:52.371 INFO:teuthology.orchestra.run.trial038.stdout:Total download size: 200 M 2026-02-24T18:20:52.371 INFO:teuthology.orchestra.run.trial038.stdout:Installed size: 893 M 2026-02-24T18:20:52.372 INFO:teuthology.orchestra.run.trial038.stdout:Downloading Packages: 2026-02-24T18:22:03.152 INFO:teuthology.orchestra.run.trial038.stdout:ceph-radosgw-debuginfo-19.2.3-689.gf90f4eae.el9 2.8 MB/s | 200 MB 01:10 2026-02-24T18:22:03.153 INFO:teuthology.orchestra.run.trial038.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:22:03.153 INFO:teuthology.orchestra.run.trial038.stdout:Total 2.8 MB/s | 200 MB 01:10 2026-02-24T18:22:03.153 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction check 2026-02-24T18:22:03.155 INFO:teuthology.orchestra.run.trial038.stdout:Transaction check succeeded. 2026-02-24T18:22:03.156 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction test 2026-02-24T18:22:03.763 INFO:teuthology.orchestra.run.trial038.stdout:Transaction test succeeded. 2026-02-24T18:22:03.763 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction 2026-02-24T18:22:13.801 INFO:teuthology.orchestra.run.trial038.stdout: Preparing : 1/1 2026-02-24T18:22:13.819 INFO:teuthology.orchestra.run.trial038.stdout: Installing : ceph-radosgw-debuginfo-2:19.2.3-689.gf90f4eae.el9.x8 1/1 2026-02-24T18:22:14.520 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: ceph-radosgw-debuginfo-2:19.2.3-689.gf90f4eae.el9.x8 1/1 2026-02-24T18:22:14.583 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : ceph-radosgw-debuginfo-2:19.2.3-689.gf90f4eae.el9.x8 1/1 2026-02-24T18:22:14.583 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:22:14.583 INFO:teuthology.orchestra.run.trial038.stdout:Installed: 2026-02-24T18:22:14.583 INFO:teuthology.orchestra.run.trial038.stdout: ceph-radosgw-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:22:14.583 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:22:14.583 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-24T18:22:14.626 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install ceph-test-debuginfo 2026-02-24T18:22:14.945 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:04:55 ago on Tue 24 Feb 2026 06:17:19 PM UTC. 2026-02-24T18:22:15.028 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-24T18:22:15.028 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-24T18:22:15.028 INFO:teuthology.orchestra.run.trial038.stdout: Package Arch Version Repo Size 2026-02-24T18:22:15.028 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-24T18:22:15.028 INFO:teuthology.orchestra.run.trial038.stdout:Installing: 2026-02-24T18:22:15.028 INFO:teuthology.orchestra.run.trial038.stdout: ceph-test-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 729 M 2026-02-24T18:22:15.029 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:22:15.029 INFO:teuthology.orchestra.run.trial038.stdout:Transaction Summary 2026-02-24T18:22:15.029 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-24T18:22:15.029 INFO:teuthology.orchestra.run.trial038.stdout:Install 1 Package 2026-02-24T18:22:15.029 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:22:15.029 INFO:teuthology.orchestra.run.trial038.stdout:Total download size: 729 M 2026-02-24T18:22:15.029 INFO:teuthology.orchestra.run.trial038.stdout:Installed size: 3.6 G 2026-02-24T18:22:15.029 INFO:teuthology.orchestra.run.trial038.stdout:Downloading Packages: 2026-02-24T18:25:56.559 INFO:teuthology.orchestra.run.trial038.stdout:ceph-test-debuginfo-19.2.3-689.gf90f4eae.el9.x8 3.3 MB/s | 729 MB 03:41 2026-02-24T18:25:56.559 INFO:teuthology.orchestra.run.trial038.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:25:56.560 INFO:teuthology.orchestra.run.trial038.stdout:Total 3.3 MB/s | 729 MB 03:41 2026-02-24T18:25:56.560 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction check 2026-02-24T18:25:56.562 INFO:teuthology.orchestra.run.trial038.stdout:Transaction check succeeded. 2026-02-24T18:25:56.562 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction test 2026-02-24T18:25:58.724 INFO:teuthology.orchestra.run.trial038.stdout:Transaction test succeeded. 2026-02-24T18:25:58.724 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction 2026-02-24T18:26:36.480 INFO:teuthology.orchestra.run.trial038.stdout: Preparing : 1/1 2026-02-24T18:26:36.501 INFO:teuthology.orchestra.run.trial038.stdout: Installing : ceph-test-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_6 1/1 2026-02-24T18:26:37.174 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: ceph-test-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_6 1/1 2026-02-24T18:26:37.237 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : ceph-test-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_6 1/1 2026-02-24T18:26:37.238 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:26:37.238 INFO:teuthology.orchestra.run.trial038.stdout:Installed: 2026-02-24T18:26:37.238 INFO:teuthology.orchestra.run.trial038.stdout: ceph-test-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:26:37.238 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:26:37.238 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-24T18:26:37.334 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install ceph-base-debuginfo 2026-02-24T18:26:37.658 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:09:18 ago on Tue 24 Feb 2026 06:17:19 PM UTC. 2026-02-24T18:26:37.724 INFO:teuthology.orchestra.run.trial038.stdout:Package ceph-base-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 is already installed. 2026-02-24T18:26:37.742 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-24T18:26:37.743 INFO:teuthology.orchestra.run.trial038.stdout:Nothing to do. 2026-02-24T18:26:37.743 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-24T18:26:37.763 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install ceph-mgr-debuginfo 2026-02-24T18:26:38.079 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:09:19 ago on Tue 24 Feb 2026 06:17:19 PM UTC. 2026-02-24T18:26:38.163 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-24T18:26:38.163 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-24T18:26:38.163 INFO:teuthology.orchestra.run.trial038.stdout: Package Arch Version Repo Size 2026-02-24T18:26:38.164 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-24T18:26:38.164 INFO:teuthology.orchestra.run.trial038.stdout:Installing: 2026-02-24T18:26:38.164 INFO:teuthology.orchestra.run.trial038.stdout: ceph-mgr-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 18 M 2026-02-24T18:26:38.164 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:26:38.164 INFO:teuthology.orchestra.run.trial038.stdout:Transaction Summary 2026-02-24T18:26:38.164 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-24T18:26:38.164 INFO:teuthology.orchestra.run.trial038.stdout:Install 1 Package 2026-02-24T18:26:38.164 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:26:38.164 INFO:teuthology.orchestra.run.trial038.stdout:Total download size: 18 M 2026-02-24T18:26:38.164 INFO:teuthology.orchestra.run.trial038.stdout:Installed size: 110 M 2026-02-24T18:26:38.164 INFO:teuthology.orchestra.run.trial038.stdout:Downloading Packages: 2026-02-24T18:26:39.484 INFO:teuthology.orchestra.run.trial038.stdout:ceph-mgr-debuginfo-19.2.3-689.gf90f4eae.el9.x86 14 MB/s | 18 MB 00:01 2026-02-24T18:26:39.485 INFO:teuthology.orchestra.run.trial038.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:26:39.485 INFO:teuthology.orchestra.run.trial038.stdout:Total 14 MB/s | 18 MB 00:01 2026-02-24T18:26:39.485 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction check 2026-02-24T18:26:39.487 INFO:teuthology.orchestra.run.trial038.stdout:Transaction check succeeded. 2026-02-24T18:26:39.487 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction test 2026-02-24T18:26:39.559 INFO:teuthology.orchestra.run.trial038.stdout:Transaction test succeeded. 2026-02-24T18:26:39.559 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction 2026-02-24T18:26:40.577 INFO:teuthology.orchestra.run.trial038.stdout: Preparing : 1/1 2026-02-24T18:26:40.595 INFO:teuthology.orchestra.run.trial038.stdout: Installing : ceph-mgr-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:26:40.729 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: ceph-mgr-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:26:40.792 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : ceph-mgr-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:26:40.793 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:26:40.793 INFO:teuthology.orchestra.run.trial038.stdout:Installed: 2026-02-24T18:26:40.793 INFO:teuthology.orchestra.run.trial038.stdout: ceph-mgr-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:26:40.793 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:26:40.793 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-24T18:26:40.820 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install ceph-mds-debuginfo 2026-02-24T18:26:41.137 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:09:22 ago on Tue 24 Feb 2026 06:17:19 PM UTC. 2026-02-24T18:26:41.221 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-24T18:26:41.221 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-24T18:26:41.221 INFO:teuthology.orchestra.run.trial038.stdout: Package Arch Version Repo Size 2026-02-24T18:26:41.221 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-24T18:26:41.221 INFO:teuthology.orchestra.run.trial038.stdout:Installing: 2026-02-24T18:26:41.221 INFO:teuthology.orchestra.run.trial038.stdout: ceph-mds-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 55 M 2026-02-24T18:26:41.221 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:26:41.221 INFO:teuthology.orchestra.run.trial038.stdout:Transaction Summary 2026-02-24T18:26:41.221 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-24T18:26:41.221 INFO:teuthology.orchestra.run.trial038.stdout:Install 1 Package 2026-02-24T18:26:41.221 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:26:41.221 INFO:teuthology.orchestra.run.trial038.stdout:Total download size: 55 M 2026-02-24T18:26:41.221 INFO:teuthology.orchestra.run.trial038.stdout:Installed size: 211 M 2026-02-24T18:26:41.221 INFO:teuthology.orchestra.run.trial038.stdout:Downloading Packages: 2026-02-24T18:26:49.741 INFO:teuthology.orchestra.run.trial038.stdout:ceph-mds-debuginfo-19.2.3-689.gf90f4eae.el9.x86 6.5 MB/s | 55 MB 00:08 2026-02-24T18:26:49.741 INFO:teuthology.orchestra.run.trial038.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:26:49.741 INFO:teuthology.orchestra.run.trial038.stdout:Total 6.5 MB/s | 55 MB 00:08 2026-02-24T18:26:49.741 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction check 2026-02-24T18:26:49.744 INFO:teuthology.orchestra.run.trial038.stdout:Transaction check succeeded. 2026-02-24T18:26:49.744 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction test 2026-02-24T18:26:49.924 INFO:teuthology.orchestra.run.trial038.stdout:Transaction test succeeded. 2026-02-24T18:26:49.924 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction 2026-02-24T18:26:52.643 INFO:teuthology.orchestra.run.trial038.stdout: Preparing : 1/1 2026-02-24T18:26:52.662 INFO:teuthology.orchestra.run.trial038.stdout: Installing : ceph-mds-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:26:52.866 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: ceph-mds-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:26:52.931 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : ceph-mds-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:26:52.931 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:26:52.931 INFO:teuthology.orchestra.run.trial038.stdout:Installed: 2026-02-24T18:26:52.931 INFO:teuthology.orchestra.run.trial038.stdout: ceph-mds-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:26:52.931 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:26:52.931 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-24T18:26:52.962 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install ceph-mon-debuginfo 2026-02-24T18:26:53.281 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:09:34 ago on Tue 24 Feb 2026 06:17:19 PM UTC. 2026-02-24T18:26:53.363 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-24T18:26:53.364 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-24T18:26:53.364 INFO:teuthology.orchestra.run.trial038.stdout: Package Arch Version Repo Size 2026-02-24T18:26:53.364 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-24T18:26:53.364 INFO:teuthology.orchestra.run.trial038.stdout:Installing: 2026-02-24T18:26:53.364 INFO:teuthology.orchestra.run.trial038.stdout: ceph-mon-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 102 M 2026-02-24T18:26:53.364 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:26:53.364 INFO:teuthology.orchestra.run.trial038.stdout:Transaction Summary 2026-02-24T18:26:53.364 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-24T18:26:53.364 INFO:teuthology.orchestra.run.trial038.stdout:Install 1 Package 2026-02-24T18:26:53.364 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:26:53.364 INFO:teuthology.orchestra.run.trial038.stdout:Total download size: 102 M 2026-02-24T18:26:53.364 INFO:teuthology.orchestra.run.trial038.stdout:Installed size: 497 M 2026-02-24T18:26:53.364 INFO:teuthology.orchestra.run.trial038.stdout:Downloading Packages: 2026-02-24T18:27:05.868 INFO:teuthology.orchestra.run.trial038.stdout:ceph-mon-debuginfo-19.2.3-689.gf90f4eae.el9.x86 8.1 MB/s | 102 MB 00:12 2026-02-24T18:27:05.869 INFO:teuthology.orchestra.run.trial038.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:27:05.869 INFO:teuthology.orchestra.run.trial038.stdout:Total 8.1 MB/s | 102 MB 00:12 2026-02-24T18:27:05.869 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction check 2026-02-24T18:27:05.871 INFO:teuthology.orchestra.run.trial038.stdout:Transaction check succeeded. 2026-02-24T18:27:05.871 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction test 2026-02-24T18:27:06.187 INFO:teuthology.orchestra.run.trial038.stdout:Transaction test succeeded. 2026-02-24T18:27:06.187 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction 2026-02-24T18:27:11.392 INFO:teuthology.orchestra.run.trial038.stdout: Preparing : 1/1 2026-02-24T18:27:11.410 INFO:teuthology.orchestra.run.trial038.stdout: Installing : ceph-mon-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:27:11.829 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: ceph-mon-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:27:11.893 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : ceph-mon-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:27:11.893 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:27:11.893 INFO:teuthology.orchestra.run.trial038.stdout:Installed: 2026-02-24T18:27:11.894 INFO:teuthology.orchestra.run.trial038.stdout: ceph-mon-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:27:11.894 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:27:11.894 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-24T18:27:11.929 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install ceph-osd-debuginfo 2026-02-24T18:27:12.249 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:09:53 ago on Tue 24 Feb 2026 06:17:19 PM UTC. 2026-02-24T18:27:12.334 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-24T18:27:12.334 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-24T18:27:12.334 INFO:teuthology.orchestra.run.trial038.stdout: Package Arch Version Repo Size 2026-02-24T18:27:12.334 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-24T18:27:12.334 INFO:teuthology.orchestra.run.trial038.stdout:Installing: 2026-02-24T18:27:12.334 INFO:teuthology.orchestra.run.trial038.stdout: ceph-osd-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 298 M 2026-02-24T18:27:12.334 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:27:12.334 INFO:teuthology.orchestra.run.trial038.stdout:Transaction Summary 2026-02-24T18:27:12.335 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-24T18:27:12.335 INFO:teuthology.orchestra.run.trial038.stdout:Install 1 Package 2026-02-24T18:27:12.335 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:27:12.335 INFO:teuthology.orchestra.run.trial038.stdout:Total download size: 298 M 2026-02-24T18:27:12.335 INFO:teuthology.orchestra.run.trial038.stdout:Installed size: 1.4 G 2026-02-24T18:27:12.335 INFO:teuthology.orchestra.run.trial038.stdout:Downloading Packages: 2026-02-24T18:28:38.449 INFO:teuthology.orchestra.run.trial038.stdout:ceph-osd-debuginfo-19.2.3-689.gf90f4eae.el9.x86 3.5 MB/s | 298 MB 01:26 2026-02-24T18:28:38.450 INFO:teuthology.orchestra.run.trial038.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:28:38.450 INFO:teuthology.orchestra.run.trial038.stdout:Total 3.5 MB/s | 298 MB 01:26 2026-02-24T18:28:38.450 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction check 2026-02-24T18:28:38.450 INFO:teuthology.orchestra.run.trial038.stdout:Transaction check succeeded. 2026-02-24T18:28:38.451 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction test 2026-02-24T18:28:39.341 INFO:teuthology.orchestra.run.trial038.stdout:Transaction test succeeded. 2026-02-24T18:28:39.342 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction 2026-02-24T18:28:54.358 INFO:teuthology.orchestra.run.trial038.stdout: Preparing : 1/1 2026-02-24T18:28:54.377 INFO:teuthology.orchestra.run.trial038.stdout: Installing : ceph-osd-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:28:55.406 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: ceph-osd-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:28:55.469 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : ceph-osd-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:28:55.470 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:28:55.470 INFO:teuthology.orchestra.run.trial038.stdout:Installed: 2026-02-24T18:28:55.470 INFO:teuthology.orchestra.run.trial038.stdout: ceph-osd-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:28:55.470 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:28:55.470 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-24T18:28:55.522 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install ceph-fuse-debuginfo 2026-02-24T18:28:55.841 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:11:36 ago on Tue 24 Feb 2026 06:17:19 PM UTC. 2026-02-24T18:28:55.924 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-24T18:28:55.925 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-24T18:28:55.925 INFO:teuthology.orchestra.run.trial038.stdout: Package Arch Version Repo Size 2026-02-24T18:28:55.925 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-24T18:28:55.925 INFO:teuthology.orchestra.run.trial038.stdout:Installing: 2026-02-24T18:28:55.926 INFO:teuthology.orchestra.run.trial038.stdout: ceph-fuse-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 6.8 M 2026-02-24T18:28:55.926 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:28:55.926 INFO:teuthology.orchestra.run.trial038.stdout:Transaction Summary 2026-02-24T18:28:55.926 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-24T18:28:55.926 INFO:teuthology.orchestra.run.trial038.stdout:Install 1 Package 2026-02-24T18:28:55.926 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:28:55.926 INFO:teuthology.orchestra.run.trial038.stdout:Total download size: 6.8 M 2026-02-24T18:28:55.926 INFO:teuthology.orchestra.run.trial038.stdout:Installed size: 36 M 2026-02-24T18:28:55.926 INFO:teuthology.orchestra.run.trial038.stdout:Downloading Packages: 2026-02-24T18:29:04.691 INFO:teuthology.orchestra.run.trial038.stdout:ceph-fuse-debuginfo-19.2.3-689.gf90f4eae.el9.x8 799 kB/s | 6.8 MB 00:08 2026-02-24T18:29:04.691 INFO:teuthology.orchestra.run.trial038.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:29:04.692 INFO:teuthology.orchestra.run.trial038.stdout:Total 799 kB/s | 6.8 MB 00:08 2026-02-24T18:29:04.692 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction check 2026-02-24T18:29:04.694 INFO:teuthology.orchestra.run.trial038.stdout:Transaction check succeeded. 2026-02-24T18:29:04.694 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction test 2026-02-24T18:29:04.732 INFO:teuthology.orchestra.run.trial038.stdout:Transaction test succeeded. 2026-02-24T18:29:04.732 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction 2026-02-24T18:29:05.126 INFO:teuthology.orchestra.run.trial038.stdout: Preparing : 1/1 2026-02-24T18:29:05.144 INFO:teuthology.orchestra.run.trial038.stdout: Installing : ceph-fuse-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_6 1/1 2026-02-24T18:29:05.236 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: ceph-fuse-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_6 1/1 2026-02-24T18:29:05.299 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : ceph-fuse-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_6 1/1 2026-02-24T18:29:05.299 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:29:05.299 INFO:teuthology.orchestra.run.trial038.stdout:Installed: 2026-02-24T18:29:05.299 INFO:teuthology.orchestra.run.trial038.stdout: ceph-fuse-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:29:05.299 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:29:05.299 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-24T18:29:05.326 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install librados-devel-debuginfo 2026-02-24T18:29:05.640 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:11:46 ago on Tue 24 Feb 2026 06:17:19 PM UTC. 2026-02-24T18:29:05.724 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-24T18:29:05.724 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-24T18:29:05.724 INFO:teuthology.orchestra.run.trial038.stdout: Package Arch Version Repo Size 2026-02-24T18:29:05.724 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-24T18:29:05.724 INFO:teuthology.orchestra.run.trial038.stdout:Installing: 2026-02-24T18:29:05.724 INFO:teuthology.orchestra.run.trial038.stdout: librados-devel-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 112 k 2026-02-24T18:29:05.724 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:29:05.724 INFO:teuthology.orchestra.run.trial038.stdout:Transaction Summary 2026-02-24T18:29:05.724 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-24T18:29:05.724 INFO:teuthology.orchestra.run.trial038.stdout:Install 1 Package 2026-02-24T18:29:05.724 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:29:05.724 INFO:teuthology.orchestra.run.trial038.stdout:Total download size: 112 k 2026-02-24T18:29:05.724 INFO:teuthology.orchestra.run.trial038.stdout:Installed size: 493 k 2026-02-24T18:29:05.724 INFO:teuthology.orchestra.run.trial038.stdout:Downloading Packages: 2026-02-24T18:29:06.070 INFO:teuthology.orchestra.run.trial038.stdout:librados-devel-debuginfo-19.2.3-689.gf90f4eae.e 325 kB/s | 112 kB 00:00 2026-02-24T18:29:06.070 INFO:teuthology.orchestra.run.trial038.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:29:06.071 INFO:teuthology.orchestra.run.trial038.stdout:Total 324 kB/s | 112 kB 00:00 2026-02-24T18:29:06.071 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction check 2026-02-24T18:29:06.073 INFO:teuthology.orchestra.run.trial038.stdout:Transaction check succeeded. 2026-02-24T18:29:06.073 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction test 2026-02-24T18:29:06.092 INFO:teuthology.orchestra.run.trial038.stdout:Transaction test succeeded. 2026-02-24T18:29:06.092 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction 2026-02-24T18:29:06.135 INFO:teuthology.orchestra.run.trial038.stdout: Preparing : 1/1 2026-02-24T18:29:06.154 INFO:teuthology.orchestra.run.trial038.stdout: Installing : librados-devel-debuginfo-2:19.2.3-689.gf90f4eae.el9. 1/1 2026-02-24T18:29:06.216 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: librados-devel-debuginfo-2:19.2.3-689.gf90f4eae.el9. 1/1 2026-02-24T18:29:06.280 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : librados-devel-debuginfo-2:19.2.3-689.gf90f4eae.el9. 1/1 2026-02-24T18:29:06.280 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:29:06.280 INFO:teuthology.orchestra.run.trial038.stdout:Installed: 2026-02-24T18:29:06.280 INFO:teuthology.orchestra.run.trial038.stdout: librados-devel-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:29:06.280 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:29:06.281 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-24T18:29:06.306 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install libcephfs2-debuginfo 2026-02-24T18:29:06.628 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:11:47 ago on Tue 24 Feb 2026 06:17:19 PM UTC. 2026-02-24T18:29:06.713 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-24T18:29:06.713 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-24T18:29:06.714 INFO:teuthology.orchestra.run.trial038.stdout: Package Arch Version Repo Size 2026-02-24T18:29:06.714 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-24T18:29:06.714 INFO:teuthology.orchestra.run.trial038.stdout:Installing: 2026-02-24T18:29:06.714 INFO:teuthology.orchestra.run.trial038.stdout: libcephfs2-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 6.0 M 2026-02-24T18:29:06.714 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:29:06.714 INFO:teuthology.orchestra.run.trial038.stdout:Transaction Summary 2026-02-24T18:29:06.714 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-24T18:29:06.714 INFO:teuthology.orchestra.run.trial038.stdout:Install 1 Package 2026-02-24T18:29:06.714 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:29:06.715 INFO:teuthology.orchestra.run.trial038.stdout:Total download size: 6.0 M 2026-02-24T18:29:06.715 INFO:teuthology.orchestra.run.trial038.stdout:Installed size: 31 M 2026-02-24T18:29:06.715 INFO:teuthology.orchestra.run.trial038.stdout:Downloading Packages: 2026-02-24T18:29:09.166 INFO:teuthology.orchestra.run.trial038.stdout:libcephfs2-debuginfo-19.2.3-689.gf90f4eae.el9.x 2.4 MB/s | 6.0 MB 00:02 2026-02-24T18:29:09.166 INFO:teuthology.orchestra.run.trial038.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:29:09.166 INFO:teuthology.orchestra.run.trial038.stdout:Total 2.4 MB/s | 6.0 MB 00:02 2026-02-24T18:29:09.166 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction check 2026-02-24T18:29:09.168 INFO:teuthology.orchestra.run.trial038.stdout:Transaction check succeeded. 2026-02-24T18:29:09.168 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction test 2026-02-24T18:29:09.204 INFO:teuthology.orchestra.run.trial038.stdout:Transaction test succeeded. 2026-02-24T18:29:09.204 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction 2026-02-24T18:29:09.550 INFO:teuthology.orchestra.run.trial038.stdout: Preparing : 1/1 2026-02-24T18:29:09.569 INFO:teuthology.orchestra.run.trial038.stdout: Installing : libcephfs2-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_ 1/1 2026-02-24T18:29:09.652 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: libcephfs2-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_ 1/1 2026-02-24T18:29:09.716 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : libcephfs2-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_ 1/1 2026-02-24T18:29:09.716 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:29:09.717 INFO:teuthology.orchestra.run.trial038.stdout:Installed: 2026-02-24T18:29:09.717 INFO:teuthology.orchestra.run.trial038.stdout: libcephfs2-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:29:09.717 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:29:09.717 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-24T18:29:09.745 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install librados2-debuginfo 2026-02-24T18:29:10.063 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:11:51 ago on Tue 24 Feb 2026 06:17:19 PM UTC. 2026-02-24T18:29:10.147 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-24T18:29:10.148 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-24T18:29:10.148 INFO:teuthology.orchestra.run.trial038.stdout: Package Arch Version Repo Size 2026-02-24T18:29:10.148 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-24T18:29:10.148 INFO:teuthology.orchestra.run.trial038.stdout:Installing: 2026-02-24T18:29:10.148 INFO:teuthology.orchestra.run.trial038.stdout: librados2-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 47 M 2026-02-24T18:29:10.148 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:29:10.148 INFO:teuthology.orchestra.run.trial038.stdout:Transaction Summary 2026-02-24T18:29:10.148 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-24T18:29:10.148 INFO:teuthology.orchestra.run.trial038.stdout:Install 1 Package 2026-02-24T18:29:10.149 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:29:10.149 INFO:teuthology.orchestra.run.trial038.stdout:Total download size: 47 M 2026-02-24T18:29:10.149 INFO:teuthology.orchestra.run.trial038.stdout:Installed size: 302 M 2026-02-24T18:29:10.149 INFO:teuthology.orchestra.run.trial038.stdout:Downloading Packages: 2026-02-24T18:29:12.114 INFO:teuthology.orchestra.run.trial038.stdout:librados2-debuginfo-19.2.3-689.gf90f4eae.el9.x8 24 MB/s | 47 MB 00:01 2026-02-24T18:29:12.115 INFO:teuthology.orchestra.run.trial038.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:29:12.115 INFO:teuthology.orchestra.run.trial038.stdout:Total 24 MB/s | 47 MB 00:01 2026-02-24T18:29:12.115 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction check 2026-02-24T18:29:12.117 INFO:teuthology.orchestra.run.trial038.stdout:Transaction check succeeded. 2026-02-24T18:29:12.117 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction test 2026-02-24T18:29:12.277 INFO:teuthology.orchestra.run.trial038.stdout:Transaction test succeeded. 2026-02-24T18:29:12.277 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction 2026-02-24T18:29:15.014 INFO:teuthology.orchestra.run.trial038.stdout: Preparing : 1/1 2026-02-24T18:29:15.033 INFO:teuthology.orchestra.run.trial038.stdout: Installing : librados2-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_6 1/1 2026-02-24T18:29:15.300 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: librados2-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_6 1/1 2026-02-24T18:29:15.363 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : librados2-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_6 1/1 2026-02-24T18:29:15.363 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:29:15.364 INFO:teuthology.orchestra.run.trial038.stdout:Installed: 2026-02-24T18:29:15.364 INFO:teuthology.orchestra.run.trial038.stdout: librados2-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:29:15.364 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:29:15.364 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-24T18:29:15.394 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install librbd1-debuginfo 2026-02-24T18:29:15.709 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:11:56 ago on Tue 24 Feb 2026 06:17:19 PM UTC. 2026-02-24T18:29:15.793 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-24T18:29:15.794 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-24T18:29:15.794 INFO:teuthology.orchestra.run.trial038.stdout: Package Arch Version Repo Size 2026-02-24T18:29:15.794 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-24T18:29:15.794 INFO:teuthology.orchestra.run.trial038.stdout:Installing: 2026-02-24T18:29:15.794 INFO:teuthology.orchestra.run.trial038.stdout: librbd1-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 55 M 2026-02-24T18:29:15.794 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:29:15.794 INFO:teuthology.orchestra.run.trial038.stdout:Transaction Summary 2026-02-24T18:29:15.794 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-24T18:29:15.794 INFO:teuthology.orchestra.run.trial038.stdout:Install 1 Package 2026-02-24T18:29:15.795 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:29:15.795 INFO:teuthology.orchestra.run.trial038.stdout:Total download size: 55 M 2026-02-24T18:29:15.795 INFO:teuthology.orchestra.run.trial038.stdout:Installed size: 374 M 2026-02-24T18:29:15.795 INFO:teuthology.orchestra.run.trial038.stdout:Downloading Packages: 2026-02-24T18:29:19.057 INFO:teuthology.orchestra.run.trial038.stdout:librbd1-debuginfo-19.2.3-689.gf90f4eae.el9.x86_ 17 MB/s | 55 MB 00:03 2026-02-24T18:29:19.058 INFO:teuthology.orchestra.run.trial038.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:29:19.058 INFO:teuthology.orchestra.run.trial038.stdout:Total 17 MB/s | 55 MB 00:03 2026-02-24T18:29:19.058 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction check 2026-02-24T18:29:19.059 INFO:teuthology.orchestra.run.trial038.stdout:Transaction check succeeded. 2026-02-24T18:29:19.059 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction test 2026-02-24T18:29:19.240 INFO:teuthology.orchestra.run.trial038.stdout:Transaction test succeeded. 2026-02-24T18:29:19.241 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction 2026-02-24T18:29:22.482 INFO:teuthology.orchestra.run.trial038.stdout: Preparing : 1/1 2026-02-24T18:29:22.500 INFO:teuthology.orchestra.run.trial038.stdout: Installing : librbd1-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:29:22.824 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: librbd1-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:29:22.888 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : librbd1-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:29:22.888 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:29:22.888 INFO:teuthology.orchestra.run.trial038.stdout:Installed: 2026-02-24T18:29:22.888 INFO:teuthology.orchestra.run.trial038.stdout: librbd1-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:29:22.889 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:29:22.889 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-24T18:29:22.920 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install python3-cephfs-debuginfo 2026-02-24T18:29:23.236 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:12:04 ago on Tue 24 Feb 2026 06:17:19 PM UTC. 2026-02-24T18:29:23.321 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-24T18:29:23.322 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-24T18:29:23.322 INFO:teuthology.orchestra.run.trial038.stdout: Package Arch Version Repo Size 2026-02-24T18:29:23.323 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-24T18:29:23.323 INFO:teuthology.orchestra.run.trial038.stdout:Installing: 2026-02-24T18:29:23.323 INFO:teuthology.orchestra.run.trial038.stdout: python3-cephfs-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 628 k 2026-02-24T18:29:23.323 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:29:23.323 INFO:teuthology.orchestra.run.trial038.stdout:Transaction Summary 2026-02-24T18:29:23.323 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-24T18:29:23.323 INFO:teuthology.orchestra.run.trial038.stdout:Install 1 Package 2026-02-24T18:29:23.323 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:29:23.323 INFO:teuthology.orchestra.run.trial038.stdout:Total download size: 628 k 2026-02-24T18:29:23.323 INFO:teuthology.orchestra.run.trial038.stdout:Installed size: 2.1 M 2026-02-24T18:29:23.324 INFO:teuthology.orchestra.run.trial038.stdout:Downloading Packages: 2026-02-24T18:29:23.922 INFO:teuthology.orchestra.run.trial038.stdout:python3-cephfs-debuginfo-19.2.3-689.gf90f4eae.e 1.0 MB/s | 628 kB 00:00 2026-02-24T18:29:23.922 INFO:teuthology.orchestra.run.trial038.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:29:23.922 INFO:teuthology.orchestra.run.trial038.stdout:Total 1.0 MB/s | 628 kB 00:00 2026-02-24T18:29:23.923 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction check 2026-02-24T18:29:23.924 INFO:teuthology.orchestra.run.trial038.stdout:Transaction check succeeded. 2026-02-24T18:29:23.924 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction test 2026-02-24T18:29:23.944 INFO:teuthology.orchestra.run.trial038.stdout:Transaction test succeeded. 2026-02-24T18:29:23.944 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction 2026-02-24T18:29:24.009 INFO:teuthology.orchestra.run.trial038.stdout: Preparing : 1/1 2026-02-24T18:29:24.027 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-cephfs-debuginfo-2:19.2.3-689.gf90f4eae.el9. 1/1 2026-02-24T18:29:24.087 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: python3-cephfs-debuginfo-2:19.2.3-689.gf90f4eae.el9. 1/1 2026-02-24T18:29:24.152 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-cephfs-debuginfo-2:19.2.3-689.gf90f4eae.el9. 1/1 2026-02-24T18:29:24.152 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:29:24.152 INFO:teuthology.orchestra.run.trial038.stdout:Installed: 2026-02-24T18:29:24.152 INFO:teuthology.orchestra.run.trial038.stdout: python3-cephfs-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:29:24.152 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:29:24.152 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-24T18:29:24.180 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install python3-rados-debuginfo 2026-02-24T18:29:24.496 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:12:05 ago on Tue 24 Feb 2026 06:17:19 PM UTC. 2026-02-24T18:29:24.580 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-24T18:29:24.581 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-24T18:29:24.581 INFO:teuthology.orchestra.run.trial038.stdout: Package Arch Version Repo Size 2026-02-24T18:29:24.581 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-24T18:29:24.581 INFO:teuthology.orchestra.run.trial038.stdout:Installing: 2026-02-24T18:29:24.581 INFO:teuthology.orchestra.run.trial038.stdout: python3-rados-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 779 k 2026-02-24T18:29:24.581 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:29:24.581 INFO:teuthology.orchestra.run.trial038.stdout:Transaction Summary 2026-02-24T18:29:24.581 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-24T18:29:24.581 INFO:teuthology.orchestra.run.trial038.stdout:Install 1 Package 2026-02-24T18:29:24.581 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:29:24.582 INFO:teuthology.orchestra.run.trial038.stdout:Total download size: 779 k 2026-02-24T18:29:24.582 INFO:teuthology.orchestra.run.trial038.stdout:Installed size: 2.8 M 2026-02-24T18:29:24.582 INFO:teuthology.orchestra.run.trial038.stdout:Downloading Packages: 2026-02-24T18:29:25.154 INFO:teuthology.orchestra.run.trial038.stdout:python3-rados-debuginfo-19.2.3-689.gf90f4eae.el 1.3 MB/s | 779 kB 00:00 2026-02-24T18:29:25.154 INFO:teuthology.orchestra.run.trial038.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:29:25.154 INFO:teuthology.orchestra.run.trial038.stdout:Total 1.3 MB/s | 779 kB 00:00 2026-02-24T18:29:25.154 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction check 2026-02-24T18:29:25.155 INFO:teuthology.orchestra.run.trial038.stdout:Transaction check succeeded. 2026-02-24T18:29:25.156 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction test 2026-02-24T18:29:25.175 INFO:teuthology.orchestra.run.trial038.stdout:Transaction test succeeded. 2026-02-24T18:29:25.176 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction 2026-02-24T18:29:25.247 INFO:teuthology.orchestra.run.trial038.stdout: Preparing : 1/1 2026-02-24T18:29:25.265 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-rados-debuginfo-2:19.2.3-689.gf90f4eae.el9.x 1/1 2026-02-24T18:29:25.326 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: python3-rados-debuginfo-2:19.2.3-689.gf90f4eae.el9.x 1/1 2026-02-24T18:29:25.389 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-rados-debuginfo-2:19.2.3-689.gf90f4eae.el9.x 1/1 2026-02-24T18:29:25.389 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:29:25.389 INFO:teuthology.orchestra.run.trial038.stdout:Installed: 2026-02-24T18:29:25.390 INFO:teuthology.orchestra.run.trial038.stdout: python3-rados-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:29:25.390 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:29:25.390 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-24T18:29:25.415 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install python3-rbd-debuginfo 2026-02-24T18:29:25.731 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:12:06 ago on Tue 24 Feb 2026 06:17:19 PM UTC. 2026-02-24T18:29:25.814 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-24T18:29:25.815 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-24T18:29:25.815 INFO:teuthology.orchestra.run.trial038.stdout: Package Arch Version Repo Size 2026-02-24T18:29:25.815 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-24T18:29:25.815 INFO:teuthology.orchestra.run.trial038.stdout:Installing: 2026-02-24T18:29:25.815 INFO:teuthology.orchestra.run.trial038.stdout: python3-rbd-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 1.1 M 2026-02-24T18:29:25.816 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:29:25.816 INFO:teuthology.orchestra.run.trial038.stdout:Transaction Summary 2026-02-24T18:29:25.816 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-24T18:29:25.816 INFO:teuthology.orchestra.run.trial038.stdout:Install 1 Package 2026-02-24T18:29:25.816 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:29:25.816 INFO:teuthology.orchestra.run.trial038.stdout:Total download size: 1.1 M 2026-02-24T18:29:25.816 INFO:teuthology.orchestra.run.trial038.stdout:Installed size: 4.3 M 2026-02-24T18:29:25.816 INFO:teuthology.orchestra.run.trial038.stdout:Downloading Packages: 2026-02-24T18:29:26.429 INFO:teuthology.orchestra.run.trial038.stdout:python3-rbd-debuginfo-19.2.3-689.gf90f4eae.el9. 1.9 MB/s | 1.1 MB 00:00 2026-02-24T18:29:26.430 INFO:teuthology.orchestra.run.trial038.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:29:26.430 INFO:teuthology.orchestra.run.trial038.stdout:Total 1.8 MB/s | 1.1 MB 00:00 2026-02-24T18:29:26.430 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction check 2026-02-24T18:29:26.431 INFO:teuthology.orchestra.run.trial038.stdout:Transaction check succeeded. 2026-02-24T18:29:26.431 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction test 2026-02-24T18:29:26.455 INFO:teuthology.orchestra.run.trial038.stdout:Transaction test succeeded. 2026-02-24T18:29:26.456 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction 2026-02-24T18:29:26.549 INFO:teuthology.orchestra.run.trial038.stdout: Preparing : 1/1 2026-02-24T18:29:26.567 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-rbd-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86 1/1 2026-02-24T18:29:26.630 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: python3-rbd-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86 1/1 2026-02-24T18:29:26.695 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-rbd-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86 1/1 2026-02-24T18:29:26.695 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:29:26.696 INFO:teuthology.orchestra.run.trial038.stdout:Installed: 2026-02-24T18:29:26.696 INFO:teuthology.orchestra.run.trial038.stdout: python3-rbd-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:29:26.696 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:29:26.696 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-24T18:29:26.722 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install python3-rgw-debuginfo 2026-02-24T18:29:27.045 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:12:08 ago on Tue 24 Feb 2026 06:17:19 PM UTC. 2026-02-24T18:29:27.130 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-24T18:29:27.131 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-24T18:29:27.131 INFO:teuthology.orchestra.run.trial038.stdout: Package Arch Version Repo Size 2026-02-24T18:29:27.131 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-24T18:29:27.131 INFO:teuthology.orchestra.run.trial038.stdout:Installing: 2026-02-24T18:29:27.132 INFO:teuthology.orchestra.run.trial038.stdout: python3-rgw-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 339 k 2026-02-24T18:29:27.132 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:29:27.132 INFO:teuthology.orchestra.run.trial038.stdout:Transaction Summary 2026-02-24T18:29:27.132 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-24T18:29:27.132 INFO:teuthology.orchestra.run.trial038.stdout:Install 1 Package 2026-02-24T18:29:27.132 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:29:27.132 INFO:teuthology.orchestra.run.trial038.stdout:Total download size: 339 k 2026-02-24T18:29:27.132 INFO:teuthology.orchestra.run.trial038.stdout:Installed size: 1.0 M 2026-02-24T18:29:27.132 INFO:teuthology.orchestra.run.trial038.stdout:Downloading Packages: 2026-02-24T18:29:27.644 INFO:teuthology.orchestra.run.trial038.stdout:python3-rgw-debuginfo-19.2.3-689.gf90f4eae.el9. 660 kB/s | 339 kB 00:00 2026-02-24T18:29:27.645 INFO:teuthology.orchestra.run.trial038.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:29:27.645 INFO:teuthology.orchestra.run.trial038.stdout:Total 659 kB/s | 339 kB 00:00 2026-02-24T18:29:27.645 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction check 2026-02-24T18:29:27.647 INFO:teuthology.orchestra.run.trial038.stdout:Transaction check succeeded. 2026-02-24T18:29:27.648 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction test 2026-02-24T18:29:27.666 INFO:teuthology.orchestra.run.trial038.stdout:Transaction test succeeded. 2026-02-24T18:29:27.666 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction 2026-02-24T18:29:27.716 INFO:teuthology.orchestra.run.trial038.stdout: Preparing : 1/1 2026-02-24T18:29:27.734 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-rgw-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86 1/1 2026-02-24T18:29:27.795 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: python3-rgw-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86 1/1 2026-02-24T18:29:27.858 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-rgw-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86 1/1 2026-02-24T18:29:27.858 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:29:27.858 INFO:teuthology.orchestra.run.trial038.stdout:Installed: 2026-02-24T18:29:27.858 INFO:teuthology.orchestra.run.trial038.stdout: python3-rgw-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:29:27.858 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:29:27.858 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-24T18:29:27.886 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install rbd-fuse-debuginfo 2026-02-24T18:29:28.203 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:12:09 ago on Tue 24 Feb 2026 06:17:19 PM UTC. 2026-02-24T18:29:28.287 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-24T18:29:28.288 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-24T18:29:28.288 INFO:teuthology.orchestra.run.trial038.stdout: Package Arch Version Repo Size 2026-02-24T18:29:28.288 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-24T18:29:28.289 INFO:teuthology.orchestra.run.trial038.stdout:Installing: 2026-02-24T18:29:28.289 INFO:teuthology.orchestra.run.trial038.stdout: rbd-fuse-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 547 k 2026-02-24T18:29:28.289 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:29:28.289 INFO:teuthology.orchestra.run.trial038.stdout:Transaction Summary 2026-02-24T18:29:28.289 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-24T18:29:28.289 INFO:teuthology.orchestra.run.trial038.stdout:Install 1 Package 2026-02-24T18:29:28.289 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:29:28.289 INFO:teuthology.orchestra.run.trial038.stdout:Total download size: 547 k 2026-02-24T18:29:28.289 INFO:teuthology.orchestra.run.trial038.stdout:Installed size: 3.0 M 2026-02-24T18:29:28.289 INFO:teuthology.orchestra.run.trial038.stdout:Downloading Packages: 2026-02-24T18:29:28.847 INFO:teuthology.orchestra.run.trial038.stdout:rbd-fuse-debuginfo-19.2.3-689.gf90f4eae.el9.x86 981 kB/s | 547 kB 00:00 2026-02-24T18:29:28.847 INFO:teuthology.orchestra.run.trial038.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:29:28.847 INFO:teuthology.orchestra.run.trial038.stdout:Total 979 kB/s | 547 kB 00:00 2026-02-24T18:29:28.847 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction check 2026-02-24T18:29:28.849 INFO:teuthology.orchestra.run.trial038.stdout:Transaction check succeeded. 2026-02-24T18:29:28.849 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction test 2026-02-24T18:29:28.869 INFO:teuthology.orchestra.run.trial038.stdout:Transaction test succeeded. 2026-02-24T18:29:28.869 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction 2026-02-24T18:29:28.933 INFO:teuthology.orchestra.run.trial038.stdout: Preparing : 1/1 2026-02-24T18:29:28.951 INFO:teuthology.orchestra.run.trial038.stdout: Installing : rbd-fuse-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:29:29.018 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: rbd-fuse-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:29:29.083 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : rbd-fuse-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:29:29.084 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:29:29.084 INFO:teuthology.orchestra.run.trial038.stdout:Installed: 2026-02-24T18:29:29.084 INFO:teuthology.orchestra.run.trial038.stdout: rbd-fuse-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:29:29.084 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:29:29.084 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-24T18:29:29.112 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install rbd-mirror-debuginfo 2026-02-24T18:29:29.436 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:12:10 ago on Tue 24 Feb 2026 06:17:19 PM UTC. 2026-02-24T18:29:29.520 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-24T18:29:29.520 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-24T18:29:29.520 INFO:teuthology.orchestra.run.trial038.stdout: Package Arch Version Repo Size 2026-02-24T18:29:29.520 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-24T18:29:29.520 INFO:teuthology.orchestra.run.trial038.stdout:Installing: 2026-02-24T18:29:29.521 INFO:teuthology.orchestra.run.trial038.stdout: rbd-mirror-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 54 M 2026-02-24T18:29:29.521 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:29:29.521 INFO:teuthology.orchestra.run.trial038.stdout:Transaction Summary 2026-02-24T18:29:29.521 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-24T18:29:29.521 INFO:teuthology.orchestra.run.trial038.stdout:Install 1 Package 2026-02-24T18:29:29.521 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:29:29.521 INFO:teuthology.orchestra.run.trial038.stdout:Total download size: 54 M 2026-02-24T18:29:29.521 INFO:teuthology.orchestra.run.trial038.stdout:Installed size: 353 M 2026-02-24T18:29:29.521 INFO:teuthology.orchestra.run.trial038.stdout:Downloading Packages: 2026-02-24T18:29:50.228 INFO:teuthology.orchestra.run.trial038.stdout:rbd-mirror-debuginfo-19.2.3-689.gf90f4eae.el9.x 2.6 MB/s | 54 MB 00:20 2026-02-24T18:29:50.229 INFO:teuthology.orchestra.run.trial038.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:29:50.229 INFO:teuthology.orchestra.run.trial038.stdout:Total 2.6 MB/s | 54 MB 00:20 2026-02-24T18:29:50.229 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction check 2026-02-24T18:29:50.231 INFO:teuthology.orchestra.run.trial038.stdout:Transaction check succeeded. 2026-02-24T18:29:50.231 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction test 2026-02-24T18:29:50.408 INFO:teuthology.orchestra.run.trial038.stdout:Transaction test succeeded. 2026-02-24T18:29:50.409 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction 2026-02-24T18:29:53.457 INFO:teuthology.orchestra.run.trial038.stdout: Preparing : 1/1 2026-02-24T18:29:53.475 INFO:teuthology.orchestra.run.trial038.stdout: Installing : rbd-mirror-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_ 1/1 2026-02-24T18:29:53.780 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: rbd-mirror-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_ 1/1 2026-02-24T18:29:53.844 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : rbd-mirror-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_ 1/1 2026-02-24T18:29:53.844 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:29:53.844 INFO:teuthology.orchestra.run.trial038.stdout:Installed: 2026-02-24T18:29:53.844 INFO:teuthology.orchestra.run.trial038.stdout: rbd-mirror-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:29:53.844 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:29:53.844 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-24T18:29:53.876 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install rbd-nbd-debuginfo 2026-02-24T18:29:54.194 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:12:35 ago on Tue 24 Feb 2026 06:17:19 PM UTC. 2026-02-24T18:29:54.280 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-24T18:29:54.280 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-24T18:29:54.280 INFO:teuthology.orchestra.run.trial038.stdout: Package Arch Version Repo Size 2026-02-24T18:29:54.280 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-24T18:29:54.280 INFO:teuthology.orchestra.run.trial038.stdout:Installing: 2026-02-24T18:29:54.280 INFO:teuthology.orchestra.run.trial038.stdout: rbd-nbd-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 1.4 M 2026-02-24T18:29:54.280 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:29:54.281 INFO:teuthology.orchestra.run.trial038.stdout:Transaction Summary 2026-02-24T18:29:54.281 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-24T18:29:54.281 INFO:teuthology.orchestra.run.trial038.stdout:Install 1 Package 2026-02-24T18:29:54.281 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:29:54.281 INFO:teuthology.orchestra.run.trial038.stdout:Total download size: 1.4 M 2026-02-24T18:29:54.281 INFO:teuthology.orchestra.run.trial038.stdout:Installed size: 6.6 M 2026-02-24T18:29:54.281 INFO:teuthology.orchestra.run.trial038.stdout:Downloading Packages: 2026-02-24T18:29:55.591 INFO:teuthology.orchestra.run.trial038.stdout:rbd-nbd-debuginfo-19.2.3-689.gf90f4eae.el9.x86_ 1.0 MB/s | 1.4 MB 00:01 2026-02-24T18:29:55.591 INFO:teuthology.orchestra.run.trial038.stdout:-------------------------------------------------------------------------------- 2026-02-24T18:29:55.591 INFO:teuthology.orchestra.run.trial038.stdout:Total 1.0 MB/s | 1.4 MB 00:01 2026-02-24T18:29:55.591 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction check 2026-02-24T18:29:55.593 INFO:teuthology.orchestra.run.trial038.stdout:Transaction check succeeded. 2026-02-24T18:29:55.593 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction test 2026-02-24T18:29:55.615 INFO:teuthology.orchestra.run.trial038.stdout:Transaction test succeeded. 2026-02-24T18:29:55.615 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction 2026-02-24T18:29:55.721 INFO:teuthology.orchestra.run.trial038.stdout: Preparing : 1/1 2026-02-24T18:29:55.740 INFO:teuthology.orchestra.run.trial038.stdout: Installing : rbd-nbd-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:29:55.805 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: rbd-nbd-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:29:55.870 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : rbd-nbd-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 1/1 2026-02-24T18:29:55.870 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:29:55.870 INFO:teuthology.orchestra.run.trial038.stdout:Installed: 2026-02-24T18:29:55.870 INFO:teuthology.orchestra.run.trial038.stdout: rbd-nbd-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-24T18:29:55.870 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:29:55.870 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-24T18:29:55.898 DEBUG:teuthology.parallel:result is None 2026-02-24T18:29:55.899 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=f90f4eaebb28b619cf76e80cbbe997656da69579 2026-02-24T18:29:56.024 DEBUG:teuthology.orchestra.run.trial038:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2026-02-24T18:29:56.045 INFO:teuthology.orchestra.run.trial038.stdout:19.2.3-689.gf90f4eae.el9 2026-02-24T18:29:56.046 INFO:teuthology.packaging:The installed version of ceph is 19.2.3-689.gf90f4eae.el9 2026-02-24T18:29:56.046 INFO:teuthology.task.install:The correct ceph version 19.2.3-689.gf90f4eae is installed. 2026-02-24T18:29:56.048 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2026-02-24T18:29:56.048 DEBUG:teuthology.orchestra.run.trial038:> set -ex 2026-02-24T18:29:56.048 DEBUG:teuthology.orchestra.run.trial038:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2026-02-24T18:29:56.109 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2026-02-24T18:29:56.109 DEBUG:teuthology.orchestra.run.trial038:> set -ex 2026-02-24T18:29:56.109 DEBUG:teuthology.orchestra.run.trial038:> sudo dd of=/usr/bin/daemon-helper 2026-02-24T18:29:56.172 DEBUG:teuthology.orchestra.run.trial038:> sudo chmod a=rx -- /usr/bin/daemon-helper 2026-02-24T18:29:56.235 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2026-02-24T18:29:56.235 DEBUG:teuthology.orchestra.run.trial038:> set -ex 2026-02-24T18:29:56.235 DEBUG:teuthology.orchestra.run.trial038:> sudo dd of=/usr/bin/adjust-ulimits 2026-02-24T18:29:56.298 DEBUG:teuthology.orchestra.run.trial038:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2026-02-24T18:29:56.362 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2026-02-24T18:29:56.362 DEBUG:teuthology.orchestra.run.trial038:> set -ex 2026-02-24T18:29:56.362 DEBUG:teuthology.orchestra.run.trial038:> sudo dd of=/usr/bin/stdin-killer 2026-02-24T18:29:56.424 DEBUG:teuthology.orchestra.run.trial038:> sudo chmod a=rx -- /usr/bin/stdin-killer 2026-02-24T18:29:56.487 INFO:teuthology.run_tasks:Running task ceph... 2026-02-24T18:29:56.574 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2026-02-24T18:29:56.574 DEBUG:teuthology.orchestra.run.trial038:> sudo chmod 777 /var/log/ceph 2026-02-24T18:29:56.599 INFO:tasks.ceph:Disabling ceph logrotate... 2026-02-24T18:29:56.599 DEBUG:teuthology.orchestra.run.trial038:> sudo rm -f -- /etc/logrotate.d/ceph 2026-02-24T18:29:56.662 INFO:tasks.ceph:Creating extra log directories... 2026-02-24T18:29:56.662 DEBUG:teuthology.orchestra.run.trial038:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2026-02-24T18:29:56.734 INFO:tasks.ceph:Creating ceph cluster ceph... 2026-02-24T18:29:56.734 INFO:tasks.ceph:config {'conf': {'global': {'osd heartbeat grace': 40, 'osd max object name len': 460, 'osd max object namespace len': 64}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon osd crush smoke test': False}, 'osd': {'debug ms': 1, 'debug osd': 20, 'osd fast shutdown': False}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(PG_', '\\(POOL_APP_NOT_ENABLED\\)', 'OSD bench result'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2026-02-24T18:29:56.734 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/mnt/teuthology/yuriw-2026-02-24_16:22:43-rados-wip-yuri6-testing-2026-02-22-1629-squid-distro-default-trial/68639', 'branch': 'wip-yuri6-testing-2026-02-22-1629-squid', 'description': 'rados/valgrind-leaks/{1-start 2-inject-leak/none centos_latest}', 'email': None, 'first_in_suite': False, 'flavor': 'default', 'job_id': '68639', 'kernel': {'branch': 'distro', 'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'trial', 'name': 'yuriw-2026-02-24_16:22:43-rados-wip-yuri6-testing-2026-02-22-1629-squid-distro-default-trial', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 2, 'size': 10}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-yuri6-testing-2026-02-22-1629-squid'}, 'ceph': {'conf': {'global': {'osd heartbeat grace': 40, 'osd max object name len': 460, 'osd max object namespace len': 64}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon osd crush smoke test': False}, 'osd': {'debug ms': 1, 'debug osd': 20, 'osd fast shutdown': False}}, 'flavor': 'default', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(PG_', '\\(POOL_APP_NOT_ENABLED\\)', 'OSD bench result'], 'sha1': 'f90f4eaebb28b619cf76e80cbbe997656da69579', 'stop-daemons-timeout': 1200, 'valgrind': {'mon': ['--tool=memcheck', '--leak-check=full', '--show-reachable=yes'], 'osd': ['--tool=memcheck']}}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'debuginfo': True, 'flavor': 'default', 'sha1': 'f90f4eaebb28b619cf76e80cbbe997656da69579'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'workunit': {'branch': 'wip-yuri6-testing-2026-02-22-1629-squid', 'sha1': 'f90f4eaebb28b619cf76e80cbbe997656da69579'}}, 'owner': 'scheduled_yuriw@soko04.front.sepia.ceph.com', 'priority': 69, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mon.b', 'mon.c', 'mgr.x', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'client.0']], 'seed': 5907, 'sha1': 'f90f4eaebb28b619cf76e80cbbe997656da69579', 'sleep_before_teardown': 0, 'subset': '111/120000', 'suite': 'rados', 'suite_branch': 'wip-yuri6-testing-2026-02-22-1629-squid', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_f90f4eaebb28b619cf76e80cbbe997656da69579/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': 'f90f4eaebb28b619cf76e80cbbe997656da69579', 'targets': {'trial038.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFs3Ils7PR9Tm15uNdKJ0W5F2g9JQYSKaSM2xRd3l2l9AKo9OmyoT62jWNp07urXvPzP8D96jrJIk/LBhfOG9Ug='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'branch': 'distro', 'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': None}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'reimage-ssh-down', 'teuthology_sha1': 'e6d9099ce03dc21193fda653159ec82465c3a94a', 'timestamp': '2026-02-24_16:22:43', 'tube': 'trial', 'user': 'yuriw', 'verbose': True, 'worker_log': '/home/teuthworker/mnt/teuthology/worker_logs/dispatcher.trial.904699'} 2026-02-24T18:29:56.734 DEBUG:teuthology.orchestra.run.trial038:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2026-02-24T18:29:56.782 DEBUG:teuthology.orchestra.run.trial038:> sudo install -d -m0777 -- /var/run/ceph 2026-02-24T18:29:56.845 DEBUG:teuthology.orchestra.run.trial038:> set -ex 2026-02-24T18:29:56.846 DEBUG:teuthology.orchestra.run.trial038:> dd if=/scratch_devs of=/dev/stdout 2026-02-24T18:29:56.902 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2026-02-24T18:29:56.902 DEBUG:teuthology.orchestra.run.trial038:> stat /dev/vg_nvme/lv_1 2026-02-24T18:29:56.959 INFO:teuthology.orchestra.run.trial038.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2026-02-24T18:29:56.959 INFO:teuthology.orchestra.run.trial038.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-24T18:29:56.960 INFO:teuthology.orchestra.run.trial038.stdout:Device: 6h/6d Inode: 1312 Links: 1 2026-02-24T18:29:56.960 INFO:teuthology.orchestra.run.trial038.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-24T18:29:56.960 INFO:teuthology.orchestra.run.trial038.stdout:Context: system_u:object_r:device_t:s0 2026-02-24T18:29:56.960 INFO:teuthology.orchestra.run.trial038.stdout:Access: 2026-02-24 18:18:30.968260167 +0000 2026-02-24T18:29:56.960 INFO:teuthology.orchestra.run.trial038.stdout:Modify: 2026-02-24 18:17:26.891858748 +0000 2026-02-24T18:29:56.960 INFO:teuthology.orchestra.run.trial038.stdout:Change: 2026-02-24 18:17:26.891858748 +0000 2026-02-24T18:29:56.960 INFO:teuthology.orchestra.run.trial038.stdout: Birth: 2026-02-24 18:17:26.891858748 +0000 2026-02-24T18:29:56.960 DEBUG:teuthology.orchestra.run.trial038:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2026-02-24T18:29:57.022 INFO:teuthology.orchestra.run.trial038.stderr:1+0 records in 2026-02-24T18:29:57.022 INFO:teuthology.orchestra.run.trial038.stderr:1+0 records out 2026-02-24T18:29:57.023 INFO:teuthology.orchestra.run.trial038.stderr:512 bytes copied, 0.000152863 s, 3.3 MB/s 2026-02-24T18:29:57.024 DEBUG:teuthology.orchestra.run.trial038:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2026-02-24T18:29:57.081 DEBUG:teuthology.orchestra.run.trial038:> stat /dev/vg_nvme/lv_2 2026-02-24T18:29:57.137 INFO:teuthology.orchestra.run.trial038.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2026-02-24T18:29:57.138 INFO:teuthology.orchestra.run.trial038.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-24T18:29:57.138 INFO:teuthology.orchestra.run.trial038.stdout:Device: 6h/6d Inode: 1292 Links: 1 2026-02-24T18:29:57.138 INFO:teuthology.orchestra.run.trial038.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-24T18:29:57.138 INFO:teuthology.orchestra.run.trial038.stdout:Context: system_u:object_r:device_t:s0 2026-02-24T18:29:57.138 INFO:teuthology.orchestra.run.trial038.stdout:Access: 2026-02-24 18:18:30.969260129 +0000 2026-02-24T18:29:57.138 INFO:teuthology.orchestra.run.trial038.stdout:Modify: 2026-02-24 18:17:26.890858786 +0000 2026-02-24T18:29:57.138 INFO:teuthology.orchestra.run.trial038.stdout:Change: 2026-02-24 18:17:26.890858786 +0000 2026-02-24T18:29:57.138 INFO:teuthology.orchestra.run.trial038.stdout: Birth: 2026-02-24 18:17:26.890858786 +0000 2026-02-24T18:29:57.139 DEBUG:teuthology.orchestra.run.trial038:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2026-02-24T18:29:57.199 INFO:teuthology.orchestra.run.trial038.stderr:1+0 records in 2026-02-24T18:29:57.199 INFO:teuthology.orchestra.run.trial038.stderr:1+0 records out 2026-02-24T18:29:57.199 INFO:teuthology.orchestra.run.trial038.stderr:512 bytes copied, 0.000142755 s, 3.6 MB/s 2026-02-24T18:29:57.200 DEBUG:teuthology.orchestra.run.trial038:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2026-02-24T18:29:57.257 DEBUG:teuthology.orchestra.run.trial038:> stat /dev/vg_nvme/lv_3 2026-02-24T18:29:57.314 INFO:teuthology.orchestra.run.trial038.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2026-02-24T18:29:57.314 INFO:teuthology.orchestra.run.trial038.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-24T18:29:57.314 INFO:teuthology.orchestra.run.trial038.stdout:Device: 6h/6d Inode: 1299 Links: 1 2026-02-24T18:29:57.314 INFO:teuthology.orchestra.run.trial038.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-24T18:29:57.314 INFO:teuthology.orchestra.run.trial038.stdout:Context: system_u:object_r:device_t:s0 2026-02-24T18:29:57.314 INFO:teuthology.orchestra.run.trial038.stdout:Access: 2026-02-24 18:18:30.969260129 +0000 2026-02-24T18:29:57.315 INFO:teuthology.orchestra.run.trial038.stdout:Modify: 2026-02-24 18:17:26.890858786 +0000 2026-02-24T18:29:57.315 INFO:teuthology.orchestra.run.trial038.stdout:Change: 2026-02-24 18:17:26.890858786 +0000 2026-02-24T18:29:57.315 INFO:teuthology.orchestra.run.trial038.stdout: Birth: 2026-02-24 18:17:26.890858786 +0000 2026-02-24T18:29:57.315 DEBUG:teuthology.orchestra.run.trial038:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2026-02-24T18:29:57.376 INFO:teuthology.orchestra.run.trial038.stderr:1+0 records in 2026-02-24T18:29:57.376 INFO:teuthology.orchestra.run.trial038.stderr:1+0 records out 2026-02-24T18:29:57.376 INFO:teuthology.orchestra.run.trial038.stderr:512 bytes copied, 0.000147583 s, 3.5 MB/s 2026-02-24T18:29:57.377 DEBUG:teuthology.orchestra.run.trial038:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2026-02-24T18:29:57.434 DEBUG:teuthology.orchestra.run.trial038:> stat /dev/vg_nvme/lv_4 2026-02-24T18:29:57.489 INFO:teuthology.orchestra.run.trial038.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2026-02-24T18:29:57.490 INFO:teuthology.orchestra.run.trial038.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-24T18:29:57.490 INFO:teuthology.orchestra.run.trial038.stdout:Device: 6h/6d Inode: 1301 Links: 1 2026-02-24T18:29:57.490 INFO:teuthology.orchestra.run.trial038.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-24T18:29:57.490 INFO:teuthology.orchestra.run.trial038.stdout:Context: system_u:object_r:device_t:s0 2026-02-24T18:29:57.490 INFO:teuthology.orchestra.run.trial038.stdout:Access: 2026-02-24 18:18:30.969260129 +0000 2026-02-24T18:29:57.490 INFO:teuthology.orchestra.run.trial038.stdout:Modify: 2026-02-24 18:17:26.890858786 +0000 2026-02-24T18:29:57.490 INFO:teuthology.orchestra.run.trial038.stdout:Change: 2026-02-24 18:17:26.890858786 +0000 2026-02-24T18:29:57.490 INFO:teuthology.orchestra.run.trial038.stdout: Birth: 2026-02-24 18:17:26.890858786 +0000 2026-02-24T18:29:57.490 DEBUG:teuthology.orchestra.run.trial038:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2026-02-24T18:29:57.554 INFO:teuthology.orchestra.run.trial038.stderr:1+0 records in 2026-02-24T18:29:57.554 INFO:teuthology.orchestra.run.trial038.stderr:1+0 records out 2026-02-24T18:29:57.554 INFO:teuthology.orchestra.run.trial038.stderr:512 bytes copied, 0.000155357 s, 3.3 MB/s 2026-02-24T18:29:57.555 DEBUG:teuthology.orchestra.run.trial038:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2026-02-24T18:29:57.610 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2026-02-24T18:29:57.610 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@trial038.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}} 2026-02-24T18:29:57.611 INFO:tasks.ceph:Generating config... 2026-02-24T18:29:57.613 INFO:tasks.ceph:[global] osd heartbeat grace = 40 2026-02-24T18:29:57.614 INFO:tasks.ceph:[global] osd max object name len = 460 2026-02-24T18:29:57.614 INFO:tasks.ceph:[global] osd max object namespace len = 64 2026-02-24T18:29:57.614 INFO:tasks.ceph:[mgr] debug mgr = 20 2026-02-24T18:29:57.614 INFO:tasks.ceph:[mgr] debug ms = 1 2026-02-24T18:29:57.614 INFO:tasks.ceph:[mon] debug mon = 20 2026-02-24T18:29:57.614 INFO:tasks.ceph:[mon] debug ms = 1 2026-02-24T18:29:57.614 INFO:tasks.ceph:[mon] debug paxos = 20 2026-02-24T18:29:57.614 INFO:tasks.ceph:[mon] mon osd crush smoke test = False 2026-02-24T18:29:57.614 INFO:tasks.ceph:[osd] debug ms = 1 2026-02-24T18:29:57.614 INFO:tasks.ceph:[osd] debug osd = 20 2026-02-24T18:29:57.615 INFO:tasks.ceph:[osd] osd fast shutdown = False 2026-02-24T18:29:57.615 INFO:tasks.ceph:Setting up mon.a... 2026-02-24T18:29:57.615 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2026-02-24T18:29:57.683 INFO:teuthology.orchestra.run.trial038.stdout:creating /etc/ceph/ceph.keyring 2026-02-24T18:29:57.686 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2026-02-24T18:29:57.762 DEBUG:teuthology.orchestra.run.trial038:> sudo chmod 0644 /etc/ceph/ceph.keyring 2026-02-24T18:29:57.825 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '10.20.193.38'), ('mon.b', '[v2:10.20.193.38:3301,v1:10.20.193.38:6790]'), ('mon.c', '[v2:10.20.193.38:3302,v1:10.20.193.38:6791]')] 2026-02-24T18:29:57.826 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '10.20.193.38,[v2:10.20.193.38:3301,v1:10.20.193.38:6790],[v2:10.20.193.38:3302,v1:10.20.193.38:6791]', 'osd heartbeat grace': 40, 'osd max object name len': 460, 'osd max object namespace len': 64}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd mclock skip benchmark': 'true', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'debug ms': 1, 'debug osd': 20, 'osd fast shutdown': False}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon osd crush smoke test': False}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok'}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}} 2026-02-24T18:29:57.828 DEBUG:teuthology.orchestra.run.trial038:> set -ex 2026-02-24T18:29:57.828 DEBUG:teuthology.orchestra.run.trial038:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2026-02-24T18:29:57.882 DEBUG:teuthology.orchestra.run.trial038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 10.20.193.38 --addv b '[v2:10.20.193.38:3301,v1:10.20.193.38:6790]' --addv c '[v2:10.20.193.38:3302,v1:10.20.193.38:6791]' --print /home/ubuntu/cephtest/ceph.monmap 2026-02-24T18:29:57.955 INFO:teuthology.orchestra.run.trial038.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2026-02-24T18:29:57.955 INFO:teuthology.orchestra.run.trial038.stdout:monmaptool: generated fsid 3487e59d-026b-4a4d-92d5-661e87a7c906 2026-02-24T18:29:57.955 INFO:teuthology.orchestra.run.trial038.stdout:setting min_mon_release = quincy 2026-02-24T18:29:57.955 INFO:teuthology.orchestra.run.trial038.stdout:epoch 0 2026-02-24T18:29:57.956 INFO:teuthology.orchestra.run.trial038.stdout:fsid 3487e59d-026b-4a4d-92d5-661e87a7c906 2026-02-24T18:29:57.956 INFO:teuthology.orchestra.run.trial038.stdout:last_changed 2026-02-24T18:29:57.954384+0000 2026-02-24T18:29:57.956 INFO:teuthology.orchestra.run.trial038.stdout:created 2026-02-24T18:29:57.954384+0000 2026-02-24T18:29:57.956 INFO:teuthology.orchestra.run.trial038.stdout:min_mon_release 17 (quincy) 2026-02-24T18:29:57.956 INFO:teuthology.orchestra.run.trial038.stdout:election_strategy: 1 2026-02-24T18:29:57.956 INFO:teuthology.orchestra.run.trial038.stdout:0: [v2:10.20.193.38:3300/0,v1:10.20.193.38:6789/0] mon.a 2026-02-24T18:29:57.956 INFO:teuthology.orchestra.run.trial038.stdout:1: [v2:10.20.193.38:3301/0,v1:10.20.193.38:6790/0] mon.b 2026-02-24T18:29:57.956 INFO:teuthology.orchestra.run.trial038.stdout:2: [v2:10.20.193.38:3302/0,v1:10.20.193.38:6791/0] mon.c 2026-02-24T18:29:57.956 INFO:teuthology.orchestra.run.trial038.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2026-02-24T18:29:57.958 DEBUG:teuthology.orchestra.run.trial038:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2026-02-24T18:29:58.012 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 3487e59d-026b-4a4d-92d5-661e87a7c906... 2026-02-24T18:29:58.013 DEBUG:teuthology.orchestra.run.trial038:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2026-02-24T18:29:58.092 INFO:teuthology.orchestra.run.trial038.stdout:[global] 2026-02-24T18:29:58.092 INFO:teuthology.orchestra.run.trial038.stdout: chdir = "" 2026-02-24T18:29:58.092 INFO:teuthology.orchestra.run.trial038.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2026-02-24T18:29:58.092 INFO:teuthology.orchestra.run.trial038.stdout: auth supported = cephx 2026-02-24T18:29:58.092 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:29:58.092 INFO:teuthology.orchestra.run.trial038.stdout: filestore xattr use omap = true 2026-02-24T18:29:58.092 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:29:58.092 INFO:teuthology.orchestra.run.trial038.stdout: mon clock drift allowed = 1.000 2026-02-24T18:29:58.092 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:29:58.092 INFO:teuthology.orchestra.run.trial038.stdout: osd crush chooseleaf type = 0 2026-02-24T18:29:58.092 INFO:teuthology.orchestra.run.trial038.stdout: auth debug = true 2026-02-24T18:29:58.093 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:29:58.093 INFO:teuthology.orchestra.run.trial038.stdout: ms die on old message = true 2026-02-24T18:29:58.093 INFO:teuthology.orchestra.run.trial038.stdout: ms die on bug = true 2026-02-24T18:29:58.093 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:29:58.093 INFO:teuthology.orchestra.run.trial038.stdout: mon max pg per osd = 10000 # >= luminous 2026-02-24T18:29:58.093 INFO:teuthology.orchestra.run.trial038.stdout: mon pg warn max object skew = 0 2026-02-24T18:29:58.093 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:29:58.093 INFO:teuthology.orchestra.run.trial038.stdout: # disable pg_autoscaler by default for new pools 2026-02-24T18:29:58.093 INFO:teuthology.orchestra.run.trial038.stdout: osd_pool_default_pg_autoscale_mode = off 2026-02-24T18:29:58.093 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:29:58.093 INFO:teuthology.orchestra.run.trial038.stdout: osd pool default size = 2 2026-02-24T18:29:58.093 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:29:58.094 INFO:teuthology.orchestra.run.trial038.stdout: mon osd allow primary affinity = true 2026-02-24T18:29:58.094 INFO:teuthology.orchestra.run.trial038.stdout: mon osd allow pg remap = true 2026-02-24T18:29:58.094 INFO:teuthology.orchestra.run.trial038.stdout: mon warn on legacy crush tunables = false 2026-02-24T18:29:58.094 INFO:teuthology.orchestra.run.trial038.stdout: mon warn on crush straw calc version zero = false 2026-02-24T18:29:58.094 INFO:teuthology.orchestra.run.trial038.stdout: mon warn on no sortbitwise = false 2026-02-24T18:29:58.094 INFO:teuthology.orchestra.run.trial038.stdout: mon warn on osd down out interval zero = false 2026-02-24T18:29:58.094 INFO:teuthology.orchestra.run.trial038.stdout: mon warn on too few osds = false 2026-02-24T18:29:58.094 INFO:teuthology.orchestra.run.trial038.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2026-02-24T18:29:58.094 INFO:teuthology.orchestra.run.trial038.stdout: mon_warn_on_pool_no_redundancy = false 2026-02-24T18:29:58.094 INFO:teuthology.orchestra.run.trial038.stdout: mon_allow_pool_size_one = true 2026-02-24T18:29:58.094 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:29:58.094 INFO:teuthology.orchestra.run.trial038.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2026-02-24T18:29:58.095 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:29:58.095 INFO:teuthology.orchestra.run.trial038.stdout: osd default data pool replay window = 5 2026-02-24T18:29:58.095 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:29:58.095 INFO:teuthology.orchestra.run.trial038.stdout: mon allow pool delete = true 2026-02-24T18:29:58.095 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:29:58.095 INFO:teuthology.orchestra.run.trial038.stdout: mon cluster log file level = debug 2026-02-24T18:29:58.095 INFO:teuthology.orchestra.run.trial038.stdout: debug asserts on shutdown = true 2026-02-24T18:29:58.095 INFO:teuthology.orchestra.run.trial038.stdout: mon health detail to clog = false 2026-02-24T18:29:58.095 INFO:teuthology.orchestra.run.trial038.stdout: mon host = "10.20.193.38,[v2:10.20.193.38:3301,v1:10.20.193.38:6790],[v2:10.20.193.38:3302,v1:10.20.193.38:6791]" 2026-02-24T18:29:58.095 INFO:teuthology.orchestra.run.trial038.stdout: osd heartbeat grace = 40 2026-02-24T18:29:58.095 INFO:teuthology.orchestra.run.trial038.stdout: osd max object name len = 460 2026-02-24T18:29:58.095 INFO:teuthology.orchestra.run.trial038.stdout: osd max object namespace len = 64 2026-02-24T18:29:58.095 INFO:teuthology.orchestra.run.trial038.stdout: fsid = 3487e59d-026b-4a4d-92d5-661e87a7c906 2026-02-24T18:29:58.096 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:29:58.096 INFO:teuthology.orchestra.run.trial038.stdout:[osd] 2026-02-24T18:29:58.096 INFO:teuthology.orchestra.run.trial038.stdout: osd journal size = 100 2026-02-24T18:29:58.096 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:29:58.096 INFO:teuthology.orchestra.run.trial038.stdout: osd scrub load threshold = 5.0 2026-02-24T18:29:58.096 INFO:teuthology.orchestra.run.trial038.stdout: osd scrub max interval = 600 2026-02-24T18:29:58.096 INFO:teuthology.orchestra.run.trial038.stdout: osd mclock profile = high_recovery_ops 2026-02-24T18:29:58.096 INFO:teuthology.orchestra.run.trial038.stdout: osd mclock skip benchmark = true 2026-02-24T18:29:58.096 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:29:58.096 INFO:teuthology.orchestra.run.trial038.stdout: osd recover clone overlap = true 2026-02-24T18:29:58.096 INFO:teuthology.orchestra.run.trial038.stdout: osd recovery max chunk = 1048576 2026-02-24T18:29:58.096 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:29:58.096 INFO:teuthology.orchestra.run.trial038.stdout: osd debug shutdown = true 2026-02-24T18:29:58.097 INFO:teuthology.orchestra.run.trial038.stdout: osd debug op order = true 2026-02-24T18:29:58.097 INFO:teuthology.orchestra.run.trial038.stdout: osd debug verify stray on activate = true 2026-02-24T18:29:58.097 INFO:teuthology.orchestra.run.trial038.stdout: osd debug trim objects = true 2026-02-24T18:29:58.097 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:29:58.097 INFO:teuthology.orchestra.run.trial038.stdout: osd open classes on start = true 2026-02-24T18:29:58.097 INFO:teuthology.orchestra.run.trial038.stdout: osd debug pg log writeout = true 2026-02-24T18:29:58.097 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:29:58.097 INFO:teuthology.orchestra.run.trial038.stdout: osd deep scrub update digest min age = 30 2026-02-24T18:29:58.097 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:29:58.097 INFO:teuthology.orchestra.run.trial038.stdout: osd map max advance = 10 2026-02-24T18:29:58.097 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:29:58.097 INFO:teuthology.orchestra.run.trial038.stdout: journal zero on create = true 2026-02-24T18:29:58.098 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:29:58.098 INFO:teuthology.orchestra.run.trial038.stdout: filestore ondisk finisher threads = 3 2026-02-24T18:29:58.098 INFO:teuthology.orchestra.run.trial038.stdout: filestore apply finisher threads = 3 2026-02-24T18:29:58.098 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:29:58.098 INFO:teuthology.orchestra.run.trial038.stdout: bdev debug aio = true 2026-02-24T18:29:58.098 INFO:teuthology.orchestra.run.trial038.stdout: osd debug misdirected ops = true 2026-02-24T18:29:58.098 INFO:teuthology.orchestra.run.trial038.stdout: debug ms = 1 2026-02-24T18:29:58.098 INFO:teuthology.orchestra.run.trial038.stdout: debug osd = 20 2026-02-24T18:29:58.098 INFO:teuthology.orchestra.run.trial038.stdout: osd fast shutdown = False 2026-02-24T18:29:58.098 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:29:58.098 INFO:teuthology.orchestra.run.trial038.stdout:[mgr] 2026-02-24T18:29:58.098 INFO:teuthology.orchestra.run.trial038.stdout: debug ms = 1 2026-02-24T18:29:58.098 INFO:teuthology.orchestra.run.trial038.stdout: debug mgr = 20 2026-02-24T18:29:58.099 INFO:teuthology.orchestra.run.trial038.stdout: debug mon = 20 2026-02-24T18:29:58.099 INFO:teuthology.orchestra.run.trial038.stdout: debug auth = 20 2026-02-24T18:29:58.099 INFO:teuthology.orchestra.run.trial038.stdout: mon reweight min pgs per osd = 4 2026-02-24T18:29:58.099 INFO:teuthology.orchestra.run.trial038.stdout: mon reweight min bytes per osd = 10 2026-02-24T18:29:58.099 INFO:teuthology.orchestra.run.trial038.stdout: mgr/telemetry/nag = false 2026-02-24T18:29:58.099 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:29:58.099 INFO:teuthology.orchestra.run.trial038.stdout:[mon] 2026-02-24T18:29:58.099 INFO:teuthology.orchestra.run.trial038.stdout: debug ms = 1 2026-02-24T18:29:58.099 INFO:teuthology.orchestra.run.trial038.stdout: debug mon = 20 2026-02-24T18:29:58.099 INFO:teuthology.orchestra.run.trial038.stdout: debug paxos = 20 2026-02-24T18:29:58.099 INFO:teuthology.orchestra.run.trial038.stdout: debug auth = 20 2026-02-24T18:29:58.099 INFO:teuthology.orchestra.run.trial038.stdout: mon data avail warn = 5 2026-02-24T18:29:58.099 INFO:teuthology.orchestra.run.trial038.stdout: mon mgr mkfs grace = 240 2026-02-24T18:29:58.100 INFO:teuthology.orchestra.run.trial038.stdout: mon reweight min pgs per osd = 4 2026-02-24T18:29:58.100 INFO:teuthology.orchestra.run.trial038.stdout: mon osd reporter subtree level = osd 2026-02-24T18:29:58.100 INFO:teuthology.orchestra.run.trial038.stdout: mon osd prime pg temp = true 2026-02-24T18:29:58.100 INFO:teuthology.orchestra.run.trial038.stdout: mon reweight min bytes per osd = 10 2026-02-24T18:29:58.100 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:29:58.100 INFO:teuthology.orchestra.run.trial038.stdout: # rotate auth tickets quickly to exercise renewal paths 2026-02-24T18:29:58.100 INFO:teuthology.orchestra.run.trial038.stdout: auth mon ticket ttl = 660 # 11m 2026-02-24T18:29:58.100 INFO:teuthology.orchestra.run.trial038.stdout: auth service ticket ttl = 240 # 4m 2026-02-24T18:29:58.100 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:29:58.100 INFO:teuthology.orchestra.run.trial038.stdout: # don't complain about insecure global_id in the test suite 2026-02-24T18:29:58.100 INFO:teuthology.orchestra.run.trial038.stdout: mon_warn_on_insecure_global_id_reclaim = false 2026-02-24T18:29:58.100 INFO:teuthology.orchestra.run.trial038.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2026-02-24T18:29:58.100 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:29:58.101 INFO:teuthology.orchestra.run.trial038.stdout: # 1m isn't quite enough 2026-02-24T18:29:58.101 INFO:teuthology.orchestra.run.trial038.stdout: mon_down_mkfs_grace = 2m 2026-02-24T18:29:58.101 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:29:58.101 INFO:teuthology.orchestra.run.trial038.stdout: mon_warn_on_filestore_osds = false 2026-02-24T18:29:58.101 INFO:teuthology.orchestra.run.trial038.stdout: mon osd crush smoke test = False 2026-02-24T18:29:58.101 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:29:58.101 INFO:teuthology.orchestra.run.trial038.stdout:[client] 2026-02-24T18:29:58.101 INFO:teuthology.orchestra.run.trial038.stdout: rgw cache enabled = true 2026-02-24T18:29:58.101 INFO:teuthology.orchestra.run.trial038.stdout: rgw enable ops log = true 2026-02-24T18:29:58.101 INFO:teuthology.orchestra.run.trial038.stdout: rgw enable usage log = true 2026-02-24T18:29:58.101 INFO:teuthology.orchestra.run.trial038.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2026-02-24T18:29:58.101 INFO:teuthology.orchestra.run.trial038.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2026-02-24T18:29:58.102 INFO:teuthology.orchestra.run.trial038.stdout:[mon.a] 2026-02-24T18:29:58.102 INFO:teuthology.orchestra.run.trial038.stdout:[mon.b] 2026-02-24T18:29:58.102 INFO:teuthology.orchestra.run.trial038.stdout:[mon.c] 2026-02-24T18:29:58.103 INFO:tasks.ceph:Creating admin key on mon.a... 2026-02-24T18:29:58.103 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2026-02-24T18:29:58.184 INFO:tasks.ceph:Copying monmap to all nodes... 2026-02-24T18:29:58.184 DEBUG:teuthology.orchestra.run.trial038:> set -ex 2026-02-24T18:29:58.184 DEBUG:teuthology.orchestra.run.trial038:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2026-02-24T18:29:58.241 DEBUG:teuthology.orchestra.run.trial038:> set -ex 2026-02-24T18:29:58.241 DEBUG:teuthology.orchestra.run.trial038:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2026-02-24T18:29:58.297 INFO:tasks.ceph:Sending monmap to node ubuntu@trial038.front.sepia.ceph.com 2026-02-24T18:29:58.298 DEBUG:teuthology.orchestra.run.trial038:> set -ex 2026-02-24T18:29:58.298 DEBUG:teuthology.orchestra.run.trial038:> sudo dd of=/etc/ceph/ceph.keyring 2026-02-24T18:29:58.298 DEBUG:teuthology.orchestra.run.trial038:> sudo chmod 0644 /etc/ceph/ceph.keyring 2026-02-24T18:29:58.370 DEBUG:teuthology.orchestra.run.trial038:> set -ex 2026-02-24T18:29:58.370 DEBUG:teuthology.orchestra.run.trial038:> dd of=/home/ubuntu/cephtest/ceph.monmap 2026-02-24T18:29:58.426 INFO:tasks.ceph:Setting up mon nodes... 2026-02-24T18:29:58.426 INFO:tasks.ceph:Setting up mgr nodes... 2026-02-24T18:29:58.427 DEBUG:teuthology.orchestra.run.trial038:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2026-02-24T18:29:58.508 INFO:teuthology.orchestra.run.trial038.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2026-02-24T18:29:58.511 DEBUG:teuthology.orchestra.run.trial038:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2026-02-24T18:29:58.593 INFO:teuthology.orchestra.run.trial038.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2026-02-24T18:29:58.595 INFO:tasks.ceph:Setting up mds nodes... 2026-02-24T18:29:58.595 INFO:tasks.ceph_client:Setting up client nodes... 2026-02-24T18:29:58.596 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2026-02-24T18:29:58.668 INFO:teuthology.orchestra.run.trial038.stdout:creating /etc/ceph/ceph.client.0.keyring 2026-02-24T18:29:58.679 INFO:tasks.ceph:Running mkfs on osd nodes... 2026-02-24T18:29:58.680 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@trial038.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}} 2026-02-24T18:29:58.680 DEBUG:teuthology.orchestra.run.trial038:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2026-02-24T18:29:58.742 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2026-02-24T18:29:58.743 INFO:tasks.ceph:role: osd.0 2026-02-24T18:29:58.743 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@trial038.front.sepia.ceph.com 2026-02-24T18:29:58.743 DEBUG:teuthology.orchestra.run.trial038:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2026-02-24T18:29:58.807 INFO:teuthology.orchestra.run.trial038.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=45875200 blks 2026-02-24T18:29:58.807 INFO:teuthology.orchestra.run.trial038.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-24T18:29:58.807 INFO:teuthology.orchestra.run.trial038.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-24T18:29:58.807 INFO:teuthology.orchestra.run.trial038.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-24T18:29:58.807 INFO:teuthology.orchestra.run.trial038.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-24T18:29:58.807 INFO:teuthology.orchestra.run.trial038.stdout: = sunit=0 swidth=0 blks 2026-02-24T18:29:58.807 INFO:teuthology.orchestra.run.trial038.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-24T18:29:58.808 INFO:teuthology.orchestra.run.trial038.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-24T18:29:58.808 INFO:teuthology.orchestra.run.trial038.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-24T18:29:58.808 INFO:teuthology.orchestra.run.trial038.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-24T18:29:59.376 INFO:teuthology.orchestra.run.trial038.stdout:Discarding blocks...Done. 2026-02-24T18:29:59.378 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@trial038.front.sepia.ceph.com -o noatime 2026-02-24T18:29:59.379 DEBUG:teuthology.orchestra.run.trial038:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2026-02-24T18:29:59.446 DEBUG:teuthology.orchestra.run.trial038:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2026-02-24T18:29:59.512 DEBUG:teuthology.orchestra.run.trial038:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2026-02-24T18:29:59.577 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2026-02-24T18:29:59.577 INFO:tasks.ceph:role: osd.1 2026-02-24T18:29:59.577 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@trial038.front.sepia.ceph.com 2026-02-24T18:29:59.577 DEBUG:teuthology.orchestra.run.trial038:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2026-02-24T18:29:59.641 INFO:teuthology.orchestra.run.trial038.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=45875200 blks 2026-02-24T18:29:59.642 INFO:teuthology.orchestra.run.trial038.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-24T18:29:59.642 INFO:teuthology.orchestra.run.trial038.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-24T18:29:59.642 INFO:teuthology.orchestra.run.trial038.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-24T18:29:59.642 INFO:teuthology.orchestra.run.trial038.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-24T18:29:59.642 INFO:teuthology.orchestra.run.trial038.stdout: = sunit=0 swidth=0 blks 2026-02-24T18:29:59.642 INFO:teuthology.orchestra.run.trial038.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-24T18:29:59.642 INFO:teuthology.orchestra.run.trial038.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-24T18:29:59.642 INFO:teuthology.orchestra.run.trial038.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-24T18:29:59.642 INFO:teuthology.orchestra.run.trial038.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-24T18:30:00.199 INFO:teuthology.orchestra.run.trial038.stdout:Discarding blocks...Done. 2026-02-24T18:30:00.201 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@trial038.front.sepia.ceph.com -o noatime 2026-02-24T18:30:00.201 DEBUG:teuthology.orchestra.run.trial038:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2026-02-24T18:30:00.269 DEBUG:teuthology.orchestra.run.trial038:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2026-02-24T18:30:00.337 DEBUG:teuthology.orchestra.run.trial038:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2026-02-24T18:30:00.401 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2026-02-24T18:30:00.402 INFO:tasks.ceph:role: osd.2 2026-02-24T18:30:00.402 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@trial038.front.sepia.ceph.com 2026-02-24T18:30:00.402 DEBUG:teuthology.orchestra.run.trial038:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2026-02-24T18:30:00.464 INFO:teuthology.orchestra.run.trial038.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=45875200 blks 2026-02-24T18:30:00.465 INFO:teuthology.orchestra.run.trial038.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-24T18:30:00.465 INFO:teuthology.orchestra.run.trial038.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-24T18:30:00.465 INFO:teuthology.orchestra.run.trial038.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-24T18:30:00.465 INFO:teuthology.orchestra.run.trial038.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-24T18:30:00.465 INFO:teuthology.orchestra.run.trial038.stdout: = sunit=0 swidth=0 blks 2026-02-24T18:30:00.465 INFO:teuthology.orchestra.run.trial038.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-24T18:30:00.465 INFO:teuthology.orchestra.run.trial038.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-24T18:30:00.465 INFO:teuthology.orchestra.run.trial038.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-24T18:30:00.465 INFO:teuthology.orchestra.run.trial038.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-24T18:30:01.021 INFO:teuthology.orchestra.run.trial038.stdout:Discarding blocks...Done. 2026-02-24T18:30:01.022 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@trial038.front.sepia.ceph.com -o noatime 2026-02-24T18:30:01.023 DEBUG:teuthology.orchestra.run.trial038:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2026-02-24T18:30:01.089 DEBUG:teuthology.orchestra.run.trial038:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2026-02-24T18:30:01.157 DEBUG:teuthology.orchestra.run.trial038:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-24T18:30:01.235 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-24T18:30:01.233+0000 7fe80d60f780 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2026-02-24T18:30:01.235 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-24T18:30:01.234+0000 7fe80d60f780 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2026-02-24T18:30:01.236 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-24T18:30:01.234+0000 7fe80d60f780 -1 bdev(0x55b018f03800 /var/lib/ceph/osd/ceph-0/block) open stat got: (1) Operation not permitted 2026-02-24T18:30:01.236 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-24T18:30:01.234+0000 7fe80d60f780 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2026-02-24T18:30:03.343 DEBUG:teuthology.orchestra.run.trial038:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2026-02-24T18:30:03.408 DEBUG:teuthology.orchestra.run.trial038:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-24T18:30:03.486 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-24T18:30:03.484+0000 7f440e80f780 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2026-02-24T18:30:03.486 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-24T18:30:03.485+0000 7f440e80f780 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2026-02-24T18:30:03.486 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-24T18:30:03.485+0000 7f440e80f780 -1 bdev(0x55f7ee2ff800 /var/lib/ceph/osd/ceph-1/block) open stat got: (1) Operation not permitted 2026-02-24T18:30:03.486 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-24T18:30:03.485+0000 7f440e80f780 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2026-02-24T18:30:05.601 DEBUG:teuthology.orchestra.run.trial038:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2026-02-24T18:30:05.666 DEBUG:teuthology.orchestra.run.trial038:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-24T18:30:05.744 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-24T18:30:05.742+0000 7faae773d780 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2026-02-24T18:30:05.744 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-24T18:30:05.743+0000 7faae773d780 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2026-02-24T18:30:05.745 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-24T18:30:05.743+0000 7faae773d780 -1 bdev(0x5632a57cd800 /var/lib/ceph/osd/ceph-2/block) open stat got: (1) Operation not permitted 2026-02-24T18:30:05.745 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-24T18:30:05.743+0000 7faae773d780 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2026-02-24T18:30:07.853 DEBUG:teuthology.orchestra.run.trial038:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2026-02-24T18:30:07.918 INFO:tasks.ceph:Reading keys from all nodes... 2026-02-24T18:30:07.919 DEBUG:teuthology.orchestra.run.trial038:> set -ex 2026-02-24T18:30:07.919 DEBUG:teuthology.orchestra.run.trial038:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2026-02-24T18:30:07.982 DEBUG:teuthology.orchestra.run.trial038:> set -ex 2026-02-24T18:30:07.982 DEBUG:teuthology.orchestra.run.trial038:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2026-02-24T18:30:08.046 DEBUG:teuthology.orchestra.run.trial038:> set -ex 2026-02-24T18:30:08.046 DEBUG:teuthology.orchestra.run.trial038:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2026-02-24T18:30:08.110 DEBUG:teuthology.orchestra.run.trial038:> set -ex 2026-02-24T18:30:08.110 DEBUG:teuthology.orchestra.run.trial038:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2026-02-24T18:30:08.174 DEBUG:teuthology.orchestra.run.trial038:> set -ex 2026-02-24T18:30:08.174 DEBUG:teuthology.orchestra.run.trial038:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2026-02-24T18:30:08.236 DEBUG:teuthology.orchestra.run.trial038:> set -ex 2026-02-24T18:30:08.236 DEBUG:teuthology.orchestra.run.trial038:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2026-02-24T18:30:08.293 INFO:tasks.ceph:Adding keys to all mons... 2026-02-24T18:30:08.293 DEBUG:teuthology.orchestra.run.trial038:> sudo tee -a /etc/ceph/ceph.keyring 2026-02-24T18:30:08.354 INFO:teuthology.orchestra.run.trial038.stdout:[mgr.x] 2026-02-24T18:30:08.355 INFO:teuthology.orchestra.run.trial038.stdout: key = AQAm7p1pkL9OHhAAg7mz+k3QR5otzNT0mHvLcQ== 2026-02-24T18:30:08.355 INFO:teuthology.orchestra.run.trial038.stdout:[mgr.y] 2026-02-24T18:30:08.355 INFO:teuthology.orchestra.run.trial038.stdout: key = AQAm7p1p+4lVIxAAnDfrEL/B9z1clwn19SOgbQ== 2026-02-24T18:30:08.355 INFO:teuthology.orchestra.run.trial038.stdout:[osd.0] 2026-02-24T18:30:08.355 INFO:teuthology.orchestra.run.trial038.stdout: key = AQAp7p1pprP6DRAAxyxOEsNUD0BxnfwCXGMZ8Q== 2026-02-24T18:30:08.355 INFO:teuthology.orchestra.run.trial038.stdout:[osd.1] 2026-02-24T18:30:08.355 INFO:teuthology.orchestra.run.trial038.stdout: key = AQAr7p1pT0XvHBAAW8CtKDYqCtROYC3sCf3IRA== 2026-02-24T18:30:08.355 INFO:teuthology.orchestra.run.trial038.stdout:[osd.2] 2026-02-24T18:30:08.355 INFO:teuthology.orchestra.run.trial038.stdout: key = AQAt7p1peoNTLBAAe/0HVuBPtY1NiuO5ng0nbQ== 2026-02-24T18:30:08.355 INFO:teuthology.orchestra.run.trial038.stdout:[client.0] 2026-02-24T18:30:08.355 INFO:teuthology.orchestra.run.trial038.stdout: key = AQAm7p1pj+PXJxAAuvVtkgsYZaJzg/evDGABYg== 2026-02-24T18:30:08.356 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2026-02-24T18:30:08.435 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2026-02-24T18:30:08.513 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-24T18:30:08.593 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-24T18:30:08.672 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-24T18:30:08.752 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2026-02-24T18:30:08.833 INFO:tasks.ceph:Running mkfs on mon nodes... 2026-02-24T18:30:08.833 DEBUG:teuthology.orchestra.run.trial038:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2026-02-24T18:30:08.897 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2026-02-24T18:30:08.983 DEBUG:teuthology.orchestra.run.trial038:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2026-02-24T18:30:09.007 DEBUG:teuthology.orchestra.run.trial038:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2026-02-24T18:30:09.071 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2026-02-24T18:30:09.152 DEBUG:teuthology.orchestra.run.trial038:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2026-02-24T18:30:09.217 DEBUG:teuthology.orchestra.run.trial038:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2026-02-24T18:30:09.282 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2026-02-24T18:30:09.366 DEBUG:teuthology.orchestra.run.trial038:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2026-02-24T18:30:09.429 DEBUG:teuthology.orchestra.run.trial038:> rm -- /home/ubuntu/cephtest/ceph.monmap 2026-02-24T18:30:09.485 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2026-02-24T18:30:09.485 DEBUG:tasks.ceph_manager:running mon.a under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--gen-suppressions=all', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/mon.a.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck', '--leak-check=full', '--show-reachable=yes'] 2026-02-24T18:30:09.486 INFO:tasks.ceph.mon.a:Restarting daemon 2026-02-24T18:30:09.486 DEBUG:teuthology.orchestra.run.trial038:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/mon.a.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i a 2026-02-24T18:30:09.527 INFO:tasks.ceph.mon.a:Started 2026-02-24T18:30:09.528 DEBUG:tasks.ceph_manager:running mon.b under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--gen-suppressions=all', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/mon.b.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck', '--leak-check=full', '--show-reachable=yes'] 2026-02-24T18:30:09.528 INFO:tasks.ceph.mon.b:Restarting daemon 2026-02-24T18:30:09.528 DEBUG:teuthology.orchestra.run.trial038:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/mon.b.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i b 2026-02-24T18:30:09.530 INFO:tasks.ceph.mon.b:Started 2026-02-24T18:30:09.530 DEBUG:tasks.ceph_manager:running mon.c under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--gen-suppressions=all', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/mon.c.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck', '--leak-check=full', '--show-reachable=yes'] 2026-02-24T18:30:09.531 INFO:tasks.ceph.mon.c:Restarting daemon 2026-02-24T18:30:09.531 DEBUG:teuthology.orchestra.run.trial038:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/mon.c.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i c 2026-02-24T18:30:09.533 INFO:tasks.ceph.mon.c:Started 2026-02-24T18:30:09.533 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2026-02-24T18:30:09.534 INFO:tasks.ceph.mgr.x:Restarting daemon 2026-02-24T18:30:09.534 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term ceph-mgr -f --cluster ceph -i x 2026-02-24T18:30:09.536 INFO:tasks.ceph.mgr.x:Started 2026-02-24T18:30:09.536 INFO:tasks.ceph.mgr.y:Restarting daemon 2026-02-24T18:30:09.536 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term ceph-mgr -f --cluster ceph -i y 2026-02-24T18:30:09.539 INFO:tasks.ceph.mgr.y:Started 2026-02-24T18:30:09.539 INFO:tasks.ceph:Setting crush tunables to default 2026-02-24T18:30:09.539 DEBUG:teuthology.orchestra.run.trial038:> sudo ceph --cluster ceph osd crush tunables default 2026-02-24T18:30:20.586 INFO:tasks.ceph.mon.c.trial038.stderr:--00:00:00:11.005 43056-- Warning: zero subprog, missing DW_AT_abstract_origin in DW_TAG_inlined_subroutine in /usr/lib/debug/.build-id/c8/2482ecb30cd94afc031e15355b32036f38cb43.debug 2026-02-24T18:30:20.586 INFO:tasks.ceph.mon.b.trial038.stderr:--00:00:00:11.005 43055-- Warning: zero subprog, missing DW_AT_abstract_origin in DW_TAG_inlined_subroutine in /usr/lib/debug/.build-id/c8/2482ecb30cd94afc031e15355b32036f38cb43.debug 2026-02-24T18:30:20.645 INFO:tasks.ceph.mon.a.trial038.stderr:--00:00:00:11.065 43054-- Warning: zero subprog, missing DW_AT_abstract_origin in DW_TAG_inlined_subroutine in /usr/lib/debug/.build-id/c8/2482ecb30cd94afc031e15355b32036f38cb43.debug 2026-02-24T18:30:26.379 INFO:tasks.ceph.mon.c.trial038.stderr:==00:00:00:16.799 43056== Warning: unimplemented fcntl command: 1036 2026-02-24T18:30:26.383 INFO:tasks.ceph.mon.b.trial038.stderr:==00:00:00:16.803 43055== Warning: unimplemented fcntl command: 1036 2026-02-24T18:30:26.466 INFO:tasks.ceph.mon.a.trial038.stderr:==00:00:00:16.886 43054== Warning: unimplemented fcntl command: 1036 2026-02-24T18:30:26.528 INFO:tasks.ceph.mon.c.trial038.stderr:==00:00:00:16.947 43056== Warning: unimplemented fcntl command: 1036 2026-02-24T18:30:26.533 INFO:tasks.ceph.mon.b.trial038.stderr:==00:00:00:16.952 43055== Warning: unimplemented fcntl command: 1036 2026-02-24T18:30:26.615 INFO:tasks.ceph.mon.a.trial038.stderr:==00:00:00:17.034 43054== Warning: unimplemented fcntl command: 1036 2026-02-24T18:30:29.191 INFO:teuthology.orchestra.run.trial038.stderr:adjusted tunables profile to default 2026-02-24T18:30:29.202 INFO:tasks.ceph:check_enable_crimson: False 2026-02-24T18:30:29.202 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2026-02-24T18:30:29.203 DEBUG:teuthology.orchestra.run.trial038:> set -ex 2026-02-24T18:30:29.203 DEBUG:teuthology.orchestra.run.trial038:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2026-02-24T18:30:29.227 DEBUG:teuthology.orchestra.run.trial038:> set -ex 2026-02-24T18:30:29.227 DEBUG:teuthology.orchestra.run.trial038:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2026-02-24T18:30:29.291 DEBUG:teuthology.orchestra.run.trial038:> set -ex 2026-02-24T18:30:29.291 DEBUG:teuthology.orchestra.run.trial038:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2026-02-24T18:30:29.354 DEBUG:teuthology.orchestra.run.trial038:> sudo ceph --cluster ceph osd new 22d4bbb5-3cf0-4c06-9552-80e01980013c 0 2026-02-24T18:30:29.394 INFO:tasks.ceph.mgr.y.trial038.stderr:2026-02-24T18:30:29.391+0000 7ff39bbce140 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2026-02-24T18:30:29.395 INFO:tasks.ceph.mgr.x.trial038.stderr:2026-02-24T18:30:29.391+0000 7f087642d140 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2026-02-24T18:30:29.518 INFO:tasks.ceph.mgr.y.trial038.stderr:2026-02-24T18:30:29.515+0000 7ff39bbce140 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2026-02-24T18:30:29.518 INFO:tasks.ceph.mgr.x.trial038.stderr:2026-02-24T18:30:29.515+0000 7f087642d140 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2026-02-24T18:30:29.545 INFO:tasks.ceph.mgr.y.trial038.stderr:2026-02-24T18:30:29.542+0000 7ff39bbce140 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2026-02-24T18:30:29.545 INFO:tasks.ceph.mgr.x.trial038.stderr:2026-02-24T18:30:29.543+0000 7f087642d140 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2026-02-24T18:30:29.579 INFO:tasks.ceph.mgr.x.trial038.stderr:2026-02-24T18:30:29.576+0000 7f087642d140 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2026-02-24T18:30:29.579 INFO:tasks.ceph.mgr.y.trial038.stderr:2026-02-24T18:30:29.576+0000 7ff39bbce140 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2026-02-24T18:30:29.602 INFO:tasks.ceph.mgr.y.trial038.stderr:2026-02-24T18:30:29.600+0000 7ff39bbce140 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2026-02-24T18:30:29.603 INFO:tasks.ceph.mgr.x.trial038.stderr:2026-02-24T18:30:29.600+0000 7f087642d140 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2026-02-24T18:30:29.629 INFO:tasks.ceph.mgr.y.trial038.stderr:2026-02-24T18:30:29.627+0000 7ff39bbce140 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2026-02-24T18:30:29.630 INFO:tasks.ceph.mgr.x.trial038.stderr:2026-02-24T18:30:29.627+0000 7f087642d140 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2026-02-24T18:30:29.675 INFO:teuthology.orchestra.run.trial038.stdout:0 2026-02-24T18:30:29.685 DEBUG:teuthology.orchestra.run.trial038:> sudo ceph --cluster ceph osd new ee9a5258-997f-4465-98f1-976c5dc010c0 1 2026-02-24T18:30:29.688 INFO:tasks.ceph.mgr.x.trial038.stderr:2026-02-24T18:30:29.686+0000 7f087642d140 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2026-02-24T18:30:29.689 INFO:tasks.ceph.mgr.y.trial038.stderr:2026-02-24T18:30:29.686+0000 7ff39bbce140 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2026-02-24T18:30:29.751 INFO:tasks.ceph.mgr.x.trial038.stderr:/usr/lib64/python3.9/site-packages/scipy/__init__.py:73: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2026-02-24T18:30:29.751 INFO:tasks.ceph.mgr.x.trial038.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2026-02-24T18:30:29.751 INFO:tasks.ceph.mgr.x.trial038.stderr: from numpy import show_config as show_numpy_config 2026-02-24T18:30:29.751 INFO:tasks.ceph.mgr.y.trial038.stderr:/usr/lib64/python3.9/site-packages/scipy/__init__.py:73: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2026-02-24T18:30:29.752 INFO:tasks.ceph.mgr.y.trial038.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2026-02-24T18:30:29.752 INFO:tasks.ceph.mgr.y.trial038.stderr: from numpy import show_config as show_numpy_config 2026-02-24T18:30:29.754 INFO:tasks.ceph.mgr.x.trial038.stderr:2026-02-24T18:30:29.752+0000 7f087642d140 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2026-02-24T18:30:29.754 INFO:tasks.ceph.mgr.y.trial038.stderr:2026-02-24T18:30:29.752+0000 7ff39bbce140 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2026-02-24T18:30:29.781 INFO:tasks.ceph.mgr.y.trial038.stderr:2026-02-24T18:30:29.779+0000 7ff39bbce140 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2026-02-24T18:30:29.781 INFO:tasks.ceph.mgr.x.trial038.stderr:2026-02-24T18:30:29.779+0000 7f087642d140 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2026-02-24T18:30:29.837 INFO:tasks.ceph.mgr.x.trial038.stderr:2026-02-24T18:30:29.835+0000 7f087642d140 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2026-02-24T18:30:29.838 INFO:tasks.ceph.mgr.y.trial038.stderr:2026-02-24T18:30:29.836+0000 7ff39bbce140 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2026-02-24T18:30:29.883 INFO:tasks.ceph.mgr.x.trial038.stderr:2026-02-24T18:30:29.881+0000 7f087642d140 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2026-02-24T18:30:29.884 INFO:tasks.ceph.mgr.y.trial038.stderr:2026-02-24T18:30:29.881+0000 7ff39bbce140 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2026-02-24T18:30:29.909 INFO:tasks.ceph.mgr.x.trial038.stderr:2026-02-24T18:30:29.907+0000 7f087642d140 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2026-02-24T18:30:29.910 INFO:tasks.ceph.mgr.y.trial038.stderr:2026-02-24T18:30:29.908+0000 7ff39bbce140 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2026-02-24T18:30:29.935 INFO:tasks.ceph.mgr.x.trial038.stderr:2026-02-24T18:30:29.933+0000 7f087642d140 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2026-02-24T18:30:29.936 INFO:tasks.ceph.mgr.y.trial038.stderr:2026-02-24T18:30:29.934+0000 7ff39bbce140 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2026-02-24T18:30:29.952 INFO:teuthology.orchestra.run.trial038.stdout:1 2026-02-24T18:30:29.961 DEBUG:teuthology.orchestra.run.trial038:> sudo ceph --cluster ceph osd new aaf542f4-f7fb-4931-b598-36efb11497b0 2 2026-02-24T18:30:29.962 INFO:tasks.ceph.mgr.x.trial038.stderr:2026-02-24T18:30:29.960+0000 7f087642d140 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2026-02-24T18:30:29.964 INFO:tasks.ceph.mgr.y.trial038.stderr:2026-02-24T18:30:29.961+0000 7ff39bbce140 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2026-02-24T18:30:29.986 INFO:tasks.ceph.mgr.x.trial038.stderr:2026-02-24T18:30:29.984+0000 7f087642d140 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2026-02-24T18:30:29.987 INFO:tasks.ceph.mgr.y.trial038.stderr:2026-02-24T18:30:29.984+0000 7ff39bbce140 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2026-02-24T18:30:30.148 INFO:teuthology.orchestra.run.trial038.stdout:2 2026-02-24T18:30:30.157 DEBUG:tasks.ceph_manager:running osd.0 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--gen-suppressions=all', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.0.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck'] 2026-02-24T18:30:30.158 INFO:tasks.ceph.osd.0:Restarting daemon 2026-02-24T18:30:30.158 DEBUG:teuthology.orchestra.run.trial038:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 2026-02-24T18:30:30.189 INFO:tasks.ceph.mgr.y.trial038.stderr:2026-02-24T18:30:30.186+0000 7ff39bbce140 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2026-02-24T18:30:30.190 INFO:tasks.ceph.mgr.x.trial038.stderr:2026-02-24T18:30:30.187+0000 7f087642d140 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2026-02-24T18:30:30.191 INFO:tasks.ceph.osd.0:Started 2026-02-24T18:30:30.191 DEBUG:tasks.ceph_manager:running osd.1 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--gen-suppressions=all', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.1.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck'] 2026-02-24T18:30:30.191 INFO:tasks.ceph.osd.1:Restarting daemon 2026-02-24T18:30:30.192 DEBUG:teuthology.orchestra.run.trial038:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 2026-02-24T18:30:30.194 INFO:tasks.ceph.osd.1:Started 2026-02-24T18:30:30.194 DEBUG:tasks.ceph_manager:running osd.2 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--gen-suppressions=all', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.2.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck'] 2026-02-24T18:30:30.194 INFO:tasks.ceph.osd.2:Restarting daemon 2026-02-24T18:30:30.194 DEBUG:teuthology.orchestra.run.trial038:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 2026-02-24T18:30:30.197 INFO:tasks.ceph.osd.2:Started 2026-02-24T18:30:30.197 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-24T18:30:30.246 INFO:tasks.ceph.mgr.x.trial038.stderr:2026-02-24T18:30:30.244+0000 7f087642d140 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2026-02-24T18:30:30.246 INFO:tasks.ceph.mgr.y.trial038.stderr:2026-02-24T18:30:30.244+0000 7ff39bbce140 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2026-02-24T18:30:30.271 INFO:tasks.ceph.mgr.y.trial038.stderr:2026-02-24T18:30:30.269+0000 7ff39bbce140 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2026-02-24T18:30:30.272 INFO:tasks.ceph.mgr.x.trial038.stderr:2026-02-24T18:30:30.269+0000 7f087642d140 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2026-02-24T18:30:30.298 INFO:tasks.ceph.mgr.y.trial038.stderr:2026-02-24T18:30:30.295+0000 7ff39bbce140 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2026-02-24T18:30:30.298 INFO:tasks.ceph.mgr.x.trial038.stderr:2026-02-24T18:30:30.295+0000 7f087642d140 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2026-02-24T18:30:30.355 INFO:tasks.ceph.mgr.y.trial038.stderr:2026-02-24T18:30:30.353+0000 7ff39bbce140 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2026-02-24T18:30:30.356 INFO:tasks.ceph.mgr.x.trial038.stderr:2026-02-24T18:30:30.354+0000 7f087642d140 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2026-02-24T18:30:30.372 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:30:30.372 INFO:teuthology.orchestra.run.trial038.stdout:{"epoch":5,"fsid":"3487e59d-026b-4a4d-92d5-661e87a7c906","created":"2026-02-24T18:30:28.512730+0000","modified":"2026-02-24T18:30:30.119568+0000","last_up_change":"0.000000","last_in_change":"2026-02-24T18:30:30.119568+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"22d4bbb5-3cf0-4c06-9552-80e01980013c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"ee9a5258-997f-4465-98f1-976c5dc010c0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"aaf542f4-f7fb-4931-b598-36efb11497b0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-24T18:30:30.379 INFO:tasks.ceph.mgr.y.trial038.stderr:2026-02-24T18:30:30.377+0000 7ff39bbce140 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2026-02-24T18:30:30.381 INFO:tasks.ceph.ceph_manager.ceph:[] 2026-02-24T18:30:30.382 INFO:tasks.ceph:Waiting for OSDs to come up 2026-02-24T18:30:30.382 INFO:tasks.ceph.mgr.x.trial038.stderr:2026-02-24T18:30:30.379+0000 7f087642d140 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2026-02-24T18:30:30.434 INFO:tasks.ceph.mgr.y.trial038.stderr:2026-02-24T18:30:30.432+0000 7ff39bbce140 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2026-02-24T18:30:30.437 INFO:tasks.ceph.mgr.x.trial038.stderr:2026-02-24T18:30:30.434+0000 7f087642d140 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2026-02-24T18:30:30.684 DEBUG:teuthology.orchestra.run.trial038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-24T18:30:30.852 INFO:tasks.ceph.mgr.x.trial038.stderr:2026-02-24T18:30:30.849+0000 7f087642d140 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2026-02-24T18:30:30.852 INFO:tasks.ceph.mgr.y.trial038.stderr:2026-02-24T18:30:30.850+0000 7ff39bbce140 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2026-02-24T18:30:30.859 INFO:teuthology.misc.health.trial038.stdout: 2026-02-24T18:30:30.860 INFO:teuthology.misc.health.trial038.stdout:{"epoch":5,"fsid":"3487e59d-026b-4a4d-92d5-661e87a7c906","created":"2026-02-24T18:30:28.512730+0000","modified":"2026-02-24T18:30:30.119568+0000","last_up_change":"0.000000","last_in_change":"2026-02-24T18:30:30.119568+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"22d4bbb5-3cf0-4c06-9552-80e01980013c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"ee9a5258-997f-4465-98f1-976c5dc010c0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"aaf542f4-f7fb-4931-b598-36efb11497b0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-24T18:30:30.870 DEBUG:teuthology.misc:0 of 3 OSDs are up 2026-02-24T18:30:30.881 INFO:tasks.ceph.mgr.y.trial038.stderr:2026-02-24T18:30:30.879+0000 7ff39bbce140 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2026-02-24T18:30:30.881 INFO:tasks.ceph.mgr.x.trial038.stderr:2026-02-24T18:30:30.879+0000 7f087642d140 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2026-02-24T18:30:31.270 INFO:tasks.ceph.mgr.x.trial038.stderr:2026-02-24T18:30:31.267+0000 7f087642d140 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2026-02-24T18:30:31.274 INFO:tasks.ceph.mgr.y.trial038.stderr:2026-02-24T18:30:31.272+0000 7ff39bbce140 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2026-02-24T18:30:32.646 INFO:tasks.ceph.mgr.x.trial038.stderr:2026-02-24T18:30:32.643+0000 7f0793bb9640 -1 mgr.server handle_report got status from non-daemon mon.c 2026-02-24T18:30:32.661 INFO:tasks.ceph.mgr.x.trial038.stderr:2026-02-24T18:30:32.659+0000 7f0793bb9640 -1 mgr.server handle_report got status from non-daemon mon.b 2026-02-24T18:30:37.172 DEBUG:teuthology.orchestra.run.trial038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-24T18:30:37.461 INFO:teuthology.misc.health.trial038.stdout: 2026-02-24T18:30:37.461 INFO:teuthology.misc.health.trial038.stdout:{"epoch":5,"fsid":"3487e59d-026b-4a4d-92d5-661e87a7c906","created":"2026-02-24T18:30:28.512730+0000","modified":"2026-02-24T18:30:30.119568+0000","last_up_change":"0.000000","last_in_change":"2026-02-24T18:30:30.119568+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"22d4bbb5-3cf0-4c06-9552-80e01980013c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"ee9a5258-997f-4465-98f1-976c5dc010c0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"aaf542f4-f7fb-4931-b598-36efb11497b0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-24T18:30:37.471 DEBUG:teuthology.misc:0 of 3 OSDs are up 2026-02-24T18:30:39.878 INFO:tasks.ceph.osd.1.trial038.stderr:--00:00:00:09.638 43563-- Warning: zero subprog, missing DW_AT_abstract_origin in DW_TAG_inlined_subroutine in /usr/lib/debug/.build-id/8c/c8adc374e188776041495b4f4f83fb04e98a9a.debug 2026-02-24T18:30:39.979 INFO:tasks.ceph.osd.0.trial038.stderr:--00:00:00:09.743 43562-- Warning: zero subprog, missing DW_AT_abstract_origin in DW_TAG_inlined_subroutine in /usr/lib/debug/.build-id/8c/c8adc374e188776041495b4f4f83fb04e98a9a.debug 2026-02-24T18:30:40.040 INFO:tasks.ceph.osd.2.trial038.stderr:--00:00:00:09.799 43564-- Warning: zero subprog, missing DW_AT_abstract_origin in DW_TAG_inlined_subroutine in /usr/lib/debug/.build-id/8c/c8adc374e188776041495b4f4f83fb04e98a9a.debug 2026-02-24T18:30:43.774 DEBUG:teuthology.orchestra.run.trial038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-24T18:30:44.057 INFO:teuthology.misc.health.trial038.stdout: 2026-02-24T18:30:44.058 INFO:teuthology.misc.health.trial038.stdout:{"epoch":5,"fsid":"3487e59d-026b-4a4d-92d5-661e87a7c906","created":"2026-02-24T18:30:28.512730+0000","modified":"2026-02-24T18:30:30.119568+0000","last_up_change":"0.000000","last_in_change":"2026-02-24T18:30:30.119568+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"22d4bbb5-3cf0-4c06-9552-80e01980013c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"ee9a5258-997f-4465-98f1-976c5dc010c0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"aaf542f4-f7fb-4931-b598-36efb11497b0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-24T18:30:44.068 DEBUG:teuthology.misc:0 of 3 OSDs are up 2026-02-24T18:30:44.342 INFO:tasks.ceph.osd.1.trial038.stderr:==00:00:00:14.102 43563== Warning: unimplemented fcntl command: 1038 2026-02-24T18:30:44.415 INFO:tasks.ceph.osd.0.trial038.stderr:==00:00:00:14.180 43562== Warning: unimplemented fcntl command: 1038 2026-02-24T18:30:44.486 INFO:tasks.ceph.osd.2.trial038.stderr:==00:00:00:14.245 43564== Warning: unimplemented fcntl command: 1038 2026-02-24T18:30:44.636 INFO:tasks.ceph.osd.1.trial038.stderr:==00:00:00:14.396 43563== Warning: unimplemented fcntl command: 1038 2026-02-24T18:30:44.664 INFO:tasks.ceph.osd.1.trial038.stderr:==00:00:00:14.425 43563== Warning: unimplemented fcntl command: 1038 2026-02-24T18:30:44.687 INFO:tasks.ceph.osd.1.trial038.stderr:==00:00:00:14.447 43563== Warning: unimplemented fcntl command: 1038 2026-02-24T18:30:44.710 INFO:tasks.ceph.osd.0.trial038.stderr:==00:00:00:14.475 43562== Warning: unimplemented fcntl command: 1038 2026-02-24T18:30:44.769 INFO:tasks.ceph.osd.2.trial038.stderr:==00:00:00:14.528 43564== Warning: unimplemented fcntl command: 1038 2026-02-24T18:30:44.790 INFO:tasks.ceph.osd.2.trial038.stderr:==00:00:00:14.549 43564== Warning: unimplemented fcntl command: 1038 2026-02-24T18:30:44.812 INFO:tasks.ceph.osd.2.trial038.stderr:==00:00:00:14.571 43564== Warning: unimplemented fcntl command: 1038 2026-02-24T18:30:44.951 INFO:tasks.ceph.osd.1.trial038.stderr:==00:00:00:14.711 43563== Warning: unimplemented fcntl command: 1038 2026-02-24T18:30:44.975 INFO:tasks.ceph.osd.0.trial038.stderr:==00:00:00:14.740 43562== Warning: unimplemented fcntl command: 1038 2026-02-24T18:30:45.002 INFO:tasks.ceph.osd.0.trial038.stderr:==00:00:00:14.766 43562== Warning: unimplemented fcntl command: 1038 2026-02-24T18:30:45.077 INFO:tasks.ceph.osd.2.trial038.stderr:==00:00:00:14.835 43564== Warning: unimplemented fcntl command: 1038 2026-02-24T18:30:45.100 INFO:tasks.ceph.osd.2.trial038.stderr:==00:00:00:14.859 43564== Warning: unimplemented fcntl command: 1038 2026-02-24T18:30:45.116 INFO:tasks.ceph.osd.2.trial038.stderr:==00:00:00:14.875 43564== Warning: unimplemented fcntl command: 1038 2026-02-24T18:30:45.216 INFO:tasks.ceph.osd.1.trial038.stderr:==00:00:00:14.976 43563== Warning: unimplemented fcntl command: 1038 2026-02-24T18:30:45.231 INFO:tasks.ceph.osd.1.trial038.stderr:==00:00:00:14.992 43563== Warning: unimplemented fcntl command: 1038 2026-02-24T18:30:45.266 INFO:tasks.ceph.osd.0.trial038.stderr:==00:00:00:15.030 43562== Warning: unimplemented fcntl command: 1038 2026-02-24T18:30:45.537 INFO:tasks.ceph.osd.0.trial038.stderr:==00:00:00:15.301 43562== Warning: unimplemented fcntl command: 1038 2026-02-24T18:30:45.553 INFO:tasks.ceph.osd.0.trial038.stderr:==00:00:00:15.317 43562== Warning: unimplemented fcntl command: 1038 2026-02-24T18:30:45.656 INFO:tasks.ceph.osd.2.trial038.stderr:2026-02-24T18:30:45.652+0000 77b8940 -1 Falling back to public interface 2026-02-24T18:30:45.773 INFO:tasks.ceph.osd.1.trial038.stderr:2026-02-24T18:30:45.769+0000 77b8940 -1 Falling back to public interface 2026-02-24T18:30:46.095 INFO:tasks.ceph.osd.0.trial038.stderr:2026-02-24T18:30:46.091+0000 77b8940 -1 Falling back to public interface 2026-02-24T18:30:50.371 DEBUG:teuthology.orchestra.run.trial038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-24T18:30:50.453 INFO:tasks.ceph.osd.2.trial038.stderr:==00:00:00:20.212 43564== Warning: unimplemented fcntl command: 1038 2026-02-24T18:30:50.612 INFO:tasks.ceph.osd.1.trial038.stderr:==00:00:00:20.372 43563== Warning: unimplemented fcntl command: 1038 2026-02-24T18:30:50.647 INFO:teuthology.misc.health.trial038.stdout: 2026-02-24T18:30:50.647 INFO:teuthology.misc.health.trial038.stdout:{"epoch":5,"fsid":"3487e59d-026b-4a4d-92d5-661e87a7c906","created":"2026-02-24T18:30:28.512730+0000","modified":"2026-02-24T18:30:30.119568+0000","last_up_change":"0.000000","last_in_change":"2026-02-24T18:30:30.119568+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"22d4bbb5-3cf0-4c06-9552-80e01980013c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"ee9a5258-997f-4465-98f1-976c5dc010c0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"aaf542f4-f7fb-4931-b598-36efb11497b0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-24T18:30:50.657 DEBUG:teuthology.misc:0 of 3 OSDs are up 2026-02-24T18:30:50.791 INFO:tasks.ceph.osd.2.trial038.stderr:==00:00:00:20.550 43564== Warning: unimplemented fcntl command: 1038 2026-02-24T18:30:50.905 INFO:tasks.ceph.osd.0.trial038.stderr:==00:00:00:20.670 43562== Warning: unimplemented fcntl command: 1038 2026-02-24T18:30:50.941 INFO:tasks.ceph.osd.1.trial038.stderr:==00:00:00:20.701 43563== Warning: unimplemented fcntl command: 1038 2026-02-24T18:30:51.093 INFO:tasks.ceph.osd.2.trial038.stderr:==00:00:00:20.851 43564== Warning: unimplemented fcntl command: 1038 2026-02-24T18:30:51.227 INFO:tasks.ceph.osd.0.trial038.stderr:==00:00:00:20.992 43562== Warning: unimplemented fcntl command: 1038 2026-02-24T18:30:51.241 INFO:tasks.ceph.osd.1.trial038.stderr:==00:00:00:21.001 43563== Warning: unimplemented fcntl command: 1038 2026-02-24T18:30:51.371 INFO:tasks.ceph.osd.2.trial038.stderr:==00:00:00:21.130 43564== Warning: unimplemented fcntl command: 1038 2026-02-24T18:30:51.525 INFO:tasks.ceph.osd.1.trial038.stderr:==00:00:00:21.285 43563== Warning: unimplemented fcntl command: 1038 2026-02-24T18:30:51.544 INFO:tasks.ceph.osd.0.trial038.stderr:==00:00:00:21.308 43562== Warning: unimplemented fcntl command: 1038 2026-02-24T18:30:51.645 INFO:tasks.ceph.osd.2.trial038.stderr:==00:00:00:21.404 43564== Warning: unimplemented fcntl command: 1038 2026-02-24T18:30:51.799 INFO:tasks.ceph.osd.1.trial038.stderr:==00:00:00:21.559 43563== Warning: unimplemented fcntl command: 1038 2026-02-24T18:30:51.829 INFO:tasks.ceph.osd.0.trial038.stderr:==00:00:00:21.593 43562== Warning: unimplemented fcntl command: 1038 2026-02-24T18:30:51.920 INFO:tasks.ceph.osd.2.trial038.stderr:==00:00:00:21.679 43564== Warning: unimplemented fcntl command: 1038 2026-02-24T18:30:52.074 INFO:tasks.ceph.osd.1.trial038.stderr:==00:00:00:21.834 43563== Warning: unimplemented fcntl command: 1038 2026-02-24T18:30:52.106 INFO:tasks.ceph.osd.0.trial038.stderr:==00:00:00:21.870 43562== Warning: unimplemented fcntl command: 1038 2026-02-24T18:30:52.198 INFO:tasks.ceph.osd.2.trial038.stderr:==00:00:00:21.957 43564== Warning: unimplemented fcntl command: 1038 2026-02-24T18:30:52.348 INFO:tasks.ceph.osd.1.trial038.stderr:==00:00:00:22.108 43563== Warning: unimplemented fcntl command: 1038 2026-02-24T18:30:52.380 INFO:tasks.ceph.osd.0.trial038.stderr:==00:00:00:22.144 43562== Warning: unimplemented fcntl command: 1038 2026-02-24T18:30:52.480 INFO:tasks.ceph.osd.2.trial038.stderr:==00:00:00:22.239 43564== Warning: unimplemented fcntl command: 1038 2026-02-24T18:30:52.622 INFO:tasks.ceph.osd.1.trial038.stderr:==00:00:00:22.382 43563== Warning: unimplemented fcntl command: 1038 2026-02-24T18:30:52.662 INFO:tasks.ceph.osd.0.trial038.stderr:==00:00:00:22.426 43562== Warning: unimplemented fcntl command: 1038 2026-02-24T18:30:52.755 INFO:tasks.ceph.osd.2.trial038.stderr:==00:00:00:22.514 43564== Warning: unimplemented fcntl command: 1038 2026-02-24T18:30:52.896 INFO:tasks.ceph.osd.1.trial038.stderr:==00:00:00:22.656 43563== Warning: unimplemented fcntl command: 1038 2026-02-24T18:30:52.936 INFO:tasks.ceph.osd.0.trial038.stderr:==00:00:00:22.700 43562== Warning: unimplemented fcntl command: 1038 2026-02-24T18:30:53.143 INFO:tasks.ceph.osd.2.trial038.stderr:==00:00:00:22.902 43564== Warning: unimplemented fcntl command: 1038 2026-02-24T18:30:53.176 INFO:tasks.ceph.osd.2.trial038.stderr:==00:00:00:22.935 43564== Warning: unimplemented fcntl command: 1038 2026-02-24T18:30:53.219 INFO:tasks.ceph.osd.2.trial038.stderr:==00:00:00:22.978 43564== Warning: unimplemented fcntl command: 1038 2026-02-24T18:30:53.233 INFO:tasks.ceph.osd.0.trial038.stderr:==00:00:00:22.997 43562== Warning: unimplemented fcntl command: 1038 2026-02-24T18:30:53.240 INFO:tasks.ceph.osd.2.trial038.stderr:==00:00:00:22.999 43564== Warning: unimplemented fcntl command: 1038 2026-02-24T18:30:53.244 INFO:tasks.ceph.osd.2.trial038.stderr:==00:00:00:23.003 43564== Warning: unimplemented fcntl command: 1038 2026-02-24T18:30:53.292 INFO:tasks.ceph.osd.1.trial038.stderr:==00:00:00:23.053 43563== Warning: unimplemented fcntl command: 1038 2026-02-24T18:30:53.330 INFO:tasks.ceph.osd.1.trial038.stderr:==00:00:00:23.091 43563== Warning: unimplemented fcntl command: 1038 2026-02-24T18:30:53.365 INFO:tasks.ceph.osd.1.trial038.stderr:==00:00:00:23.125 43563== Warning: unimplemented fcntl command: 1038 2026-02-24T18:30:53.398 INFO:tasks.ceph.osd.1.trial038.stderr:==00:00:00:23.158 43563== Warning: unimplemented fcntl command: 1038 2026-02-24T18:30:53.402 INFO:tasks.ceph.osd.1.trial038.stderr:==00:00:00:23.163 43563== Warning: unimplemented fcntl command: 1038 2026-02-24T18:30:53.638 INFO:tasks.ceph.osd.0.trial038.stderr:==00:00:00:23.403 43562== Warning: unimplemented fcntl command: 1038 2026-02-24T18:30:53.675 INFO:tasks.ceph.osd.0.trial038.stderr:==00:00:00:23.440 43562== Warning: unimplemented fcntl command: 1038 2026-02-24T18:30:53.711 INFO:tasks.ceph.osd.0.trial038.stderr:==00:00:00:23.475 43562== Warning: unimplemented fcntl command: 1038 2026-02-24T18:30:53.741 INFO:tasks.ceph.osd.0.trial038.stderr:==00:00:00:23.506 43562== Warning: unimplemented fcntl command: 1038 2026-02-24T18:30:53.746 INFO:tasks.ceph.osd.0.trial038.stderr:==00:00:00:23.510 43562== Warning: unimplemented fcntl command: 1038 2026-02-24T18:30:56.960 DEBUG:teuthology.orchestra.run.trial038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-24T18:30:57.238 INFO:teuthology.misc.health.trial038.stdout: 2026-02-24T18:30:57.239 INFO:teuthology.misc.health.trial038.stdout:{"epoch":5,"fsid":"3487e59d-026b-4a4d-92d5-661e87a7c906","created":"2026-02-24T18:30:28.512730+0000","modified":"2026-02-24T18:30:30.119568+0000","last_up_change":"0.000000","last_in_change":"2026-02-24T18:30:30.119568+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"22d4bbb5-3cf0-4c06-9552-80e01980013c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"ee9a5258-997f-4465-98f1-976c5dc010c0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"aaf542f4-f7fb-4931-b598-36efb11497b0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-24T18:30:57.250 DEBUG:teuthology.misc:0 of 3 OSDs are up 2026-02-24T18:31:03.552 DEBUG:teuthology.orchestra.run.trial038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-24T18:31:03.838 INFO:teuthology.misc.health.trial038.stdout: 2026-02-24T18:31:03.838 INFO:teuthology.misc.health.trial038.stdout:{"epoch":5,"fsid":"3487e59d-026b-4a4d-92d5-661e87a7c906","created":"2026-02-24T18:30:28.512730+0000","modified":"2026-02-24T18:30:30.119568+0000","last_up_change":"0.000000","last_in_change":"2026-02-24T18:30:30.119568+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"22d4bbb5-3cf0-4c06-9552-80e01980013c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"ee9a5258-997f-4465-98f1-976c5dc010c0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"aaf542f4-f7fb-4931-b598-36efb11497b0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-24T18:31:03.848 DEBUG:teuthology.misc:0 of 3 OSDs are up 2026-02-24T18:31:04.742 INFO:tasks.ceph.osd.2.trial038.stderr:==00:00:00:34.500 43564== Warning: unimplemented fcntl command: 1038 2026-02-24T18:31:04.919 INFO:tasks.ceph.osd.1.trial038.stderr:==00:00:00:34.679 43563== Warning: unimplemented fcntl command: 1038 2026-02-24T18:31:05.288 INFO:tasks.ceph.osd.0.trial038.stderr:==00:00:00:35.052 43562== Warning: unimplemented fcntl command: 1038 2026-02-24T18:31:10.151 DEBUG:teuthology.orchestra.run.trial038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-24T18:31:10.439 INFO:teuthology.misc.health.trial038.stdout: 2026-02-24T18:31:10.439 INFO:teuthology.misc.health.trial038.stdout:{"epoch":5,"fsid":"3487e59d-026b-4a4d-92d5-661e87a7c906","created":"2026-02-24T18:30:28.512730+0000","modified":"2026-02-24T18:30:30.119568+0000","last_up_change":"0.000000","last_in_change":"2026-02-24T18:30:30.119568+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"22d4bbb5-3cf0-4c06-9552-80e01980013c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"ee9a5258-997f-4465-98f1-976c5dc010c0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"aaf542f4-f7fb-4931-b598-36efb11497b0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-24T18:31:10.450 DEBUG:teuthology.misc:0 of 3 OSDs are up 2026-02-24T18:31:16.752 DEBUG:teuthology.orchestra.run.trial038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-24T18:31:17.046 INFO:teuthology.misc.health.trial038.stdout: 2026-02-24T18:31:17.046 INFO:teuthology.misc.health.trial038.stdout:{"epoch":5,"fsid":"3487e59d-026b-4a4d-92d5-661e87a7c906","created":"2026-02-24T18:30:28.512730+0000","modified":"2026-02-24T18:30:30.119568+0000","last_up_change":"0.000000","last_in_change":"2026-02-24T18:30:30.119568+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"22d4bbb5-3cf0-4c06-9552-80e01980013c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"ee9a5258-997f-4465-98f1-976c5dc010c0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"aaf542f4-f7fb-4931-b598-36efb11497b0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-24T18:31:17.058 DEBUG:teuthology.misc:0 of 3 OSDs are up 2026-02-24T18:31:23.360 DEBUG:teuthology.orchestra.run.trial038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-24T18:31:23.669 INFO:teuthology.misc.health.trial038.stdout: 2026-02-24T18:31:23.670 INFO:teuthology.misc.health.trial038.stdout:{"epoch":5,"fsid":"3487e59d-026b-4a4d-92d5-661e87a7c906","created":"2026-02-24T18:30:28.512730+0000","modified":"2026-02-24T18:30:30.119568+0000","last_up_change":"0.000000","last_in_change":"2026-02-24T18:30:30.119568+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"22d4bbb5-3cf0-4c06-9552-80e01980013c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"ee9a5258-997f-4465-98f1-976c5dc010c0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"aaf542f4-f7fb-4931-b598-36efb11497b0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-24T18:31:23.680 DEBUG:teuthology.misc:0 of 3 OSDs are up 2026-02-24T18:31:29.983 DEBUG:teuthology.orchestra.run.trial038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-24T18:31:30.285 INFO:teuthology.misc.health.trial038.stdout: 2026-02-24T18:31:30.285 INFO:teuthology.misc.health.trial038.stdout:{"epoch":5,"fsid":"3487e59d-026b-4a4d-92d5-661e87a7c906","created":"2026-02-24T18:30:28.512730+0000","modified":"2026-02-24T18:30:30.119568+0000","last_up_change":"0.000000","last_in_change":"2026-02-24T18:30:30.119568+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"22d4bbb5-3cf0-4c06-9552-80e01980013c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"ee9a5258-997f-4465-98f1-976c5dc010c0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"aaf542f4-f7fb-4931-b598-36efb11497b0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-24T18:31:30.295 DEBUG:teuthology.misc:0 of 3 OSDs are up 2026-02-24T18:31:36.598 DEBUG:teuthology.orchestra.run.trial038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-24T18:31:36.892 INFO:teuthology.misc.health.trial038.stdout: 2026-02-24T18:31:36.893 INFO:teuthology.misc.health.trial038.stdout:{"epoch":5,"fsid":"3487e59d-026b-4a4d-92d5-661e87a7c906","created":"2026-02-24T18:30:28.512730+0000","modified":"2026-02-24T18:30:30.119568+0000","last_up_change":"0.000000","last_in_change":"2026-02-24T18:30:30.119568+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"22d4bbb5-3cf0-4c06-9552-80e01980013c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"ee9a5258-997f-4465-98f1-976c5dc010c0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"aaf542f4-f7fb-4931-b598-36efb11497b0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-24T18:31:36.904 DEBUG:teuthology.misc:0 of 3 OSDs are up 2026-02-24T18:31:43.207 DEBUG:teuthology.orchestra.run.trial038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-24T18:31:43.494 INFO:teuthology.misc.health.trial038.stdout: 2026-02-24T18:31:43.494 INFO:teuthology.misc.health.trial038.stdout:{"epoch":5,"fsid":"3487e59d-026b-4a4d-92d5-661e87a7c906","created":"2026-02-24T18:30:28.512730+0000","modified":"2026-02-24T18:30:30.119568+0000","last_up_change":"0.000000","last_in_change":"2026-02-24T18:30:30.119568+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"22d4bbb5-3cf0-4c06-9552-80e01980013c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"ee9a5258-997f-4465-98f1-976c5dc010c0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"aaf542f4-f7fb-4931-b598-36efb11497b0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-24T18:31:43.506 DEBUG:teuthology.misc:0 of 3 OSDs are up 2026-02-24T18:31:49.808 DEBUG:teuthology.orchestra.run.trial038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-24T18:31:50.095 INFO:teuthology.misc.health.trial038.stdout: 2026-02-24T18:31:50.095 INFO:teuthology.misc.health.trial038.stdout:{"epoch":5,"fsid":"3487e59d-026b-4a4d-92d5-661e87a7c906","created":"2026-02-24T18:30:28.512730+0000","modified":"2026-02-24T18:30:30.119568+0000","last_up_change":"0.000000","last_in_change":"2026-02-24T18:30:30.119568+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"22d4bbb5-3cf0-4c06-9552-80e01980013c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"ee9a5258-997f-4465-98f1-976c5dc010c0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"aaf542f4-f7fb-4931-b598-36efb11497b0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-24T18:31:50.106 DEBUG:teuthology.misc:0 of 3 OSDs are up 2026-02-24T18:31:51.224 INFO:tasks.ceph.osd.2.trial038.stderr:2026-02-24T18:31:51.222+0000 77b8940 -1 osd.2 0 log_to_monitors true 2026-02-24T18:31:51.377 INFO:tasks.ceph.osd.1.trial038.stderr:2026-02-24T18:31:51.373+0000 77b8940 -1 osd.1 0 log_to_monitors true 2026-02-24T18:31:51.922 INFO:tasks.ceph.osd.0.trial038.stderr:2026-02-24T18:31:51.916+0000 77b8940 -1 osd.0 0 log_to_monitors true 2026-02-24T18:31:53.239 INFO:tasks.ceph.osd.2.trial038.stderr:2026-02-24T18:31:53.236+0000 d2b0640 -1 osd.2 0 waiting for initial osdmap 2026-02-24T18:31:53.250 INFO:tasks.ceph.osd.1.trial038.stderr:2026-02-24T18:31:53.248+0000 d2b0640 -1 osd.1 0 waiting for initial osdmap 2026-02-24T18:31:53.424 INFO:tasks.ceph.osd.1.trial038.stderr:2026-02-24T18:31:53.415+0000 15c7e640 -1 osd.1 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-24T18:31:53.442 INFO:tasks.ceph.osd.2.trial038.stderr:2026-02-24T18:31:53.435+0000 15c7e640 -1 osd.2 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-24T18:31:54.360 INFO:tasks.ceph.osd.0.trial038.stderr:2026-02-24T18:31:54.357+0000 d2b0640 -1 osd.0 0 waiting for initial osdmap 2026-02-24T18:31:54.547 INFO:tasks.ceph.osd.0.trial038.stderr:2026-02-24T18:31:54.539+0000 15c7e640 -1 osd.0 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-24T18:31:56.409 DEBUG:teuthology.orchestra.run.trial038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-24T18:31:56.721 INFO:teuthology.misc.health.trial038.stdout: 2026-02-24T18:31:56.721 INFO:teuthology.misc.health.trial038.stdout:{"epoch":10,"fsid":"3487e59d-026b-4a4d-92d5-661e87a7c906","created":"2026-02-24T18:30:28.512730+0000","modified":"2026-02-24T18:31:56.258459+0000","last_up_change":"2026-02-24T18:31:55.194260+0000","last_in_change":"2026-02-24T18:30:30.119568+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-24T18:31:55.621631+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"10","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"22d4bbb5-3cf0-4c06-9552-80e01980013c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.38:6818","nonce":3879277939},{"type":"v1","addr":"10.20.193.38:6819","nonce":3879277939}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.38:6820","nonce":3879277939},{"type":"v1","addr":"10.20.193.38:6821","nonce":3879277939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.38:6824","nonce":3879277939},{"type":"v1","addr":"10.20.193.38:6825","nonce":3879277939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.38:6822","nonce":3879277939},{"type":"v1","addr":"10.20.193.38:6823","nonce":3879277939}]},"public_addr":"10.20.193.38:6819/3879277939","cluster_addr":"10.20.193.38:6821/3879277939","heartbeat_back_addr":"10.20.193.38:6825/3879277939","heartbeat_front_addr":"10.20.193.38:6823/3879277939","state":["exists","up"]},{"osd":1,"uuid":"ee9a5258-997f-4465-98f1-976c5dc010c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.38:6810","nonce":1372092871},{"type":"v1","addr":"10.20.193.38:6811","nonce":1372092871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.38:6812","nonce":1372092871},{"type":"v1","addr":"10.20.193.38:6813","nonce":1372092871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.38:6816","nonce":1372092871},{"type":"v1","addr":"10.20.193.38:6817","nonce":1372092871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.38:6814","nonce":1372092871},{"type":"v1","addr":"10.20.193.38:6815","nonce":1372092871}]},"public_addr":"10.20.193.38:6811/1372092871","cluster_addr":"10.20.193.38:6813/1372092871","heartbeat_back_addr":"10.20.193.38:6817/1372092871","heartbeat_front_addr":"10.20.193.38:6815/1372092871","state":["exists","up"]},{"osd":2,"uuid":"aaf542f4-f7fb-4931-b598-36efb11497b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.38:6802","nonce":1853955099},{"type":"v1","addr":"10.20.193.38:6803","nonce":1853955099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.38:6804","nonce":1853955099},{"type":"v1","addr":"10.20.193.38:6805","nonce":1853955099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.38:6808","nonce":1853955099},{"type":"v1","addr":"10.20.193.38:6809","nonce":1853955099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.38:6806","nonce":1853955099},{"type":"v1","addr":"10.20.193.38:6807","nonce":1853955099}]},"public_addr":"10.20.193.38:6803/1853955099","cluster_addr":"10.20.193.38:6805/1853955099","heartbeat_back_addr":"10.20.193.38:6809/1853955099","heartbeat_front_addr":"10.20.193.38:6807/1853955099","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-24T18:31:53.310115+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-24T18:31:52.758406+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-24T18:31:52.648429+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-24T18:31:56.730 DEBUG:teuthology.misc:3 of 3 OSDs are up 2026-02-24T18:31:56.730 INFO:tasks.ceph:Creating RBD pool 2026-02-24T18:31:56.731 DEBUG:teuthology.orchestra.run.trial038:> sudo ceph --cluster ceph osd pool create rbd 8 2026-02-24T18:31:57.427 INFO:teuthology.orchestra.run.trial038.stderr:pool 'rbd' created 2026-02-24T18:31:57.438 DEBUG:teuthology.orchestra.run.trial038:> rbd --cluster ceph pool init rbd 2026-02-24T18:32:01.036 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2026-02-24T18:32:01.036 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2026-02-24T18:32:01.037 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2026-02-24T18:32:01.037 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2026-02-24T18:32:01.038 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2026-02-24T18:32:01.410 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:32:01.419 INFO:teuthology.orchestra.run.trial038.stdout:{"epoch":5,"flags":0,"active_gid":4100,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.38:6800","nonce":2635059424},{"type":"v1","addr":"10.20.193.38:6801","nonce":2635059424}]},"active_addr":"10.20.193.38:6801/2635059424","active_change":"2026-02-24T18:30:31.339448+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4099,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_pg_upmap_activity":{"name":"update_pg_upmap_activity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Updates pg_upmap activity stats to be used in `balancer status detail`","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"quay.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":true,"error_string":"","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number. if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_pg_upmap_activity":{"name":"update_pg_upmap_activity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Updates pg_upmap activity stats to be used in `balancer status detail`","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"quay.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":true,"error_string":"","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number. if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"force_disabled_modules":{},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"10.20.193.38:0","nonce":2830524520}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"10.20.193.38:0","nonce":4287784790}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"10.20.193.38:0","nonce":1039326155}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"10.20.193.38:0","nonce":2230614572}]}]} 2026-02-24T18:32:01.426 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2026-02-24T18:32:01.426 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2026-02-24T18:32:01.427 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-24T18:32:01.797 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:32:01.797 INFO:teuthology.orchestra.run.trial038.stdout:{"epoch":14,"fsid":"3487e59d-026b-4a4d-92d5-661e87a7c906","created":"2026-02-24T18:30:28.512730+0000","modified":"2026-02-24T18:32:00.521984+0000","last_up_change":"2026-02-24T18:31:55.194260+0000","last_in_change":"2026-02-24T18:30:30.119568+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-24T18:31:55.621631+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2026-02-24T18:31:57.010626+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":14,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"22d4bbb5-3cf0-4c06-9552-80e01980013c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.38:6818","nonce":3879277939},{"type":"v1","addr":"10.20.193.38:6819","nonce":3879277939}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.38:6820","nonce":3879277939},{"type":"v1","addr":"10.20.193.38:6821","nonce":3879277939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.38:6824","nonce":3879277939},{"type":"v1","addr":"10.20.193.38:6825","nonce":3879277939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.38:6822","nonce":3879277939},{"type":"v1","addr":"10.20.193.38:6823","nonce":3879277939}]},"public_addr":"10.20.193.38:6819/3879277939","cluster_addr":"10.20.193.38:6821/3879277939","heartbeat_back_addr":"10.20.193.38:6825/3879277939","heartbeat_front_addr":"10.20.193.38:6823/3879277939","state":["exists","up"]},{"osd":1,"uuid":"ee9a5258-997f-4465-98f1-976c5dc010c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.38:6810","nonce":1372092871},{"type":"v1","addr":"10.20.193.38:6811","nonce":1372092871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.38:6812","nonce":1372092871},{"type":"v1","addr":"10.20.193.38:6813","nonce":1372092871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.38:6816","nonce":1372092871},{"type":"v1","addr":"10.20.193.38:6817","nonce":1372092871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.38:6814","nonce":1372092871},{"type":"v1","addr":"10.20.193.38:6815","nonce":1372092871}]},"public_addr":"10.20.193.38:6811/1372092871","cluster_addr":"10.20.193.38:6813/1372092871","heartbeat_back_addr":"10.20.193.38:6817/1372092871","heartbeat_front_addr":"10.20.193.38:6815/1372092871","state":["exists","up"]},{"osd":2,"uuid":"aaf542f4-f7fb-4931-b598-36efb11497b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.38:6802","nonce":1853955099},{"type":"v1","addr":"10.20.193.38:6803","nonce":1853955099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.38:6804","nonce":1853955099},{"type":"v1","addr":"10.20.193.38:6805","nonce":1853955099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.38:6808","nonce":1853955099},{"type":"v1","addr":"10.20.193.38:6809","nonce":1853955099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.38:6806","nonce":1853955099},{"type":"v1","addr":"10.20.193.38:6807","nonce":1853955099}]},"public_addr":"10.20.193.38:6803/1853955099","cluster_addr":"10.20.193.38:6805/1853955099","heartbeat_back_addr":"10.20.193.38:6809/1853955099","heartbeat_front_addr":"10.20.193.38:6807/1853955099","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-24T18:31:53.310115+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-24T18:31:52.758406+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-24T18:31:52.648429+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-24T18:32:01.808 INFO:tasks.ceph.ceph_manager.ceph:all up! 2026-02-24T18:32:01.808 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-24T18:32:02.131 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:32:02.131 INFO:teuthology.orchestra.run.trial038.stdout:{"epoch":14,"fsid":"3487e59d-026b-4a4d-92d5-661e87a7c906","created":"2026-02-24T18:30:28.512730+0000","modified":"2026-02-24T18:32:00.521984+0000","last_up_change":"2026-02-24T18:31:55.194260+0000","last_in_change":"2026-02-24T18:30:30.119568+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-24T18:31:55.621631+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2026-02-24T18:31:57.010626+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":14,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"22d4bbb5-3cf0-4c06-9552-80e01980013c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.38:6818","nonce":3879277939},{"type":"v1","addr":"10.20.193.38:6819","nonce":3879277939}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.38:6820","nonce":3879277939},{"type":"v1","addr":"10.20.193.38:6821","nonce":3879277939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.38:6824","nonce":3879277939},{"type":"v1","addr":"10.20.193.38:6825","nonce":3879277939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.38:6822","nonce":3879277939},{"type":"v1","addr":"10.20.193.38:6823","nonce":3879277939}]},"public_addr":"10.20.193.38:6819/3879277939","cluster_addr":"10.20.193.38:6821/3879277939","heartbeat_back_addr":"10.20.193.38:6825/3879277939","heartbeat_front_addr":"10.20.193.38:6823/3879277939","state":["exists","up"]},{"osd":1,"uuid":"ee9a5258-997f-4465-98f1-976c5dc010c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.38:6810","nonce":1372092871},{"type":"v1","addr":"10.20.193.38:6811","nonce":1372092871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.38:6812","nonce":1372092871},{"type":"v1","addr":"10.20.193.38:6813","nonce":1372092871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.38:6816","nonce":1372092871},{"type":"v1","addr":"10.20.193.38:6817","nonce":1372092871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.38:6814","nonce":1372092871},{"type":"v1","addr":"10.20.193.38:6815","nonce":1372092871}]},"public_addr":"10.20.193.38:6811/1372092871","cluster_addr":"10.20.193.38:6813/1372092871","heartbeat_back_addr":"10.20.193.38:6817/1372092871","heartbeat_front_addr":"10.20.193.38:6815/1372092871","state":["exists","up"]},{"osd":2,"uuid":"aaf542f4-f7fb-4931-b598-36efb11497b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.38:6802","nonce":1853955099},{"type":"v1","addr":"10.20.193.38:6803","nonce":1853955099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.38:6804","nonce":1853955099},{"type":"v1","addr":"10.20.193.38:6805","nonce":1853955099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.38:6808","nonce":1853955099},{"type":"v1","addr":"10.20.193.38:6809","nonce":1853955099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.38:6806","nonce":1853955099},{"type":"v1","addr":"10.20.193.38:6807","nonce":1853955099}]},"public_addr":"10.20.193.38:6803/1853955099","cluster_addr":"10.20.193.38:6805/1853955099","heartbeat_back_addr":"10.20.193.38:6809/1853955099","heartbeat_front_addr":"10.20.193.38:6807/1853955099","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-24T18:31:53.310115+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-24T18:31:52.758406+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-24T18:31:52.648429+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-24T18:32:02.143 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2026-02-24T18:32:02.144 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2026-02-24T18:32:02.145 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2026-02-24T18:32:02.295 INFO:teuthology.orchestra.run.trial038.stdout:34359738372 2026-02-24T18:32:02.296 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2026-02-24T18:32:02.297 INFO:teuthology.orchestra.run.trial038.stdout:38654705667 2026-02-24T18:32:02.298 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2026-02-24T18:32:02.307 INFO:teuthology.orchestra.run.trial038.stdout:34359738373 2026-02-24T18:32:02.307 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2026-02-24T18:32:02.584 INFO:teuthology.orchestra.run.trial038.stdout:34359738371 2026-02-24T18:32:02.596 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738372 got 34359738371 for osd.1 2026-02-24T18:32:02.596 INFO:teuthology.orchestra.run.trial038.stdout:38654705666 2026-02-24T18:32:02.606 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705667 got 38654705666 for osd.0 2026-02-24T18:32:02.642 INFO:teuthology.orchestra.run.trial038.stdout:34359738372 2026-02-24T18:32:02.652 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738373 got 34359738372 for osd.2 2026-02-24T18:32:03.597 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2026-02-24T18:32:03.606 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2026-02-24T18:32:03.653 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2026-02-24T18:32:03.886 INFO:teuthology.orchestra.run.trial038.stdout:34359738371 2026-02-24T18:32:03.896 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738372 got 34359738371 for osd.1 2026-02-24T18:32:03.921 INFO:teuthology.orchestra.run.trial038.stdout:38654705666 2026-02-24T18:32:03.932 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705667 got 38654705666 for osd.0 2026-02-24T18:32:03.964 INFO:teuthology.orchestra.run.trial038.stdout:34359738372 2026-02-24T18:32:03.974 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738373 got 34359738372 for osd.2 2026-02-24T18:32:04.898 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2026-02-24T18:32:04.932 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2026-02-24T18:32:04.974 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2026-02-24T18:32:05.190 INFO:teuthology.orchestra.run.trial038.stdout:34359738372 2026-02-24T18:32:05.202 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738372 got 34359738372 for osd.1 2026-02-24T18:32:05.202 DEBUG:teuthology.parallel:result is None 2026-02-24T18:32:05.223 INFO:teuthology.orchestra.run.trial038.stdout:38654705667 2026-02-24T18:32:05.233 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705667 got 38654705667 for osd.0 2026-02-24T18:32:05.234 DEBUG:teuthology.parallel:result is None 2026-02-24T18:32:05.284 INFO:teuthology.orchestra.run.trial038.stdout:34359738373 2026-02-24T18:32:05.295 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738373 got 34359738373 for osd.2 2026-02-24T18:32:05.295 DEBUG:teuthology.parallel:result is None 2026-02-24T18:32:05.295 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2026-02-24T18:32:05.295 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-24T18:32:05.568 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:32:05.569 INFO:teuthology.orchestra.run.trial038.stderr:dumped all 2026-02-24T18:32:05.577 INFO:teuthology.orchestra.run.trial038.stdout:{"pg_ready":true,"pg_map":{"version":49,"stamp":"2026-02-24T18:32:05.490108+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459299,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":46,"num_read_kb":37,"num_write":59,"num_write_kb":586,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":35,"ondisk_log_size":35,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":314572800,"kb_used":82432,"kb_used_data":1272,"kb_used_omap":4,"kb_used_meta":80827,"kb_avail":314490368,"statfs":{"total":322122547200,"available":322038136832,"internally_reserved":0,"allocated":1302528,"data_stored":1025095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4779,"internal_metadata":82767189},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 0 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 1 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":131099,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":17,"num_write_kb":263,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.568531"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":14,"state":"active+clean","last_fresh":"2026-02-24T18:32:01.011913+0000","last_change":"2026-02-24T18:32:01.014264+0000","last_active":"2026-02-24T18:32:01.011913+0000","last_peered":"2026-02-24T18:32:01.011913+0000","last_clean":"2026-02-24T18:32:01.011913+0000","last_became_active":"2026-02-24T18:31:58.659266+0000","last_became_peered":"2026-02-24T18:31:58.659266+0000","last_unstale":"2026-02-24T18:32:01.011913+0000","last_undegraded":"2026-02-24T18:32:01.011913+0000","last_fullsized":"2026-02-24T18:32:01.011913+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T18:31:57.356642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T18:31:57.356642+0000","last_clean_scrub_stamp":"2026-02-24T18:31:57.356642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:39:42.788531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.058966485999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":14,"state":"active+clean","last_fresh":"2026-02-24T18:32:01.014807+0000","last_change":"2026-02-24T18:32:01.018496+0000","last_active":"2026-02-24T18:32:01.014807+0000","last_peered":"2026-02-24T18:32:01.014807+0000","last_clean":"2026-02-24T18:32:01.014807+0000","last_became_active":"2026-02-24T18:31:58.566235+0000","last_became_peered":"2026-02-24T18:31:58.566235+0000","last_unstale":"2026-02-24T18:32:01.014807+0000","last_undegraded":"2026-02-24T18:32:01.014807+0000","last_fullsized":"2026-02-24T18:32:01.014807+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T18:31:57.356642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T18:31:57.356642+0000","last_clean_scrub_stamp":"2026-02-24T18:31:57.356642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:58:35.800025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.062211901999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":14,"state":"active+clean","last_fresh":"2026-02-24T18:32:01.007649+0000","last_change":"2026-02-24T18:32:01.011034+0000","last_active":"2026-02-24T18:32:01.007649+0000","last_peered":"2026-02-24T18:32:01.007649+0000","last_clean":"2026-02-24T18:32:01.007649+0000","last_became_active":"2026-02-24T18:31:58.598935+0000","last_became_peered":"2026-02-24T18:31:58.598935+0000","last_unstale":"2026-02-24T18:32:01.007649+0000","last_undegraded":"2026-02-24T18:32:01.007649+0000","last_fullsized":"2026-02-24T18:32:01.007649+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T18:31:57.356642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T18:31:57.356642+0000","last_clean_scrub_stamp":"2026-02-24T18:31:57.356642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:09:02.143983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.057800998999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":14,"state":"active+clean","last_fresh":"2026-02-24T18:32:00.993585+0000","last_change":"2026-02-24T18:32:01.005474+0000","last_active":"2026-02-24T18:32:00.993585+0000","last_peered":"2026-02-24T18:32:00.993585+0000","last_clean":"2026-02-24T18:32:00.993585+0000","last_became_active":"2026-02-24T18:31:58.569720+0000","last_became_peered":"2026-02-24T18:31:58.569720+0000","last_unstale":"2026-02-24T18:32:00.993585+0000","last_undegraded":"2026-02-24T18:32:00.993585+0000","last_fullsized":"2026-02-24T18:32:00.993585+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T18:31:57.356642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T18:31:57.356642+0000","last_clean_scrub_stamp":"2026-02-24T18:31:57.356642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:43:48.192030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.05352788,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"14'2","reported_seq":21,"reported_epoch":14,"state":"active+clean","last_fresh":"2026-02-24T18:32:00.980531+0000","last_change":"2026-02-24T18:32:00.980531+0000","last_active":"2026-02-24T18:32:00.980531+0000","last_peered":"2026-02-24T18:32:00.980531+0000","last_clean":"2026-02-24T18:32:00.980531+0000","last_became_active":"2026-02-24T18:31:58.581467+0000","last_became_peered":"2026-02-24T18:31:58.581467+0000","last_unstale":"2026-02-24T18:32:00.980531+0000","last_undegraded":"2026-02-24T18:32:00.980531+0000","last_fullsized":"2026-02-24T18:32:00.980531+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T18:31:57.356642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T18:31:57.356642+0000","last_clean_scrub_stamp":"2026-02-24T18:31:57.356642+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:11:16.111193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.039524833000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":14,"state":"active+clean","last_fresh":"2026-02-24T18:32:01.089376+0000","last_change":"2026-02-24T18:32:01.094116+0000","last_active":"2026-02-24T18:32:01.089376+0000","last_peered":"2026-02-24T18:32:01.089376+0000","last_clean":"2026-02-24T18:32:01.089376+0000","last_became_active":"2026-02-24T18:31:58.752939+0000","last_became_peered":"2026-02-24T18:31:58.752939+0000","last_unstale":"2026-02-24T18:32:01.089376+0000","last_undegraded":"2026-02-24T18:32:01.089376+0000","last_fullsized":"2026-02-24T18:32:01.089376+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T18:31:57.356642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T18:31:57.356642+0000","last_clean_scrub_stamp":"2026-02-24T18:31:57.356642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:47:09.928859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.048502458999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":14,"state":"active+clean","last_fresh":"2026-02-24T18:32:01.057354+0000","last_change":"2026-02-24T18:32:01.085916+0000","last_active":"2026-02-24T18:32:01.057354+0000","last_peered":"2026-02-24T18:32:01.057354+0000","last_clean":"2026-02-24T18:32:01.057354+0000","last_became_active":"2026-02-24T18:31:58.746390+0000","last_became_peered":"2026-02-24T18:31:58.746390+0000","last_unstale":"2026-02-24T18:32:01.057354+0000","last_undegraded":"2026-02-24T18:32:01.057354+0000","last_fullsized":"2026-02-24T18:32:01.057354+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T18:31:57.356642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T18:31:57.356642+0000","last_clean_scrub_stamp":"2026-02-24T18:31:57.356642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:09:03.999608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.041638965999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"12'1","reported_seq":20,"reported_epoch":14,"state":"active+clean","last_fresh":"2026-02-24T18:32:00.969335+0000","last_change":"2026-02-24T18:32:00.998783+0000","last_active":"2026-02-24T18:32:00.969335+0000","last_peered":"2026-02-24T18:32:00.969335+0000","last_clean":"2026-02-24T18:32:00.969335+0000","last_became_active":"2026-02-24T18:31:58.627943+0000","last_became_peered":"2026-02-24T18:31:58.627943+0000","last_unstale":"2026-02-24T18:32:00.969335+0000","last_undegraded":"2026-02-24T18:32:00.969335+0000","last_fullsized":"2026-02-24T18:32:00.969335+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T18:31:57.356642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T18:31:57.356642+0000","last_clean_scrub_stamp":"2026-02-24T18:31:57.356642+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:30:35.582393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.050969995999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"14'32","reported_seq":60,"reported_epoch":14,"state":"active+clean","last_fresh":"2026-02-24T18:32:01.718640+0000","last_change":"2026-02-24T18:31:57.756734+0000","last_active":"2026-02-24T18:32:01.718640+0000","last_peered":"2026-02-24T18:32:01.718640+0000","last_clean":"2026-02-24T18:32:01.718640+0000","last_became_active":"2026-02-24T18:31:57.737892+0000","last_became_peered":"2026-02-24T18:31:57.737892+0000","last_unstale":"2026-02-24T18:32:01.718640+0000","last_undegraded":"2026-02-24T18:32:01.718640+0000","last_fullsized":"2026-02-24T18:32:01.718640+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T18:31:56.258459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T18:31:56.258459+0000","last_clean_scrub_stamp":"2026-02-24T18:31:56.258459+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:20:14.513250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":46,"num_read_kb":37,"num_write":57,"num_write_kb":584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":46,"num_read_kb":37,"num_write":57,"num_write_kb":584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":925696,"data_stored":918560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":32,"ondisk_log_size":32,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":1,"up_from":8,"seq":34359738373,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27668,"kb_used_data":576,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":104829932,"statfs":{"total":107374182400,"available":107345850368,"internally_reserved":0,"allocated":589824,"data_stored":494798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 1 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":0,"up_from":9,"seq":38654705668,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27672,"kb_used_data":576,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":104829928,"statfs":{"total":107374182400,"available":107345846272,"internally_reserved":0,"allocated":589824,"data_stored":494798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27589065},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 0 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":2,"up_from":8,"seq":34359738374,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27092,"kb_used_data":120,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":104830508,"statfs":{"total":107374182400,"available":107346440192,"internally_reserved":0,"allocated":122880,"data_stored":35499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":462848,"data_stored":459280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":462848,"data_stored":459280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-24T18:32:05.578 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-24T18:32:05.852 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:32:05.853 INFO:teuthology.orchestra.run.trial038.stderr:dumped all 2026-02-24T18:32:05.860 INFO:teuthology.orchestra.run.trial038.stdout:{"pg_ready":true,"pg_map":{"version":49,"stamp":"2026-02-24T18:32:05.490108+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459299,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":46,"num_read_kb":37,"num_write":59,"num_write_kb":586,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":35,"ondisk_log_size":35,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":314572800,"kb_used":82432,"kb_used_data":1272,"kb_used_omap":4,"kb_used_meta":80827,"kb_avail":314490368,"statfs":{"total":322122547200,"available":322038136832,"internally_reserved":0,"allocated":1302528,"data_stored":1025095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4779,"internal_metadata":82767189},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 0 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 1 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":131099,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":17,"num_write_kb":263,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.568531"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":14,"state":"active+clean","last_fresh":"2026-02-24T18:32:01.011913+0000","last_change":"2026-02-24T18:32:01.014264+0000","last_active":"2026-02-24T18:32:01.011913+0000","last_peered":"2026-02-24T18:32:01.011913+0000","last_clean":"2026-02-24T18:32:01.011913+0000","last_became_active":"2026-02-24T18:31:58.659266+0000","last_became_peered":"2026-02-24T18:31:58.659266+0000","last_unstale":"2026-02-24T18:32:01.011913+0000","last_undegraded":"2026-02-24T18:32:01.011913+0000","last_fullsized":"2026-02-24T18:32:01.011913+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T18:31:57.356642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T18:31:57.356642+0000","last_clean_scrub_stamp":"2026-02-24T18:31:57.356642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:39:42.788531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.058966485999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":14,"state":"active+clean","last_fresh":"2026-02-24T18:32:01.014807+0000","last_change":"2026-02-24T18:32:01.018496+0000","last_active":"2026-02-24T18:32:01.014807+0000","last_peered":"2026-02-24T18:32:01.014807+0000","last_clean":"2026-02-24T18:32:01.014807+0000","last_became_active":"2026-02-24T18:31:58.566235+0000","last_became_peered":"2026-02-24T18:31:58.566235+0000","last_unstale":"2026-02-24T18:32:01.014807+0000","last_undegraded":"2026-02-24T18:32:01.014807+0000","last_fullsized":"2026-02-24T18:32:01.014807+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T18:31:57.356642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T18:31:57.356642+0000","last_clean_scrub_stamp":"2026-02-24T18:31:57.356642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:58:35.800025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.062211901999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":14,"state":"active+clean","last_fresh":"2026-02-24T18:32:01.007649+0000","last_change":"2026-02-24T18:32:01.011034+0000","last_active":"2026-02-24T18:32:01.007649+0000","last_peered":"2026-02-24T18:32:01.007649+0000","last_clean":"2026-02-24T18:32:01.007649+0000","last_became_active":"2026-02-24T18:31:58.598935+0000","last_became_peered":"2026-02-24T18:31:58.598935+0000","last_unstale":"2026-02-24T18:32:01.007649+0000","last_undegraded":"2026-02-24T18:32:01.007649+0000","last_fullsized":"2026-02-24T18:32:01.007649+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T18:31:57.356642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T18:31:57.356642+0000","last_clean_scrub_stamp":"2026-02-24T18:31:57.356642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:09:02.143983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.057800998999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":14,"state":"active+clean","last_fresh":"2026-02-24T18:32:00.993585+0000","last_change":"2026-02-24T18:32:01.005474+0000","last_active":"2026-02-24T18:32:00.993585+0000","last_peered":"2026-02-24T18:32:00.993585+0000","last_clean":"2026-02-24T18:32:00.993585+0000","last_became_active":"2026-02-24T18:31:58.569720+0000","last_became_peered":"2026-02-24T18:31:58.569720+0000","last_unstale":"2026-02-24T18:32:00.993585+0000","last_undegraded":"2026-02-24T18:32:00.993585+0000","last_fullsized":"2026-02-24T18:32:00.993585+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T18:31:57.356642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T18:31:57.356642+0000","last_clean_scrub_stamp":"2026-02-24T18:31:57.356642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:43:48.192030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.05352788,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"14'2","reported_seq":21,"reported_epoch":14,"state":"active+clean","last_fresh":"2026-02-24T18:32:00.980531+0000","last_change":"2026-02-24T18:32:00.980531+0000","last_active":"2026-02-24T18:32:00.980531+0000","last_peered":"2026-02-24T18:32:00.980531+0000","last_clean":"2026-02-24T18:32:00.980531+0000","last_became_active":"2026-02-24T18:31:58.581467+0000","last_became_peered":"2026-02-24T18:31:58.581467+0000","last_unstale":"2026-02-24T18:32:00.980531+0000","last_undegraded":"2026-02-24T18:32:00.980531+0000","last_fullsized":"2026-02-24T18:32:00.980531+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T18:31:57.356642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T18:31:57.356642+0000","last_clean_scrub_stamp":"2026-02-24T18:31:57.356642+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:11:16.111193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.039524833000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":14,"state":"active+clean","last_fresh":"2026-02-24T18:32:01.089376+0000","last_change":"2026-02-24T18:32:01.094116+0000","last_active":"2026-02-24T18:32:01.089376+0000","last_peered":"2026-02-24T18:32:01.089376+0000","last_clean":"2026-02-24T18:32:01.089376+0000","last_became_active":"2026-02-24T18:31:58.752939+0000","last_became_peered":"2026-02-24T18:31:58.752939+0000","last_unstale":"2026-02-24T18:32:01.089376+0000","last_undegraded":"2026-02-24T18:32:01.089376+0000","last_fullsized":"2026-02-24T18:32:01.089376+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T18:31:57.356642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T18:31:57.356642+0000","last_clean_scrub_stamp":"2026-02-24T18:31:57.356642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:47:09.928859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.048502458999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":14,"state":"active+clean","last_fresh":"2026-02-24T18:32:01.057354+0000","last_change":"2026-02-24T18:32:01.085916+0000","last_active":"2026-02-24T18:32:01.057354+0000","last_peered":"2026-02-24T18:32:01.057354+0000","last_clean":"2026-02-24T18:32:01.057354+0000","last_became_active":"2026-02-24T18:31:58.746390+0000","last_became_peered":"2026-02-24T18:31:58.746390+0000","last_unstale":"2026-02-24T18:32:01.057354+0000","last_undegraded":"2026-02-24T18:32:01.057354+0000","last_fullsized":"2026-02-24T18:32:01.057354+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T18:31:57.356642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T18:31:57.356642+0000","last_clean_scrub_stamp":"2026-02-24T18:31:57.356642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:09:03.999608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.041638965999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"12'1","reported_seq":20,"reported_epoch":14,"state":"active+clean","last_fresh":"2026-02-24T18:32:00.969335+0000","last_change":"2026-02-24T18:32:00.998783+0000","last_active":"2026-02-24T18:32:00.969335+0000","last_peered":"2026-02-24T18:32:00.969335+0000","last_clean":"2026-02-24T18:32:00.969335+0000","last_became_active":"2026-02-24T18:31:58.627943+0000","last_became_peered":"2026-02-24T18:31:58.627943+0000","last_unstale":"2026-02-24T18:32:00.969335+0000","last_undegraded":"2026-02-24T18:32:00.969335+0000","last_fullsized":"2026-02-24T18:32:00.969335+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T18:31:57.356642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T18:31:57.356642+0000","last_clean_scrub_stamp":"2026-02-24T18:31:57.356642+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:30:35.582393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.050969995999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"14'32","reported_seq":60,"reported_epoch":14,"state":"active+clean","last_fresh":"2026-02-24T18:32:01.718640+0000","last_change":"2026-02-24T18:31:57.756734+0000","last_active":"2026-02-24T18:32:01.718640+0000","last_peered":"2026-02-24T18:32:01.718640+0000","last_clean":"2026-02-24T18:32:01.718640+0000","last_became_active":"2026-02-24T18:31:57.737892+0000","last_became_peered":"2026-02-24T18:31:57.737892+0000","last_unstale":"2026-02-24T18:32:01.718640+0000","last_undegraded":"2026-02-24T18:32:01.718640+0000","last_fullsized":"2026-02-24T18:32:01.718640+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T18:31:56.258459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T18:31:56.258459+0000","last_clean_scrub_stamp":"2026-02-24T18:31:56.258459+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:20:14.513250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":46,"num_read_kb":37,"num_write":57,"num_write_kb":584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":46,"num_read_kb":37,"num_write":57,"num_write_kb":584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":925696,"data_stored":918560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":32,"ondisk_log_size":32,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":1,"up_from":8,"seq":34359738373,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27668,"kb_used_data":576,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":104829932,"statfs":{"total":107374182400,"available":107345850368,"internally_reserved":0,"allocated":589824,"data_stored":494798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 1 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":0,"up_from":9,"seq":38654705668,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27672,"kb_used_data":576,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":104829928,"statfs":{"total":107374182400,"available":107345846272,"internally_reserved":0,"allocated":589824,"data_stored":494798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27589065},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 0 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":2,"up_from":8,"seq":34359738374,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27092,"kb_used_data":120,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":104830508,"statfs":{"total":107374182400,"available":107346440192,"internally_reserved":0,"allocated":122880,"data_stored":35499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":462848,"data_stored":459280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":462848,"data_stored":459280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-24T18:32:05.862 INFO:tasks.ceph.ceph_manager.ceph:clean! 2026-02-24T18:32:05.862 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2026-02-24T18:32:05.862 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2026-02-24T18:32:05.862 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:32:06.161 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:32:06.161 INFO:teuthology.orchestra.run.trial038.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"2 OSD(s) experiencing slow operations in BlueStore","count":2},"muted":false}},"mutes":[]} 2026-02-24T18:32:09.172 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:32:09.477 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:32:09.477 INFO:teuthology.orchestra.run.trial038.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"2 OSD(s) experiencing slow operations in BlueStore","count":2},"muted":false}},"mutes":[]} 2026-02-24T18:32:12.488 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:32:12.786 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:32:12.786 INFO:teuthology.orchestra.run.trial038.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"2 OSD(s) experiencing slow operations in BlueStore","count":2},"muted":false}},"mutes":[]} 2026-02-24T18:32:15.797 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:32:16.089 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:32:16.089 INFO:teuthology.orchestra.run.trial038.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"2 OSD(s) experiencing slow operations in BlueStore","count":2},"muted":false}},"mutes":[]} 2026-02-24T18:32:19.099 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:32:19.398 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:32:19.398 INFO:teuthology.orchestra.run.trial038.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"2 OSD(s) experiencing slow operations in BlueStore","count":2},"muted":false}},"mutes":[]} 2026-02-24T18:32:22.409 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:32:22.704 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:32:22.704 INFO:teuthology.orchestra.run.trial038.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"2 OSD(s) experiencing slow operations in BlueStore","count":2},"muted":false}},"mutes":[]} 2026-02-24T18:32:25.716 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:32:26.015 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:32:26.015 INFO:teuthology.orchestra.run.trial038.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"2 OSD(s) experiencing slow operations in BlueStore","count":2},"muted":false}},"mutes":[]} 2026-02-24T18:32:29.026 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:32:29.328 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:32:29.328 INFO:teuthology.orchestra.run.trial038.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"2 OSD(s) experiencing slow operations in BlueStore","count":2},"muted":false}},"mutes":[]} 2026-02-24T18:32:32.339 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:32:32.638 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:32:32.638 INFO:teuthology.orchestra.run.trial038.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"2 OSD(s) experiencing slow operations in BlueStore","count":2},"muted":false}},"mutes":[]} 2026-02-24T18:32:35.648 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:32:35.945 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:32:35.946 INFO:teuthology.orchestra.run.trial038.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"2 OSD(s) experiencing slow operations in BlueStore","count":2},"muted":false}},"mutes":[]} 2026-02-24T18:32:38.957 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:32:39.255 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:32:39.255 INFO:teuthology.orchestra.run.trial038.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"2 OSD(s) experiencing slow operations in BlueStore","count":2},"muted":false}},"mutes":[]} 2026-02-24T18:32:42.266 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:32:42.564 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:32:42.564 INFO:teuthology.orchestra.run.trial038.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"2 OSD(s) experiencing slow operations in BlueStore","count":2},"muted":false}},"mutes":[]} 2026-02-24T18:32:45.576 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:32:45.877 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:32:45.877 INFO:teuthology.orchestra.run.trial038.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"2 OSD(s) experiencing slow operations in BlueStore","count":2},"muted":false}},"mutes":[]} 2026-02-24T18:32:48.889 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:32:49.189 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:32:49.189 INFO:teuthology.orchestra.run.trial038.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"2 OSD(s) experiencing slow operations in BlueStore","count":2},"muted":false}},"mutes":[]} 2026-02-24T18:32:52.201 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:32:52.501 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:32:52.501 INFO:teuthology.orchestra.run.trial038.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"2 OSD(s) experiencing slow operations in BlueStore","count":2},"muted":false}},"mutes":[]} 2026-02-24T18:32:55.513 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:32:55.808 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:32:55.808 INFO:teuthology.orchestra.run.trial038.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"2 OSD(s) experiencing slow operations in BlueStore","count":2},"muted":false}},"mutes":[]} 2026-02-24T18:32:58.818 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:32:59.115 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:32:59.116 INFO:teuthology.orchestra.run.trial038.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"2 OSD(s) experiencing slow operations in BlueStore","count":2},"muted":false}},"mutes":[]} 2026-02-24T18:33:02.127 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:33:02.426 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:33:02.426 INFO:teuthology.orchestra.run.trial038.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"2 OSD(s) experiencing slow operations in BlueStore","count":2},"muted":false}},"mutes":[]} 2026-02-24T18:33:05.436 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:33:05.733 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:33:05.733 INFO:teuthology.orchestra.run.trial038.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"2 OSD(s) experiencing slow operations in BlueStore","count":2},"muted":false}},"mutes":[]} 2026-02-24T18:33:08.744 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:33:09.041 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:33:09.042 INFO:teuthology.orchestra.run.trial038.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"2 OSD(s) experiencing slow operations in BlueStore","count":2},"muted":false}},"mutes":[]} 2026-02-24T18:33:12.053 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:33:12.348 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:33:12.349 INFO:teuthology.orchestra.run.trial038.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"2 OSD(s) experiencing slow operations in BlueStore","count":2},"muted":false}},"mutes":[]} 2026-02-24T18:33:15.359 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:33:15.657 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:33:15.657 INFO:teuthology.orchestra.run.trial038.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"2 OSD(s) experiencing slow operations in BlueStore","count":2},"muted":false}},"mutes":[]} 2026-02-24T18:33:18.667 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:33:18.970 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:33:18.970 INFO:teuthology.orchestra.run.trial038.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"2 OSD(s) experiencing slow operations in BlueStore","count":2},"muted":false}},"mutes":[]} 2026-02-24T18:33:21.982 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:33:22.272 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:33:22.273 INFO:teuthology.orchestra.run.trial038.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"2 OSD(s) experiencing slow operations in BlueStore","count":2},"muted":false}},"mutes":[]} 2026-02-24T18:33:25.283 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:33:25.586 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:33:25.586 INFO:teuthology.orchestra.run.trial038.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"2 OSD(s) experiencing slow operations in BlueStore","count":2},"muted":false}},"mutes":[]} 2026-02-24T18:33:28.597 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:33:28.889 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:33:28.889 INFO:teuthology.orchestra.run.trial038.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"2 OSD(s) experiencing slow operations in BlueStore","count":2},"muted":false}},"mutes":[]} 2026-02-24T18:33:31.899 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:33:32.194 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:33:32.194 INFO:teuthology.orchestra.run.trial038.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"2 OSD(s) experiencing slow operations in BlueStore","count":2},"muted":false}},"mutes":[]} 2026-02-24T18:33:35.204 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:33:35.503 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:33:35.503 INFO:teuthology.orchestra.run.trial038.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"2 OSD(s) experiencing slow operations in BlueStore","count":2},"muted":false}},"mutes":[]} 2026-02-24T18:33:38.515 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:33:38.815 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:33:38.815 INFO:teuthology.orchestra.run.trial038.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"2 OSD(s) experiencing slow operations in BlueStore","count":2},"muted":false}},"mutes":[]} 2026-02-24T18:33:41.827 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:33:42.120 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:33:42.120 INFO:teuthology.orchestra.run.trial038.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"2 OSD(s) experiencing slow operations in BlueStore","count":2},"muted":false}},"mutes":[]} 2026-02-24T18:33:45.130 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:33:45.424 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:33:45.424 INFO:teuthology.orchestra.run.trial038.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"2 OSD(s) experiencing slow operations in BlueStore","count":2},"muted":false}},"mutes":[]} 2026-02-24T18:33:48.434 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:33:48.728 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:33:48.729 INFO:teuthology.orchestra.run.trial038.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"2 OSD(s) experiencing slow operations in BlueStore","count":2},"muted":false}},"mutes":[]} 2026-02-24T18:33:51.739 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:33:52.037 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:33:52.038 INFO:teuthology.orchestra.run.trial038.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"2 OSD(s) experiencing slow operations in BlueStore","count":2},"muted":false}},"mutes":[]} 2026-02-24T18:33:55.050 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:33:55.345 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:33:55.346 INFO:teuthology.orchestra.run.trial038.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"2 OSD(s) experiencing slow operations in BlueStore","count":2},"muted":false}},"mutes":[]} 2026-02-24T18:33:58.358 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:33:58.657 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:33:58.657 INFO:teuthology.orchestra.run.trial038.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"2 OSD(s) experiencing slow operations in BlueStore","count":2},"muted":false}},"mutes":[]} 2026-02-24T18:34:01.667 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:34:01.961 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:34:01.962 INFO:teuthology.orchestra.run.trial038.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"2 OSD(s) experiencing slow operations in BlueStore","count":2},"muted":false}},"mutes":[]} 2026-02-24T18:34:04.973 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:34:05.271 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:34:05.271 INFO:teuthology.orchestra.run.trial038.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"2 OSD(s) experiencing slow operations in BlueStore","count":2},"muted":false}},"mutes":[]} 2026-02-24T18:34:08.281 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:34:08.573 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:34:08.573 INFO:teuthology.orchestra.run.trial038.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"2 OSD(s) experiencing slow operations in BlueStore","count":2},"muted":false}},"mutes":[]} 2026-02-24T18:34:11.584 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:34:11.881 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:34:11.881 INFO:teuthology.orchestra.run.trial038.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"2 OSD(s) experiencing slow operations in BlueStore","count":2},"muted":false}},"mutes":[]} 2026-02-24T18:34:14.892 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:34:15.184 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:34:15.184 INFO:teuthology.orchestra.run.trial038.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"2 OSD(s) experiencing slow operations in BlueStore","count":2},"muted":false}},"mutes":[]} 2026-02-24T18:34:18.194 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:34:18.493 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:34:18.493 INFO:teuthology.orchestra.run.trial038.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"2 OSD(s) experiencing slow operations in BlueStore","count":2},"muted":false}},"mutes":[]} 2026-02-24T18:34:21.504 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:34:21.800 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:34:21.801 INFO:teuthology.orchestra.run.trial038.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"2 OSD(s) experiencing slow operations in BlueStore","count":2},"muted":false}},"mutes":[]} 2026-02-24T18:34:24.811 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:34:25.106 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:34:25.107 INFO:teuthology.orchestra.run.trial038.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"2 OSD(s) experiencing slow operations in BlueStore","count":2},"muted":false}},"mutes":[]} 2026-02-24T18:34:28.116 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:34:28.413 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:34:28.413 INFO:teuthology.orchestra.run.trial038.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"2 OSD(s) experiencing slow operations in BlueStore","count":2},"muted":false}},"mutes":[]} 2026-02-24T18:34:31.424 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:34:31.720 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:34:31.721 INFO:teuthology.orchestra.run.trial038.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"2 OSD(s) experiencing slow operations in BlueStore","count":2},"muted":false}},"mutes":[]} 2026-02-24T18:34:34.732 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:34:35.028 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:34:35.028 INFO:teuthology.orchestra.run.trial038.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"2 OSD(s) experiencing slow operations in BlueStore","count":2},"muted":false}},"mutes":[]} 2026-02-24T18:34:38.038 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:34:38.332 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:34:38.332 INFO:teuthology.orchestra.run.trial038.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"2 OSD(s) experiencing slow operations in BlueStore","count":2},"muted":false}},"mutes":[]} 2026-02-24T18:34:41.342 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:34:41.639 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:34:41.640 INFO:teuthology.orchestra.run.trial038.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"2 OSD(s) experiencing slow operations in BlueStore","count":2},"muted":false}},"mutes":[]} 2026-02-24T18:34:44.650 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:34:44.951 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:34:44.952 INFO:teuthology.orchestra.run.trial038.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"2 OSD(s) experiencing slow operations in BlueStore","count":2},"muted":false}},"mutes":[]} 2026-02-24T18:34:47.962 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:34:48.252 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:34:48.252 INFO:teuthology.orchestra.run.trial038.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"2 OSD(s) experiencing slow operations in BlueStore","count":2},"muted":false}},"mutes":[]} 2026-02-24T18:34:51.262 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:34:51.571 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:34:51.571 INFO:teuthology.orchestra.run.trial038.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"2 OSD(s) experiencing slow operations in BlueStore","count":2},"muted":false}},"mutes":[]} 2026-02-24T18:34:54.581 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:34:54.879 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:34:54.879 INFO:teuthology.orchestra.run.trial038.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"2 OSD(s) experiencing slow operations in BlueStore","count":2},"muted":false}},"mutes":[]} 2026-02-24T18:34:57.890 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:34:58.190 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:34:58.190 INFO:teuthology.orchestra.run.trial038.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"2 OSD(s) experiencing slow operations in BlueStore","count":2},"muted":false}},"mutes":[]} 2026-02-24T18:35:01.201 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:35:01.506 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:35:01.507 INFO:teuthology.orchestra.run.trial038.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"2 OSD(s) experiencing slow operations in BlueStore","count":2},"muted":false}},"mutes":[]} 2026-02-24T18:35:04.517 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:35:04.814 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:35:04.815 INFO:teuthology.orchestra.run.trial038.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"2 OSD(s) experiencing slow operations in BlueStore","count":2},"muted":false}},"mutes":[]} 2026-02-24T18:35:07.826 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:35:08.119 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:35:08.119 INFO:teuthology.orchestra.run.trial038.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"2 OSD(s) experiencing slow operations in BlueStore","count":2},"muted":false}},"mutes":[]} 2026-02-24T18:35:11.130 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:35:11.429 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:35:11.429 INFO:teuthology.orchestra.run.trial038.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"2 OSD(s) experiencing slow operations in BlueStore","count":2},"muted":false}},"mutes":[]} 2026-02-24T18:35:14.439 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:35:14.735 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:35:14.735 INFO:teuthology.orchestra.run.trial038.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"2 OSD(s) experiencing slow operations in BlueStore","count":2},"muted":false}},"mutes":[]} 2026-02-24T18:35:17.745 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:35:18.038 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:35:18.038 INFO:teuthology.orchestra.run.trial038.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"2 OSD(s) experiencing slow operations in BlueStore","count":2},"muted":false}},"mutes":[]} 2026-02-24T18:35:21.048 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:35:21.343 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:35:21.343 INFO:teuthology.orchestra.run.trial038.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"2 OSD(s) experiencing slow operations in BlueStore","count":2},"muted":false}},"mutes":[]} 2026-02-24T18:35:24.353 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:35:24.653 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:35:24.653 INFO:teuthology.orchestra.run.trial038.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"2 OSD(s) experiencing slow operations in BlueStore","count":2},"muted":false}},"mutes":[]} 2026-02-24T18:35:27.663 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:35:27.959 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:35:27.959 INFO:teuthology.orchestra.run.trial038.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"2 OSD(s) experiencing slow operations in BlueStore","count":2},"muted":false}},"mutes":[]} 2026-02-24T18:35:30.970 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:35:31.266 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:35:31.266 INFO:teuthology.orchestra.run.trial038.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"2 OSD(s) experiencing slow operations in BlueStore","count":2},"muted":false}},"mutes":[]} 2026-02-24T18:35:34.276 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:35:34.580 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:35:34.580 INFO:teuthology.orchestra.run.trial038.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"2 OSD(s) experiencing slow operations in BlueStore","count":2},"muted":false}},"mutes":[]} 2026-02-24T18:35:37.591 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:35:37.890 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:35:37.891 INFO:teuthology.orchestra.run.trial038.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"2 OSD(s) experiencing slow operations in BlueStore","count":2},"muted":false}},"mutes":[]} 2026-02-24T18:35:40.902 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:35:41.203 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:35:41.204 INFO:teuthology.orchestra.run.trial038.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"2 OSD(s) experiencing slow operations in BlueStore","count":2},"muted":false}},"mutes":[]} 2026-02-24T18:35:44.214 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:35:44.512 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:35:44.512 INFO:teuthology.orchestra.run.trial038.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"2 OSD(s) experiencing slow operations in BlueStore","count":2},"muted":false}},"mutes":[]} 2026-02-24T18:35:47.522 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:35:47.822 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:35:47.822 INFO:teuthology.orchestra.run.trial038.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"2 OSD(s) experiencing slow operations in BlueStore","count":2},"muted":false}},"mutes":[]} 2026-02-24T18:35:50.831 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:35:51.130 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:35:51.131 INFO:teuthology.orchestra.run.trial038.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"2 OSD(s) experiencing slow operations in BlueStore","count":2},"muted":false}},"mutes":[]} 2026-02-24T18:35:54.141 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:35:54.436 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:35:54.436 INFO:teuthology.orchestra.run.trial038.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"2 OSD(s) experiencing slow operations in BlueStore","count":2},"muted":false}},"mutes":[]} 2026-02-24T18:35:57.446 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:35:57.748 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:35:57.749 INFO:teuthology.orchestra.run.trial038.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"2 OSD(s) experiencing slow operations in BlueStore","count":2},"muted":false}},"mutes":[]} 2026-02-24T18:36:00.760 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:36:01.051 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:36:01.052 INFO:teuthology.orchestra.run.trial038.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"2 OSD(s) experiencing slow operations in BlueStore","count":2},"muted":false}},"mutes":[]} 2026-02-24T18:36:04.062 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:36:04.355 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:36:04.355 INFO:teuthology.orchestra.run.trial038.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"2 OSD(s) experiencing slow operations in BlueStore","count":2},"muted":false}},"mutes":[]} 2026-02-24T18:36:07.365 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:36:07.656 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:36:07.656 INFO:teuthology.orchestra.run.trial038.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"2 OSD(s) experiencing slow operations in BlueStore","count":2},"muted":false}},"mutes":[]} 2026-02-24T18:36:10.666 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:36:10.963 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:36:10.963 INFO:teuthology.orchestra.run.trial038.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"2 OSD(s) experiencing slow operations in BlueStore","count":2},"muted":false}},"mutes":[]} 2026-02-24T18:36:13.974 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:36:14.268 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:36:14.268 INFO:teuthology.orchestra.run.trial038.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"2 OSD(s) experiencing slow operations in BlueStore","count":2},"muted":false}},"mutes":[]} 2026-02-24T18:36:17.279 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:36:17.585 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:36:17.585 INFO:teuthology.orchestra.run.trial038.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"2 OSD(s) experiencing slow operations in BlueStore","count":2},"muted":false}},"mutes":[]} 2026-02-24T18:36:20.596 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:36:20.892 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:36:20.892 INFO:teuthology.orchestra.run.trial038.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"2 OSD(s) experiencing slow operations in BlueStore","count":2},"muted":false}},"mutes":[]} 2026-02-24T18:36:23.903 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:36:24.203 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:36:24.203 INFO:teuthology.orchestra.run.trial038.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"2 OSD(s) experiencing slow operations in BlueStore","count":2},"muted":false}},"mutes":[]} 2026-02-24T18:36:27.214 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:36:27.514 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:36:27.514 INFO:teuthology.orchestra.run.trial038.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"2 OSD(s) experiencing slow operations in BlueStore","count":2},"muted":false}},"mutes":[]} 2026-02-24T18:36:30.525 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:36:30.822 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:36:30.823 INFO:teuthology.orchestra.run.trial038.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"2 OSD(s) experiencing slow operations in BlueStore","count":2},"muted":false}},"mutes":[]} 2026-02-24T18:36:33.834 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:36:34.125 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:36:34.126 INFO:teuthology.orchestra.run.trial038.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"2 OSD(s) experiencing slow operations in BlueStore","count":2},"muted":false}},"mutes":[]} 2026-02-24T18:36:37.136 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:36:37.438 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:36:37.439 INFO:teuthology.orchestra.run.trial038.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"2 OSD(s) experiencing slow operations in BlueStore","count":2},"muted":false}},"mutes":[]} 2026-02-24T18:36:40.449 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:36:40.744 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:36:40.744 INFO:teuthology.orchestra.run.trial038.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"2 OSD(s) experiencing slow operations in BlueStore","count":2},"muted":false}},"mutes":[]} 2026-02-24T18:36:43.755 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:36:44.051 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:36:44.051 INFO:teuthology.orchestra.run.trial038.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"2 OSD(s) experiencing slow operations in BlueStore","count":2},"muted":false}},"mutes":[]} 2026-02-24T18:36:47.062 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:36:47.354 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:36:47.355 INFO:teuthology.orchestra.run.trial038.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"2 OSD(s) experiencing slow operations in BlueStore","count":2},"muted":false}},"mutes":[]} 2026-02-24T18:36:50.364 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:36:50.663 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:36:50.663 INFO:teuthology.orchestra.run.trial038.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"2 OSD(s) experiencing slow operations in BlueStore","count":2},"muted":false}},"mutes":[]} 2026-02-24T18:36:53.674 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:36:53.970 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:36:53.970 INFO:teuthology.orchestra.run.trial038.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"2 OSD(s) experiencing slow operations in BlueStore","count":2},"muted":false}},"mutes":[]} 2026-02-24T18:36:56.980 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:36:57.278 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:36:57.278 INFO:teuthology.orchestra.run.trial038.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"2 OSD(s) experiencing slow operations in BlueStore","count":2},"muted":false}},"mutes":[]} 2026-02-24T18:37:00.288 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:37:00.581 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:37:00.581 INFO:teuthology.orchestra.run.trial038.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"2 OSD(s) experiencing slow operations in BlueStore","count":2},"muted":false}},"mutes":[]} 2026-02-24T18:37:03.591 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:37:03.889 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:37:03.889 INFO:teuthology.orchestra.run.trial038.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"2 OSD(s) experiencing slow operations in BlueStore","count":2},"muted":false}},"mutes":[]} 2026-02-24T18:37:06.900 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-24T18:37:07.193 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:37:07.194 INFO:teuthology.orchestra.run.trial038.stdout:{"status":"HEALTH_WARN","checks":{"BLUESTORE_SLOW_OP_ALERT":{"severity":"HEALTH_WARN","summary":{"message":"2 OSD(s) experiencing slow operations in BlueStore","count":2},"muted":false}},"mutes":[]} 2026-02-24T18:37:07.202 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2026-02-24T18:37:07.202 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-24T18:37:07.477 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:37:07.478 INFO:teuthology.orchestra.run.trial038.stderr:dumped all 2026-02-24T18:37:07.486 INFO:teuthology.orchestra.run.trial038.stdout:{"pg_ready":true,"pg_map":{"version":200,"stamp":"2026-02-24T18:37:05.533152+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459299,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":46,"num_read_kb":37,"num_write":59,"num_write_kb":586,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":35,"ondisk_log_size":35,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":314572800,"kb_used":82188,"kb_used_data":1296,"kb_used_omap":4,"kb_used_meta":80827,"kb_avail":314490612,"statfs":{"total":322122547200,"available":322038386688,"internally_reserved":0,"allocated":1327104,"data_stored":1034677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4779,"internal_metadata":82767189},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 0 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 1 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001829"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":23,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-24T18:34:10.299362+0000","last_change":"2026-02-24T18:32:01.014264+0000","last_active":"2026-02-24T18:34:10.299362+0000","last_peered":"2026-02-24T18:34:10.299362+0000","last_clean":"2026-02-24T18:34:10.299362+0000","last_became_active":"2026-02-24T18:31:58.659266+0000","last_became_peered":"2026-02-24T18:31:58.659266+0000","last_unstale":"2026-02-24T18:34:10.299362+0000","last_undegraded":"2026-02-24T18:34:10.299362+0000","last_fullsized":"2026-02-24T18:34:10.299362+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T18:31:57.356642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T18:31:57.356642+0000","last_clean_scrub_stamp":"2026-02-24T18:31:57.356642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:39:42.788531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.058966485999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":23,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-24T18:34:10.299199+0000","last_change":"2026-02-24T18:32:01.018496+0000","last_active":"2026-02-24T18:34:10.299199+0000","last_peered":"2026-02-24T18:34:10.299199+0000","last_clean":"2026-02-24T18:34:10.299199+0000","last_became_active":"2026-02-24T18:31:58.566235+0000","last_became_peered":"2026-02-24T18:31:58.566235+0000","last_unstale":"2026-02-24T18:34:10.299199+0000","last_undegraded":"2026-02-24T18:34:10.299199+0000","last_fullsized":"2026-02-24T18:34:10.299199+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T18:31:57.356642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T18:31:57.356642+0000","last_clean_scrub_stamp":"2026-02-24T18:31:57.356642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:58:35.800025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.062211901999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":23,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-24T18:34:05.297257+0000","last_change":"2026-02-24T18:32:01.011034+0000","last_active":"2026-02-24T18:34:05.297257+0000","last_peered":"2026-02-24T18:34:05.297257+0000","last_clean":"2026-02-24T18:34:05.297257+0000","last_became_active":"2026-02-24T18:31:58.598935+0000","last_became_peered":"2026-02-24T18:31:58.598935+0000","last_unstale":"2026-02-24T18:34:05.297257+0000","last_undegraded":"2026-02-24T18:34:05.297257+0000","last_fullsized":"2026-02-24T18:34:05.297257+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T18:31:57.356642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T18:31:57.356642+0000","last_clean_scrub_stamp":"2026-02-24T18:31:57.356642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:09:02.143983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.057800998999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-24T18:34:05.297132+0000","last_change":"2026-02-24T18:32:01.005474+0000","last_active":"2026-02-24T18:34:05.297132+0000","last_peered":"2026-02-24T18:34:05.297132+0000","last_clean":"2026-02-24T18:34:05.297132+0000","last_became_active":"2026-02-24T18:31:58.569720+0000","last_became_peered":"2026-02-24T18:31:58.569720+0000","last_unstale":"2026-02-24T18:34:05.297132+0000","last_undegraded":"2026-02-24T18:34:05.297132+0000","last_fullsized":"2026-02-24T18:34:05.297132+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T18:31:57.356642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T18:31:57.356642+0000","last_clean_scrub_stamp":"2026-02-24T18:31:57.356642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:43:48.192030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.05352788,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"14'2","reported_seq":25,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-24T18:34:05.464306+0000","last_change":"2026-02-24T18:32:00.980531+0000","last_active":"2026-02-24T18:34:05.464306+0000","last_peered":"2026-02-24T18:34:05.464306+0000","last_clean":"2026-02-24T18:34:05.464306+0000","last_became_active":"2026-02-24T18:31:58.581467+0000","last_became_peered":"2026-02-24T18:31:58.581467+0000","last_unstale":"2026-02-24T18:34:05.464306+0000","last_undegraded":"2026-02-24T18:34:05.464306+0000","last_fullsized":"2026-02-24T18:34:05.464306+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T18:31:57.356642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T18:31:57.356642+0000","last_clean_scrub_stamp":"2026-02-24T18:31:57.356642+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:11:16.111193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.039524833000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":23,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-24T18:34:05.282296+0000","last_change":"2026-02-24T18:32:01.094116+0000","last_active":"2026-02-24T18:34:05.282296+0000","last_peered":"2026-02-24T18:34:05.282296+0000","last_clean":"2026-02-24T18:34:05.282296+0000","last_became_active":"2026-02-24T18:31:58.752939+0000","last_became_peered":"2026-02-24T18:31:58.752939+0000","last_unstale":"2026-02-24T18:34:05.282296+0000","last_undegraded":"2026-02-24T18:34:05.282296+0000","last_fullsized":"2026-02-24T18:34:05.282296+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T18:31:57.356642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T18:31:57.356642+0000","last_clean_scrub_stamp":"2026-02-24T18:31:57.356642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:47:09.928859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.048502458999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":23,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-24T18:34:00.280552+0000","last_change":"2026-02-24T18:32:01.085916+0000","last_active":"2026-02-24T18:34:00.280552+0000","last_peered":"2026-02-24T18:34:00.280552+0000","last_clean":"2026-02-24T18:34:00.280552+0000","last_became_active":"2026-02-24T18:31:58.746390+0000","last_became_peered":"2026-02-24T18:31:58.746390+0000","last_unstale":"2026-02-24T18:34:00.280552+0000","last_undegraded":"2026-02-24T18:34:00.280552+0000","last_fullsized":"2026-02-24T18:34:00.280552+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T18:31:57.356642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T18:31:57.356642+0000","last_clean_scrub_stamp":"2026-02-24T18:31:57.356642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:09:03.999608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.041638965999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"12'1","reported_seq":24,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-24T18:34:05.296947+0000","last_change":"2026-02-24T18:32:00.998783+0000","last_active":"2026-02-24T18:34:05.296947+0000","last_peered":"2026-02-24T18:34:05.296947+0000","last_clean":"2026-02-24T18:34:05.296947+0000","last_became_active":"2026-02-24T18:31:58.627943+0000","last_became_peered":"2026-02-24T18:31:58.627943+0000","last_unstale":"2026-02-24T18:34:05.296947+0000","last_undegraded":"2026-02-24T18:34:05.296947+0000","last_fullsized":"2026-02-24T18:34:05.296947+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T18:31:57.356642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T18:31:57.356642+0000","last_clean_scrub_stamp":"2026-02-24T18:31:57.356642+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:30:35.582393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.050969995999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"14'32","reported_seq":64,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-24T18:34:00.294846+0000","last_change":"2026-02-24T18:31:57.756734+0000","last_active":"2026-02-24T18:34:00.294846+0000","last_peered":"2026-02-24T18:34:00.294846+0000","last_clean":"2026-02-24T18:34:00.294846+0000","last_became_active":"2026-02-24T18:31:57.737892+0000","last_became_peered":"2026-02-24T18:31:57.737892+0000","last_unstale":"2026-02-24T18:34:00.294846+0000","last_undegraded":"2026-02-24T18:34:00.294846+0000","last_fullsized":"2026-02-24T18:34:00.294846+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T18:31:56.258459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T18:31:56.258459+0000","last_clean_scrub_stamp":"2026-02-24T18:31:56.258459+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:20:14.513250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":46,"num_read_kb":37,"num_write":57,"num_write_kb":584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":46,"num_read_kb":37,"num_write":57,"num_write_kb":584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":925696,"data_stored":918560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":32,"ondisk_log_size":32,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":1,"up_from":8,"seq":34359738433,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27548,"kb_used_data":584,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":104830052,"statfs":{"total":107374182400,"available":107345973248,"internally_reserved":0,"allocated":598016,"data_stored":497992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 1 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":0,"up_from":9,"seq":38654705727,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27548,"kb_used_data":584,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":104830052,"statfs":{"total":107374182400,"available":107345973248,"internally_reserved":0,"allocated":598016,"data_stored":497992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27589065},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 0 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":2,"up_from":8,"seq":34359738434,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27092,"kb_used_data":128,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":104830508,"statfs":{"total":107374182400,"available":107346440192,"internally_reserved":0,"allocated":131072,"data_stored":38693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":462848,"data_stored":459280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":462848,"data_stored":459280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-24T18:37:07.488 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-24T18:37:07.762 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:37:07.763 INFO:teuthology.orchestra.run.trial038.stderr:dumped all 2026-02-24T18:37:07.771 INFO:teuthology.orchestra.run.trial038.stdout:{"pg_ready":true,"pg_map":{"version":201,"stamp":"2026-02-24T18:37:07.533521+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459299,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":46,"num_read_kb":37,"num_write":59,"num_write_kb":586,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":35,"ondisk_log_size":35,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":314572800,"kb_used":82188,"kb_used_data":1296,"kb_used_omap":4,"kb_used_meta":80827,"kb_avail":314490612,"statfs":{"total":322122547200,"available":322038386688,"internally_reserved":0,"allocated":1327104,"data_stored":1034677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4779,"internal_metadata":82767189},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 0 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 1 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001897"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":23,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-24T18:34:10.299362+0000","last_change":"2026-02-24T18:32:01.014264+0000","last_active":"2026-02-24T18:34:10.299362+0000","last_peered":"2026-02-24T18:34:10.299362+0000","last_clean":"2026-02-24T18:34:10.299362+0000","last_became_active":"2026-02-24T18:31:58.659266+0000","last_became_peered":"2026-02-24T18:31:58.659266+0000","last_unstale":"2026-02-24T18:34:10.299362+0000","last_undegraded":"2026-02-24T18:34:10.299362+0000","last_fullsized":"2026-02-24T18:34:10.299362+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T18:31:57.356642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T18:31:57.356642+0000","last_clean_scrub_stamp":"2026-02-24T18:31:57.356642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:39:42.788531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.058966485999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":23,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-24T18:34:10.299199+0000","last_change":"2026-02-24T18:32:01.018496+0000","last_active":"2026-02-24T18:34:10.299199+0000","last_peered":"2026-02-24T18:34:10.299199+0000","last_clean":"2026-02-24T18:34:10.299199+0000","last_became_active":"2026-02-24T18:31:58.566235+0000","last_became_peered":"2026-02-24T18:31:58.566235+0000","last_unstale":"2026-02-24T18:34:10.299199+0000","last_undegraded":"2026-02-24T18:34:10.299199+0000","last_fullsized":"2026-02-24T18:34:10.299199+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T18:31:57.356642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T18:31:57.356642+0000","last_clean_scrub_stamp":"2026-02-24T18:31:57.356642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:58:35.800025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.062211901999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":23,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-24T18:34:05.297257+0000","last_change":"2026-02-24T18:32:01.011034+0000","last_active":"2026-02-24T18:34:05.297257+0000","last_peered":"2026-02-24T18:34:05.297257+0000","last_clean":"2026-02-24T18:34:05.297257+0000","last_became_active":"2026-02-24T18:31:58.598935+0000","last_became_peered":"2026-02-24T18:31:58.598935+0000","last_unstale":"2026-02-24T18:34:05.297257+0000","last_undegraded":"2026-02-24T18:34:05.297257+0000","last_fullsized":"2026-02-24T18:34:05.297257+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T18:31:57.356642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T18:31:57.356642+0000","last_clean_scrub_stamp":"2026-02-24T18:31:57.356642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:09:02.143983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.057800998999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-24T18:34:05.297132+0000","last_change":"2026-02-24T18:32:01.005474+0000","last_active":"2026-02-24T18:34:05.297132+0000","last_peered":"2026-02-24T18:34:05.297132+0000","last_clean":"2026-02-24T18:34:05.297132+0000","last_became_active":"2026-02-24T18:31:58.569720+0000","last_became_peered":"2026-02-24T18:31:58.569720+0000","last_unstale":"2026-02-24T18:34:05.297132+0000","last_undegraded":"2026-02-24T18:34:05.297132+0000","last_fullsized":"2026-02-24T18:34:05.297132+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T18:31:57.356642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T18:31:57.356642+0000","last_clean_scrub_stamp":"2026-02-24T18:31:57.356642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:43:48.192030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.05352788,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"14'2","reported_seq":25,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-24T18:34:05.464306+0000","last_change":"2026-02-24T18:32:00.980531+0000","last_active":"2026-02-24T18:34:05.464306+0000","last_peered":"2026-02-24T18:34:05.464306+0000","last_clean":"2026-02-24T18:34:05.464306+0000","last_became_active":"2026-02-24T18:31:58.581467+0000","last_became_peered":"2026-02-24T18:31:58.581467+0000","last_unstale":"2026-02-24T18:34:05.464306+0000","last_undegraded":"2026-02-24T18:34:05.464306+0000","last_fullsized":"2026-02-24T18:34:05.464306+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T18:31:57.356642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T18:31:57.356642+0000","last_clean_scrub_stamp":"2026-02-24T18:31:57.356642+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:11:16.111193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.039524833000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":23,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-24T18:34:05.282296+0000","last_change":"2026-02-24T18:32:01.094116+0000","last_active":"2026-02-24T18:34:05.282296+0000","last_peered":"2026-02-24T18:34:05.282296+0000","last_clean":"2026-02-24T18:34:05.282296+0000","last_became_active":"2026-02-24T18:31:58.752939+0000","last_became_peered":"2026-02-24T18:31:58.752939+0000","last_unstale":"2026-02-24T18:34:05.282296+0000","last_undegraded":"2026-02-24T18:34:05.282296+0000","last_fullsized":"2026-02-24T18:34:05.282296+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T18:31:57.356642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T18:31:57.356642+0000","last_clean_scrub_stamp":"2026-02-24T18:31:57.356642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:47:09.928859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.048502458999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":23,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-24T18:34:00.280552+0000","last_change":"2026-02-24T18:32:01.085916+0000","last_active":"2026-02-24T18:34:00.280552+0000","last_peered":"2026-02-24T18:34:00.280552+0000","last_clean":"2026-02-24T18:34:00.280552+0000","last_became_active":"2026-02-24T18:31:58.746390+0000","last_became_peered":"2026-02-24T18:31:58.746390+0000","last_unstale":"2026-02-24T18:34:00.280552+0000","last_undegraded":"2026-02-24T18:34:00.280552+0000","last_fullsized":"2026-02-24T18:34:00.280552+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T18:31:57.356642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T18:31:57.356642+0000","last_clean_scrub_stamp":"2026-02-24T18:31:57.356642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:09:03.999608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.041638965999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"12'1","reported_seq":24,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-24T18:34:05.296947+0000","last_change":"2026-02-24T18:32:00.998783+0000","last_active":"2026-02-24T18:34:05.296947+0000","last_peered":"2026-02-24T18:34:05.296947+0000","last_clean":"2026-02-24T18:34:05.296947+0000","last_became_active":"2026-02-24T18:31:58.627943+0000","last_became_peered":"2026-02-24T18:31:58.627943+0000","last_unstale":"2026-02-24T18:34:05.296947+0000","last_undegraded":"2026-02-24T18:34:05.296947+0000","last_fullsized":"2026-02-24T18:34:05.296947+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T18:31:57.356642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T18:31:57.356642+0000","last_clean_scrub_stamp":"2026-02-24T18:31:57.356642+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:30:35.582393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.050969995999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"14'32","reported_seq":64,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-24T18:34:00.294846+0000","last_change":"2026-02-24T18:31:57.756734+0000","last_active":"2026-02-24T18:34:00.294846+0000","last_peered":"2026-02-24T18:34:00.294846+0000","last_clean":"2026-02-24T18:34:00.294846+0000","last_became_active":"2026-02-24T18:31:57.737892+0000","last_became_peered":"2026-02-24T18:31:57.737892+0000","last_unstale":"2026-02-24T18:34:00.294846+0000","last_undegraded":"2026-02-24T18:34:00.294846+0000","last_fullsized":"2026-02-24T18:34:00.294846+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T18:31:56.258459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T18:31:56.258459+0000","last_clean_scrub_stamp":"2026-02-24T18:31:56.258459+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:20:14.513250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":46,"num_read_kb":37,"num_write":57,"num_write_kb":584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":46,"num_read_kb":37,"num_write":57,"num_write_kb":584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":925696,"data_stored":918560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":32,"ondisk_log_size":32,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":1,"up_from":8,"seq":34359738433,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27548,"kb_used_data":584,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":104830052,"statfs":{"total":107374182400,"available":107345973248,"internally_reserved":0,"allocated":598016,"data_stored":497992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 1 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":0,"up_from":9,"seq":38654705728,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27548,"kb_used_data":584,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":104830052,"statfs":{"total":107374182400,"available":107345973248,"internally_reserved":0,"allocated":598016,"data_stored":497992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27589065},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 0 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":2,"up_from":8,"seq":34359738434,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27092,"kb_used_data":128,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":104830508,"statfs":{"total":107374182400,"available":107346440192,"internally_reserved":0,"allocated":131072,"data_stored":38693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":462848,"data_stored":459280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":462848,"data_stored":459280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-24T18:37:07.772 INFO:tasks.ceph.ceph_manager.ceph:clean! 2026-02-24T18:37:07.773 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-24T18:37:08.052 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:37:08.053 INFO:teuthology.orchestra.run.trial038.stderr:dumped all 2026-02-24T18:37:08.062 INFO:teuthology.orchestra.run.trial038.stdout:{"pg_ready":true,"pg_map":{"version":201,"stamp":"2026-02-24T18:37:07.533521+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459299,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":46,"num_read_kb":37,"num_write":59,"num_write_kb":586,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":35,"ondisk_log_size":35,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":314572800,"kb_used":82188,"kb_used_data":1296,"kb_used_omap":4,"kb_used_meta":80827,"kb_avail":314490612,"statfs":{"total":322122547200,"available":322038386688,"internally_reserved":0,"allocated":1327104,"data_stored":1034677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4779,"internal_metadata":82767189},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 0 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 1 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001897"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":23,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-24T18:34:10.299362+0000","last_change":"2026-02-24T18:32:01.014264+0000","last_active":"2026-02-24T18:34:10.299362+0000","last_peered":"2026-02-24T18:34:10.299362+0000","last_clean":"2026-02-24T18:34:10.299362+0000","last_became_active":"2026-02-24T18:31:58.659266+0000","last_became_peered":"2026-02-24T18:31:58.659266+0000","last_unstale":"2026-02-24T18:34:10.299362+0000","last_undegraded":"2026-02-24T18:34:10.299362+0000","last_fullsized":"2026-02-24T18:34:10.299362+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T18:31:57.356642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T18:31:57.356642+0000","last_clean_scrub_stamp":"2026-02-24T18:31:57.356642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:39:42.788531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.058966485999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":23,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-24T18:34:10.299199+0000","last_change":"2026-02-24T18:32:01.018496+0000","last_active":"2026-02-24T18:34:10.299199+0000","last_peered":"2026-02-24T18:34:10.299199+0000","last_clean":"2026-02-24T18:34:10.299199+0000","last_became_active":"2026-02-24T18:31:58.566235+0000","last_became_peered":"2026-02-24T18:31:58.566235+0000","last_unstale":"2026-02-24T18:34:10.299199+0000","last_undegraded":"2026-02-24T18:34:10.299199+0000","last_fullsized":"2026-02-24T18:34:10.299199+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T18:31:57.356642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T18:31:57.356642+0000","last_clean_scrub_stamp":"2026-02-24T18:31:57.356642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:58:35.800025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.062211901999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":23,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-24T18:34:05.297257+0000","last_change":"2026-02-24T18:32:01.011034+0000","last_active":"2026-02-24T18:34:05.297257+0000","last_peered":"2026-02-24T18:34:05.297257+0000","last_clean":"2026-02-24T18:34:05.297257+0000","last_became_active":"2026-02-24T18:31:58.598935+0000","last_became_peered":"2026-02-24T18:31:58.598935+0000","last_unstale":"2026-02-24T18:34:05.297257+0000","last_undegraded":"2026-02-24T18:34:05.297257+0000","last_fullsized":"2026-02-24T18:34:05.297257+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T18:31:57.356642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T18:31:57.356642+0000","last_clean_scrub_stamp":"2026-02-24T18:31:57.356642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:09:02.143983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.057800998999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-24T18:34:05.297132+0000","last_change":"2026-02-24T18:32:01.005474+0000","last_active":"2026-02-24T18:34:05.297132+0000","last_peered":"2026-02-24T18:34:05.297132+0000","last_clean":"2026-02-24T18:34:05.297132+0000","last_became_active":"2026-02-24T18:31:58.569720+0000","last_became_peered":"2026-02-24T18:31:58.569720+0000","last_unstale":"2026-02-24T18:34:05.297132+0000","last_undegraded":"2026-02-24T18:34:05.297132+0000","last_fullsized":"2026-02-24T18:34:05.297132+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T18:31:57.356642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T18:31:57.356642+0000","last_clean_scrub_stamp":"2026-02-24T18:31:57.356642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:43:48.192030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.05352788,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"14'2","reported_seq":25,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-24T18:34:05.464306+0000","last_change":"2026-02-24T18:32:00.980531+0000","last_active":"2026-02-24T18:34:05.464306+0000","last_peered":"2026-02-24T18:34:05.464306+0000","last_clean":"2026-02-24T18:34:05.464306+0000","last_became_active":"2026-02-24T18:31:58.581467+0000","last_became_peered":"2026-02-24T18:31:58.581467+0000","last_unstale":"2026-02-24T18:34:05.464306+0000","last_undegraded":"2026-02-24T18:34:05.464306+0000","last_fullsized":"2026-02-24T18:34:05.464306+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T18:31:57.356642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T18:31:57.356642+0000","last_clean_scrub_stamp":"2026-02-24T18:31:57.356642+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:11:16.111193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.039524833000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":23,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-24T18:34:05.282296+0000","last_change":"2026-02-24T18:32:01.094116+0000","last_active":"2026-02-24T18:34:05.282296+0000","last_peered":"2026-02-24T18:34:05.282296+0000","last_clean":"2026-02-24T18:34:05.282296+0000","last_became_active":"2026-02-24T18:31:58.752939+0000","last_became_peered":"2026-02-24T18:31:58.752939+0000","last_unstale":"2026-02-24T18:34:05.282296+0000","last_undegraded":"2026-02-24T18:34:05.282296+0000","last_fullsized":"2026-02-24T18:34:05.282296+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T18:31:57.356642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T18:31:57.356642+0000","last_clean_scrub_stamp":"2026-02-24T18:31:57.356642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:47:09.928859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.048502458999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":23,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-24T18:34:00.280552+0000","last_change":"2026-02-24T18:32:01.085916+0000","last_active":"2026-02-24T18:34:00.280552+0000","last_peered":"2026-02-24T18:34:00.280552+0000","last_clean":"2026-02-24T18:34:00.280552+0000","last_became_active":"2026-02-24T18:31:58.746390+0000","last_became_peered":"2026-02-24T18:31:58.746390+0000","last_unstale":"2026-02-24T18:34:00.280552+0000","last_undegraded":"2026-02-24T18:34:00.280552+0000","last_fullsized":"2026-02-24T18:34:00.280552+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T18:31:57.356642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T18:31:57.356642+0000","last_clean_scrub_stamp":"2026-02-24T18:31:57.356642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:09:03.999608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.041638965999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"12'1","reported_seq":24,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-24T18:34:05.296947+0000","last_change":"2026-02-24T18:32:00.998783+0000","last_active":"2026-02-24T18:34:05.296947+0000","last_peered":"2026-02-24T18:34:05.296947+0000","last_clean":"2026-02-24T18:34:05.296947+0000","last_became_active":"2026-02-24T18:31:58.627943+0000","last_became_peered":"2026-02-24T18:31:58.627943+0000","last_unstale":"2026-02-24T18:34:05.296947+0000","last_undegraded":"2026-02-24T18:34:05.296947+0000","last_fullsized":"2026-02-24T18:34:05.296947+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T18:31:57.356642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T18:31:57.356642+0000","last_clean_scrub_stamp":"2026-02-24T18:31:57.356642+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:30:35.582393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.050969995999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"14'32","reported_seq":64,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-24T18:34:00.294846+0000","last_change":"2026-02-24T18:31:57.756734+0000","last_active":"2026-02-24T18:34:00.294846+0000","last_peered":"2026-02-24T18:34:00.294846+0000","last_clean":"2026-02-24T18:34:00.294846+0000","last_became_active":"2026-02-24T18:31:57.737892+0000","last_became_peered":"2026-02-24T18:31:57.737892+0000","last_unstale":"2026-02-24T18:34:00.294846+0000","last_undegraded":"2026-02-24T18:34:00.294846+0000","last_fullsized":"2026-02-24T18:34:00.294846+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T18:31:56.258459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T18:31:56.258459+0000","last_clean_scrub_stamp":"2026-02-24T18:31:56.258459+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:20:14.513250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":46,"num_read_kb":37,"num_write":57,"num_write_kb":584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":46,"num_read_kb":37,"num_write":57,"num_write_kb":584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":925696,"data_stored":918560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":32,"ondisk_log_size":32,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":1,"up_from":8,"seq":34359738433,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27548,"kb_used_data":584,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":104830052,"statfs":{"total":107374182400,"available":107345973248,"internally_reserved":0,"allocated":598016,"data_stored":497992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 1 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":0,"up_from":9,"seq":38654705728,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27548,"kb_used_data":584,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":104830052,"statfs":{"total":107374182400,"available":107345973248,"internally_reserved":0,"allocated":598016,"data_stored":497992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27589065},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 0 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":2,"up_from":8,"seq":34359738434,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27092,"kb_used_data":128,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":104830508,"statfs":{"total":107374182400,"available":107346440192,"internally_reserved":0,"allocated":131072,"data_stored":38693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":462848,"data_stored":459280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":462848,"data_stored":459280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-24T18:37:08.062 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-24T18:37:08.377 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:37:08.377 INFO:teuthology.orchestra.run.trial038.stdout:{"epoch":15,"fsid":"3487e59d-026b-4a4d-92d5-661e87a7c906","created":"2026-02-24T18:30:28.512730+0000","modified":"2026-02-24T18:32:07.394454+0000","last_up_change":"2026-02-24T18:31:55.194260+0000","last_in_change":"2026-02-24T18:30:30.119568+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-24T18:31:55.621631+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2026-02-24T18:31:57.010626+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":14,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"22d4bbb5-3cf0-4c06-9552-80e01980013c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.38:6818","nonce":3879277939},{"type":"v1","addr":"10.20.193.38:6819","nonce":3879277939}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.38:6820","nonce":3879277939},{"type":"v1","addr":"10.20.193.38:6821","nonce":3879277939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.38:6824","nonce":3879277939},{"type":"v1","addr":"10.20.193.38:6825","nonce":3879277939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.38:6822","nonce":3879277939},{"type":"v1","addr":"10.20.193.38:6823","nonce":3879277939}]},"public_addr":"10.20.193.38:6819/3879277939","cluster_addr":"10.20.193.38:6821/3879277939","heartbeat_back_addr":"10.20.193.38:6825/3879277939","heartbeat_front_addr":"10.20.193.38:6823/3879277939","state":["exists","up"]},{"osd":1,"uuid":"ee9a5258-997f-4465-98f1-976c5dc010c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.38:6810","nonce":1372092871},{"type":"v1","addr":"10.20.193.38:6811","nonce":1372092871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.38:6812","nonce":1372092871},{"type":"v1","addr":"10.20.193.38:6813","nonce":1372092871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.38:6816","nonce":1372092871},{"type":"v1","addr":"10.20.193.38:6817","nonce":1372092871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.38:6814","nonce":1372092871},{"type":"v1","addr":"10.20.193.38:6815","nonce":1372092871}]},"public_addr":"10.20.193.38:6811/1372092871","cluster_addr":"10.20.193.38:6813/1372092871","heartbeat_back_addr":"10.20.193.38:6817/1372092871","heartbeat_front_addr":"10.20.193.38:6815/1372092871","state":["exists","up"]},{"osd":2,"uuid":"aaf542f4-f7fb-4931-b598-36efb11497b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.38:6802","nonce":1853955099},{"type":"v1","addr":"10.20.193.38:6803","nonce":1853955099}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.38:6804","nonce":1853955099},{"type":"v1","addr":"10.20.193.38:6805","nonce":1853955099}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.38:6808","nonce":1853955099},{"type":"v1","addr":"10.20.193.38:6809","nonce":1853955099}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.38:6806","nonce":1853955099},{"type":"v1","addr":"10.20.193.38:6807","nonce":1853955099}]},"public_addr":"10.20.193.38:6803/1853955099","cluster_addr":"10.20.193.38:6805/1853955099","heartbeat_back_addr":"10.20.193.38:6809/1853955099","heartbeat_front_addr":"10.20.193.38:6807/1853955099","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-24T18:31:53.310115+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-24T18:31:52.758406+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-24T18:31:52.648429+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-24T18:37:09.388 INFO:tasks.ceph:Scrubbing osd.0 2026-02-24T18:37:09.388 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2026-02-24T18:37:09.517 INFO:teuthology.orchestra.run.trial038.stdout:{ 2026-02-24T18:37:09.517 INFO:teuthology.orchestra.run.trial038.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_degraded = '' osd_recovery_sleep_degraded_hdd = '' osd_recovery_sleep_degraded_hybrid = '' osd_recovery_sleep_degraded_ssd = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2026-02-24T18:37:09.517 INFO:teuthology.orchestra.run.trial038.stdout:} 2026-02-24T18:37:09.525 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2026-02-24T18:37:09.808 INFO:teuthology.orchestra.run.trial038.stderr:instructed osd(s) 0 to deep-scrub 2026-02-24T18:37:09.817 INFO:tasks.ceph:Scrubbing osd.1 2026-02-24T18:37:09.818 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2026-02-24T18:37:09.945 INFO:teuthology.orchestra.run.trial038.stdout:{ 2026-02-24T18:37:09.945 INFO:teuthology.orchestra.run.trial038.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_degraded = '' osd_recovery_sleep_degraded_hdd = '' osd_recovery_sleep_degraded_hybrid = '' osd_recovery_sleep_degraded_ssd = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2026-02-24T18:37:09.946 INFO:teuthology.orchestra.run.trial038.stdout:} 2026-02-24T18:37:09.955 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2026-02-24T18:37:10.235 INFO:teuthology.orchestra.run.trial038.stderr:instructed osd(s) 1 to deep-scrub 2026-02-24T18:37:10.245 INFO:tasks.ceph:Scrubbing osd.2 2026-02-24T18:37:10.245 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2026-02-24T18:37:10.379 INFO:teuthology.orchestra.run.trial038.stdout:{ 2026-02-24T18:37:10.379 INFO:teuthology.orchestra.run.trial038.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_degraded = '' osd_recovery_sleep_degraded_hdd = '' osd_recovery_sleep_degraded_hybrid = '' osd_recovery_sleep_degraded_ssd = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2026-02-24T18:37:10.379 INFO:teuthology.orchestra.run.trial038.stdout:} 2026-02-24T18:37:10.387 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2026-02-24T18:37:10.664 INFO:teuthology.orchestra.run.trial038.stderr:instructed osd(s) 2 to deep-scrub 2026-02-24T18:37:10.673 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-24T18:37:10.947 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:37:10.947 INFO:teuthology.orchestra.run.trial038.stderr:dumped all 2026-02-24T18:37:10.955 INFO:teuthology.orchestra.run.trial038.stdout:{"pg_ready":true,"pg_map":{"version":202,"stamp":"2026-02-24T18:37:09.533740+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459299,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":46,"num_read_kb":37,"num_write":59,"num_write_kb":586,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":35,"ondisk_log_size":35,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":314572800,"kb_used":82188,"kb_used_data":1296,"kb_used_omap":4,"kb_used_meta":80827,"kb_avail":314490612,"statfs":{"total":322122547200,"available":322038386688,"internally_reserved":0,"allocated":1327104,"data_stored":1034677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4779,"internal_metadata":82767189},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 0 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 1 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001875"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":23,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-24T18:34:10.299362+0000","last_change":"2026-02-24T18:32:01.014264+0000","last_active":"2026-02-24T18:34:10.299362+0000","last_peered":"2026-02-24T18:34:10.299362+0000","last_clean":"2026-02-24T18:34:10.299362+0000","last_became_active":"2026-02-24T18:31:58.659266+0000","last_became_peered":"2026-02-24T18:31:58.659266+0000","last_unstale":"2026-02-24T18:34:10.299362+0000","last_undegraded":"2026-02-24T18:34:10.299362+0000","last_fullsized":"2026-02-24T18:34:10.299362+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T18:31:57.356642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T18:31:57.356642+0000","last_clean_scrub_stamp":"2026-02-24T18:31:57.356642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:39:42.788531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.058966485999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":23,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-24T18:34:10.299199+0000","last_change":"2026-02-24T18:32:01.018496+0000","last_active":"2026-02-24T18:34:10.299199+0000","last_peered":"2026-02-24T18:34:10.299199+0000","last_clean":"2026-02-24T18:34:10.299199+0000","last_became_active":"2026-02-24T18:31:58.566235+0000","last_became_peered":"2026-02-24T18:31:58.566235+0000","last_unstale":"2026-02-24T18:34:10.299199+0000","last_undegraded":"2026-02-24T18:34:10.299199+0000","last_fullsized":"2026-02-24T18:34:10.299199+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T18:31:57.356642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T18:31:57.356642+0000","last_clean_scrub_stamp":"2026-02-24T18:31:57.356642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:58:35.800025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.062211901999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":23,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-24T18:34:05.297257+0000","last_change":"2026-02-24T18:32:01.011034+0000","last_active":"2026-02-24T18:34:05.297257+0000","last_peered":"2026-02-24T18:34:05.297257+0000","last_clean":"2026-02-24T18:34:05.297257+0000","last_became_active":"2026-02-24T18:31:58.598935+0000","last_became_peered":"2026-02-24T18:31:58.598935+0000","last_unstale":"2026-02-24T18:34:05.297257+0000","last_undegraded":"2026-02-24T18:34:05.297257+0000","last_fullsized":"2026-02-24T18:34:05.297257+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T18:31:57.356642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T18:31:57.356642+0000","last_clean_scrub_stamp":"2026-02-24T18:31:57.356642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:09:02.143983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.057800998999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-24T18:34:05.297132+0000","last_change":"2026-02-24T18:32:01.005474+0000","last_active":"2026-02-24T18:34:05.297132+0000","last_peered":"2026-02-24T18:34:05.297132+0000","last_clean":"2026-02-24T18:34:05.297132+0000","last_became_active":"2026-02-24T18:31:58.569720+0000","last_became_peered":"2026-02-24T18:31:58.569720+0000","last_unstale":"2026-02-24T18:34:05.297132+0000","last_undegraded":"2026-02-24T18:34:05.297132+0000","last_fullsized":"2026-02-24T18:34:05.297132+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T18:31:57.356642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T18:31:57.356642+0000","last_clean_scrub_stamp":"2026-02-24T18:31:57.356642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T03:43:48.192030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.05352788,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"14'2","reported_seq":25,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-24T18:34:05.464306+0000","last_change":"2026-02-24T18:32:00.980531+0000","last_active":"2026-02-24T18:34:05.464306+0000","last_peered":"2026-02-24T18:34:05.464306+0000","last_clean":"2026-02-24T18:34:05.464306+0000","last_became_active":"2026-02-24T18:31:58.581467+0000","last_became_peered":"2026-02-24T18:31:58.581467+0000","last_unstale":"2026-02-24T18:34:05.464306+0000","last_undegraded":"2026-02-24T18:34:05.464306+0000","last_fullsized":"2026-02-24T18:34:05.464306+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T18:31:57.356642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T18:31:57.356642+0000","last_clean_scrub_stamp":"2026-02-24T18:31:57.356642+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:11:16.111193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.039524833000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":23,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-24T18:34:05.282296+0000","last_change":"2026-02-24T18:32:01.094116+0000","last_active":"2026-02-24T18:34:05.282296+0000","last_peered":"2026-02-24T18:34:05.282296+0000","last_clean":"2026-02-24T18:34:05.282296+0000","last_became_active":"2026-02-24T18:31:58.752939+0000","last_became_peered":"2026-02-24T18:31:58.752939+0000","last_unstale":"2026-02-24T18:34:05.282296+0000","last_undegraded":"2026-02-24T18:34:05.282296+0000","last_fullsized":"2026-02-24T18:34:05.282296+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T18:31:57.356642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T18:31:57.356642+0000","last_clean_scrub_stamp":"2026-02-24T18:31:57.356642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:47:09.928859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.048502458999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":23,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-24T18:34:00.280552+0000","last_change":"2026-02-24T18:32:01.085916+0000","last_active":"2026-02-24T18:34:00.280552+0000","last_peered":"2026-02-24T18:34:00.280552+0000","last_clean":"2026-02-24T18:34:00.280552+0000","last_became_active":"2026-02-24T18:31:58.746390+0000","last_became_peered":"2026-02-24T18:31:58.746390+0000","last_unstale":"2026-02-24T18:34:00.280552+0000","last_undegraded":"2026-02-24T18:34:00.280552+0000","last_fullsized":"2026-02-24T18:34:00.280552+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T18:31:57.356642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T18:31:57.356642+0000","last_clean_scrub_stamp":"2026-02-24T18:31:57.356642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T05:09:03.999608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.041638965999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"12'1","reported_seq":24,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-24T18:34:05.296947+0000","last_change":"2026-02-24T18:32:00.998783+0000","last_active":"2026-02-24T18:34:05.296947+0000","last_peered":"2026-02-24T18:34:05.296947+0000","last_clean":"2026-02-24T18:34:05.296947+0000","last_became_active":"2026-02-24T18:31:58.627943+0000","last_became_peered":"2026-02-24T18:31:58.627943+0000","last_unstale":"2026-02-24T18:34:05.296947+0000","last_undegraded":"2026-02-24T18:34:05.296947+0000","last_fullsized":"2026-02-24T18:34:05.296947+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T18:31:57.356642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T18:31:57.356642+0000","last_clean_scrub_stamp":"2026-02-24T18:31:57.356642+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T06:30:35.582393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.050969995999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"14'32","reported_seq":64,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-24T18:34:00.294846+0000","last_change":"2026-02-24T18:31:57.756734+0000","last_active":"2026-02-24T18:34:00.294846+0000","last_peered":"2026-02-24T18:34:00.294846+0000","last_clean":"2026-02-24T18:34:00.294846+0000","last_became_active":"2026-02-24T18:31:57.737892+0000","last_became_peered":"2026-02-24T18:31:57.737892+0000","last_unstale":"2026-02-24T18:34:00.294846+0000","last_undegraded":"2026-02-24T18:34:00.294846+0000","last_fullsized":"2026-02-24T18:34:00.294846+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T18:31:56.258459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T18:31:56.258459+0000","last_clean_scrub_stamp":"2026-02-24T18:31:56.258459+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T00:20:14.513250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":46,"num_read_kb":37,"num_write":57,"num_write_kb":584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":46,"num_read_kb":37,"num_write":57,"num_write_kb":584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":925696,"data_stored":918560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":32,"ondisk_log_size":32,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":1,"up_from":8,"seq":34359738433,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27548,"kb_used_data":584,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":104830052,"statfs":{"total":107374182400,"available":107345973248,"internally_reserved":0,"allocated":598016,"data_stored":497992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 1 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":0,"up_from":9,"seq":38654705728,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27548,"kb_used_data":584,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":104830052,"statfs":{"total":107374182400,"available":107345973248,"internally_reserved":0,"allocated":598016,"data_stored":497992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27589065},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 0 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":2,"up_from":8,"seq":34359738434,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27092,"kb_used_data":128,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":104830508,"statfs":{"total":107374182400,"available":107346440192,"internally_reserved":0,"allocated":131072,"data_stored":38693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":462848,"data_stored":459280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":462848,"data_stored":459280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-24T18:37:10.962 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2026-02-24T18:31:57.356642+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=18, tm_min=31, tm_sec=57, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=18, tm_min=37, tm_sec=8, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T18:37:10.962 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2026-02-24T18:31:57.356642+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=18, tm_min=31, tm_sec=57, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=18, tm_min=37, tm_sec=8, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T18:37:10.962 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2026-02-24T18:31:57.356642+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=18, tm_min=31, tm_sec=57, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=18, tm_min=37, tm_sec=8, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T18:37:10.962 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2026-02-24T18:31:57.356642+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=18, tm_min=31, tm_sec=57, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=18, tm_min=37, tm_sec=8, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T18:37:10.963 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2026-02-24T18:31:57.356642+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=18, tm_min=31, tm_sec=57, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=18, tm_min=37, tm_sec=8, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T18:37:10.963 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2026-02-24T18:31:57.356642+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=18, tm_min=31, tm_sec=57, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=18, tm_min=37, tm_sec=8, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T18:37:10.963 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2026-02-24T18:31:57.356642+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=18, tm_min=31, tm_sec=57, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=18, tm_min=37, tm_sec=8, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T18:37:10.963 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2026-02-24T18:31:57.356642+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=18, tm_min=31, tm_sec=57, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=18, tm_min=37, tm_sec=8, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T18:37:10.963 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2026-02-24T18:31:56.258459+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=18, tm_min=31, tm_sec=56, tm_wday=1, tm_yday=55, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=24, tm_hour=18, tm_min=37, tm_sec=8, tm_wday=1, tm_yday=55, tm_isdst=0) 2026-02-24T18:37:10.963 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2026-02-24T18:37:30.965 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-24T18:37:31.241 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-24T18:37:31.242 INFO:teuthology.orchestra.run.trial038.stderr:dumped all 2026-02-24T18:37:31.251 INFO:teuthology.orchestra.run.trial038.stdout:{"pg_ready":true,"pg_map":{"version":212,"stamp":"2026-02-24T18:37:29.536479+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459299,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":46,"num_read_kb":37,"num_write":59,"num_write_kb":586,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":314572800,"kb_used":82176,"kb_used_data":1296,"kb_used_omap":4,"kb_used_meta":80827,"kb_avail":314490624,"statfs":{"total":322122547200,"available":322038398976,"internally_reserved":0,"allocated":1327104,"data_stored":1034677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4779,"internal_metadata":82767189},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 0 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"," osd: 1 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001625"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":32,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-24T18:37:15.695717+0000","last_change":"2026-02-24T18:37:15.696033+0000","last_active":"2026-02-24T18:37:15.695717+0000","last_peered":"2026-02-24T18:37:15.695717+0000","last_clean":"2026-02-24T18:37:15.695717+0000","last_became_active":"2026-02-24T18:31:58.659266+0000","last_became_peered":"2026-02-24T18:31:58.659266+0000","last_unstale":"2026-02-24T18:37:15.695717+0000","last_undegraded":"2026-02-24T18:37:15.695717+0000","last_fullsized":"2026-02-24T18:37:15.695717+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T18:37:15.693913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T18:37:15.693913+0000","last_clean_scrub_stamp":"2026-02-24T18:37:15.693913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T18:47:59.708485+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0.058966485999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":32,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-24T18:37:12.561313+0000","last_change":"2026-02-24T18:37:12.561706+0000","last_active":"2026-02-24T18:37:12.561313+0000","last_peered":"2026-02-24T18:37:12.561313+0000","last_clean":"2026-02-24T18:37:12.561313+0000","last_became_active":"2026-02-24T18:31:58.566235+0000","last_became_peered":"2026-02-24T18:31:58.566235+0000","last_unstale":"2026-02-24T18:37:12.561313+0000","last_undegraded":"2026-02-24T18:37:12.561313+0000","last_fullsized":"2026-02-24T18:37:12.561313+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T18:37:12.559837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T18:37:12.559837+0000","last_clean_scrub_stamp":"2026-02-24T18:37:12.559837+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:22:08.346235+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0.062211901999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":32,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-24T18:37:13.611226+0000","last_change":"2026-02-24T18:37:13.611720+0000","last_active":"2026-02-24T18:37:13.611226+0000","last_peered":"2026-02-24T18:37:13.611226+0000","last_clean":"2026-02-24T18:37:13.611226+0000","last_became_active":"2026-02-24T18:31:58.598935+0000","last_became_peered":"2026-02-24T18:31:58.598935+0000","last_unstale":"2026-02-24T18:37:13.611226+0000","last_undegraded":"2026-02-24T18:37:13.611226+0000","last_fullsized":"2026-02-24T18:37:13.611226+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T18:37:13.609371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T18:37:13.609371+0000","last_clean_scrub_stamp":"2026-02-24T18:37:13.609371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:32:05.190325+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0.057800998999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":32,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-24T18:37:11.521097+0000","last_change":"2026-02-24T18:37:11.521436+0000","last_active":"2026-02-24T18:37:11.521097+0000","last_peered":"2026-02-24T18:37:11.521097+0000","last_clean":"2026-02-24T18:37:11.521097+0000","last_became_active":"2026-02-24T18:31:58.569720+0000","last_became_peered":"2026-02-24T18:31:58.569720+0000","last_unstale":"2026-02-24T18:37:11.521097+0000","last_undegraded":"2026-02-24T18:37:11.521097+0000","last_fullsized":"2026-02-24T18:37:11.521097+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T18:37:11.519656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T18:37:11.519656+0000","last_clean_scrub_stamp":"2026-02-24T18:37:11.519656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T04:46:23.446801+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0.05352788,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"15'3","reported_seq":35,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-24T18:37:10.325240+0000","last_change":"2026-02-24T18:37:10.330646+0000","last_active":"2026-02-24T18:37:10.325240+0000","last_peered":"2026-02-24T18:37:10.325240+0000","last_clean":"2026-02-24T18:37:10.325240+0000","last_became_active":"2026-02-24T18:31:58.581467+0000","last_became_peered":"2026-02-24T18:31:58.581467+0000","last_unstale":"2026-02-24T18:37:10.325240+0000","last_undegraded":"2026-02-24T18:37:10.325240+0000","last_fullsized":"2026-02-24T18:37:10.325240+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"15'3","last_scrub_stamp":"2026-02-24T18:37:10.322131+0000","last_deep_scrub":"15'3","last_deep_scrub_stamp":"2026-02-24T18:37:10.322131+0000","last_clean_scrub_stamp":"2026-02-24T18:37:10.322131+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:17:35.845245+0000","scrub_duration":250,"objects_trimmed":0,"snaptrim_duration":0.039524833000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":32,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-24T18:37:11.074071+0000","last_change":"2026-02-24T18:37:11.078770+0000","last_active":"2026-02-24T18:37:11.074071+0000","last_peered":"2026-02-24T18:37:11.074071+0000","last_clean":"2026-02-24T18:37:11.074071+0000","last_became_active":"2026-02-24T18:31:58.752939+0000","last_became_peered":"2026-02-24T18:31:58.752939+0000","last_unstale":"2026-02-24T18:37:11.074071+0000","last_undegraded":"2026-02-24T18:37:11.074071+0000","last_fullsized":"2026-02-24T18:37:11.074071+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T18:37:11.070581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T18:37:11.070581+0000","last_clean_scrub_stamp":"2026-02-24T18:37:11.070581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T23:33:14.378385+0000","scrub_duration":122,"objects_trimmed":0,"snaptrim_duration":0.048502458999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":32,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-24T18:37:12.003812+0000","last_change":"2026-02-24T18:37:12.004132+0000","last_active":"2026-02-24T18:37:12.003812+0000","last_peered":"2026-02-24T18:37:12.003812+0000","last_clean":"2026-02-24T18:37:12.003812+0000","last_became_active":"2026-02-24T18:31:58.746390+0000","last_became_peered":"2026-02-24T18:31:58.746390+0000","last_unstale":"2026-02-24T18:37:12.003812+0000","last_undegraded":"2026-02-24T18:37:12.003812+0000","last_fullsized":"2026-02-24T18:37:12.003812+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-24T18:37:12.003080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-24T18:37:12.003080+0000","last_clean_scrub_stamp":"2026-02-24T18:37:12.003080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T02:27:32.705464+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0.041638965999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"15'2","reported_seq":34,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-24T18:37:14.711985+0000","last_change":"2026-02-24T18:37:14.712503+0000","last_active":"2026-02-24T18:37:14.711985+0000","last_peered":"2026-02-24T18:37:14.711985+0000","last_clean":"2026-02-24T18:37:14.711985+0000","last_became_active":"2026-02-24T18:31:58.627943+0000","last_became_peered":"2026-02-24T18:31:58.627943+0000","last_unstale":"2026-02-24T18:37:14.711985+0000","last_undegraded":"2026-02-24T18:37:14.711985+0000","last_fullsized":"2026-02-24T18:37:14.711985+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"15'2","last_scrub_stamp":"2026-02-24T18:37:14.711197+0000","last_deep_scrub":"15'2","last_deep_scrub_stamp":"2026-02-24T18:37:14.711197+0000","last_clean_scrub_stamp":"2026-02-24T18:37:14.711197+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-26T01:01:10.875114+0000","scrub_duration":79,"objects_trimmed":0,"snaptrim_duration":0.050969995999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"15'34","reported_seq":75,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-24T18:37:10.697016+0000","last_change":"2026-02-24T18:37:10.700336+0000","last_active":"2026-02-24T18:37:10.697016+0000","last_peered":"2026-02-24T18:37:10.697016+0000","last_clean":"2026-02-24T18:37:10.697016+0000","last_became_active":"2026-02-24T18:31:57.737892+0000","last_became_peered":"2026-02-24T18:31:57.737892+0000","last_unstale":"2026-02-24T18:37:10.697016+0000","last_undegraded":"2026-02-24T18:37:10.697016+0000","last_fullsized":"2026-02-24T18:37:10.697016+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"15'34","last_scrub_stamp":"2026-02-24T18:37:10.694884+0000","last_deep_scrub":"15'34","last_deep_scrub_stamp":"2026-02-24T18:37:10.694884+0000","last_clean_scrub_stamp":"2026-02-24T18:37:10.694884+0000","objects_scrubbed":2,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-25T22:58:35.082781+0000","scrub_duration":218,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":46,"num_read_kb":37,"num_write":57,"num_write_kb":584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":46,"num_read_kb":37,"num_write":57,"num_write_kb":584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":925696,"data_stored":918560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":34,"ondisk_log_size":34,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":1,"up_from":8,"seq":34359738437,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27544,"kb_used_data":584,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":104830056,"statfs":{"total":107374182400,"available":107345977344,"internally_reserved":0,"allocated":598016,"data_stored":497992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 1 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":0,"up_from":9,"seq":38654705732,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27544,"kb_used_data":584,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":104830056,"statfs":{"total":107374182400,"available":107345977344,"internally_reserved":0,"allocated":598016,"data_stored":497992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27589065},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[" osd: 0 BLUESTORE_SLOW_OP_ALERT:observed slow operation indications in BlueStore"]},{"osd":2,"up_from":8,"seq":34359738438,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27088,"kb_used_data":128,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":104830512,"statfs":{"total":107374182400,"available":107346444288,"internally_reserved":0,"allocated":131072,"data_stored":38693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":462848,"data_stored":459280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":462848,"data_stored":459280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-24T18:37:31.253 DEBUG:teuthology.orchestra.run.trial038:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2026-02-24T18:37:31.600 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e6d9099ce03dc21193fda653159ec82465c3a94a/teuthology/contextutil.py", line 34, in nested yield vars File "/home/teuthworker/src/github.com_ceph_ceph-c_f90f4eaebb28b619cf76e80cbbe997656da69579/qa/tasks/ceph.py", line 1922, in task healthy(ctx=ctx, config=dict(cluster=config['cluster'])) File "/home/teuthworker/src/github.com_ceph_ceph-c_f90f4eaebb28b619cf76e80cbbe997656da69579/qa/tasks/ceph.py", line 1484, in healthy manager.wait_until_healthy(timeout=300) File "/home/teuthworker/src/github.com_ceph_ceph-c_f90f4eaebb28b619cf76e80cbbe997656da69579/qa/tasks/ceph_manager.py", line 3303, in wait_until_healthy assert time.time() - start < timeout, \ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ AssertionError: timeout expired in wait_until_healthy 2026-02-24T18:37:31.604 INFO:teuthology.misc:Shutting down mds daemons... 2026-02-24T18:37:31.604 INFO:teuthology.misc:Shutting down osd daemons... 2026-02-24T18:37:31.605 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2026-02-24T18:37:31.605 INFO:teuthology.orchestra.run:waiting for 1200 2026-02-24T18:37:31.612 INFO:tasks.ceph.osd.0.trial038.stderr:2026-02-24T18:37:31.606+0000 caaf640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 0 (PID: 43550) UID: 0 2026-02-24T18:37:31.613 INFO:tasks.ceph.osd.0.trial038.stderr:2026-02-24T18:37:31.607+0000 caaf640 -1 osd.0 15 *** Got signal Terminated *** 2026-02-24T18:37:37.706 INFO:tasks.ceph.osd.0:Stopped 2026-02-24T18:37:37.707 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2026-02-24T18:37:37.707 INFO:teuthology.orchestra.run:waiting for 1200 2026-02-24T18:37:37.714 INFO:tasks.ceph.osd.1.trial038.stderr:2026-02-24T18:37:37.707+0000 caaf640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 1 (PID: 43553) UID: 0 2026-02-24T18:37:37.715 INFO:tasks.ceph.osd.1.trial038.stderr:2026-02-24T18:37:37.708+0000 caaf640 -1 osd.1 17 *** Got signal Terminated *** 2026-02-24T18:37:38.403 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~0s 2026-02-24T18:37:43.808 INFO:tasks.ceph.osd.1:Stopped 2026-02-24T18:37:43.809 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2026-02-24T18:37:43.809 INFO:teuthology.orchestra.run:waiting for 1200 2026-02-24T18:37:43.816 INFO:tasks.ceph.osd.2.trial038.stderr:2026-02-24T18:37:43.809+0000 caaf640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-osd -f --cluster ceph -i 2 (PID: 43557) UID: 0 2026-02-24T18:37:43.817 INFO:tasks.ceph.osd.2.trial038.stderr:2026-02-24T18:37:43.810+0000 caaf640 -1 osd.2 19 *** Got signal Terminated *** 2026-02-24T18:37:44.007 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~0s 2026-02-24T18:37:44.007 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 has been restored 2026-02-24T18:37:49.510 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~0s 2026-02-24T18:37:49.511 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 has been restored 2026-02-24T18:37:49.910 INFO:tasks.ceph.osd.2:Stopped 2026-02-24T18:37:49.911 INFO:teuthology.misc:Shutting down mgr daemons... 2026-02-24T18:37:49.911 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2026-02-24T18:37:49.911 INFO:teuthology.orchestra.run:waiting for 1200 2026-02-24T18:37:49.912 INFO:tasks.ceph.mgr.x.trial038.stderr:2026-02-24T18:37:49.910+0000 7f0872bce640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term ceph-mgr -f --cluster ceph -i x (PID: 43047) UID: 0 2026-02-24T18:37:49.912 INFO:tasks.ceph.mgr.x.trial038.stderr:2026-02-24T18:37:49.910+0000 7f0872bce640 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2026-02-24T18:37:49.945 INFO:tasks.ceph.mgr.x:Stopped 2026-02-24T18:37:49.945 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2026-02-24T18:37:49.945 INFO:teuthology.orchestra.run:waiting for 1200 2026-02-24T18:37:50.012 INFO:tasks.ceph.mgr.y:Stopped 2026-02-24T18:37:50.012 INFO:teuthology.misc:Shutting down mon daemons... 2026-02-24T18:37:50.013 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2026-02-24T18:37:50.013 INFO:teuthology.orchestra.run:waiting for 1200 2026-02-24T18:37:50.060 INFO:tasks.ceph.mon.a.trial038.stderr:2026-02-24T18:37:50.053+0000 aca6640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/mon.a.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i a (PID: 43024) UID: 0 2026-02-24T18:37:50.060 INFO:tasks.ceph.mon.a.trial038.stderr:2026-02-24T18:37:50.053+0000 aca6640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2026-02-24T18:37:54.713 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~0s 2026-02-24T18:37:54.713 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 has been restored 2026-02-24T18:37:56.115 INFO:tasks.ceph.mon.a:Stopped 2026-02-24T18:37:56.115 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2026-02-24T18:37:56.116 INFO:teuthology.orchestra.run:waiting for 1200 2026-02-24T18:37:56.124 INFO:tasks.ceph.mon.b.trial038.stderr:2026-02-24T18:37:56.117+0000 aca6640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/mon.b.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i b (PID: 43037) UID: 0 2026-02-24T18:37:56.125 INFO:tasks.ceph.mon.b.trial038.stderr:2026-02-24T18:37:56.118+0000 aca6640 -1 mon.b@1(peon) e1 *** Got Signal Terminated *** 2026-02-24T18:37:59.814 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~0s 2026-02-24T18:37:59.815 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a has been restored 2026-02-24T18:38:02.217 INFO:tasks.ceph.mon.b:Stopped 2026-02-24T18:38:02.217 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2026-02-24T18:38:02.217 INFO:teuthology.orchestra.run:waiting for 1200 2026-02-24T18:38:02.226 INFO:tasks.ceph.mon.c.trial038.stderr:2026-02-24T18:38:02.219+0000 aca6640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --gen-suppressions=all --xml=yes --xml-file=/var/log/ceph/valgrind/mon.c.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i c (PID: 43044) UID: 0 2026-02-24T18:38:02.226 INFO:tasks.ceph.mon.c.trial038.stderr:2026-02-24T18:38:02.220+0000 aca6640 -1 mon.c@2(probing) e1 *** Got Signal Terminated *** 2026-02-24T18:38:04.816 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~0s 2026-02-24T18:38:04.816 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b has been restored 2026-02-24T18:38:08.320 INFO:tasks.ceph.mon.c:Stopped 2026-02-24T18:38:08.320 INFO:tasks.ceph:Checking cluster log for badness... 2026-02-24T18:38:08.321 DEBUG:teuthology.orchestra.run.trial038:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(PG_' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'OSD bench result' | head -n 1 2026-02-24T18:38:08.349 INFO:teuthology.orchestra.run.trial038.stdout:2026-02-24T18:31:56.256593+0000 mon.a (mon.0) 62 : cluster [WRN] Health check failed: 2 OSD(s) experiencing slow operations in BlueStore (BLUESTORE_SLOW_OP_ALERT) 2026-02-24T18:38:08.350 WARNING:tasks.ceph:Found errors (ERR|WRN|SEC) in cluster log 2026-02-24T18:38:08.350 DEBUG:teuthology.orchestra.run.trial038:> sudo egrep '\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(PG_' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'OSD bench result' | head -n 1 2026-02-24T18:38:08.416 DEBUG:teuthology.orchestra.run.trial038:> sudo egrep '\[ERR\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(PG_' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'OSD bench result' | head -n 1 2026-02-24T18:38:08.482 DEBUG:teuthology.orchestra.run.trial038:> sudo egrep '\[WRN\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(PG_' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'OSD bench result' | head -n 1 2026-02-24T18:38:08.547 INFO:teuthology.orchestra.run.trial038.stdout:2026-02-24T18:31:56.256593+0000 mon.a (mon.0) 62 : cluster [WRN] Health check failed: 2 OSD(s) experiencing slow operations in BlueStore (BLUESTORE_SLOW_OP_ALERT) 2026-02-24T18:38:08.548 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@trial038.front.sepia.ceph.com 2026-02-24T18:38:08.548 DEBUG:teuthology.orchestra.run.trial038:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2026-02-24T18:38:08.617 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@trial038.front.sepia.ceph.com 2026-02-24T18:38:08.617 DEBUG:teuthology.orchestra.run.trial038:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2026-02-24T18:38:08.691 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@trial038.front.sepia.ceph.com 2026-02-24T18:38:08.691 DEBUG:teuthology.orchestra.run.trial038:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2026-02-24T18:38:08.761 INFO:tasks.ceph:Archiving mon data... 2026-02-24T18:38:08.763 DEBUG:teuthology.misc:Transferring archived files from trial038:/var/lib/ceph/mon/ceph-a to /home/teuthworker/mnt/teuthology/yuriw-2026-02-24_16:22:43-rados-wip-yuri6-testing-2026-02-22-1629-squid-distro-default-trial/68639/data/mon.a.tgz 2026-02-24T18:38:08.763 DEBUG:teuthology.orchestra.run.trial038:> mktemp 2026-02-24T18:38:08.819 INFO:teuthology.orchestra.run.trial038.stdout:/tmp/tmp.aHhOcww9uW 2026-02-24T18:38:08.820 DEBUG:teuthology.orchestra.run.trial038:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.aHhOcww9uW 2026-02-24T18:38:08.915 DEBUG:teuthology.orchestra.run.trial038:> sudo chmod 0666 /tmp/tmp.aHhOcww9uW 2026-02-24T18:38:08.998 DEBUG:teuthology.orchestra.remote:trial038:/tmp/tmp.aHhOcww9uW is 275KB 2026-02-24T18:38:09.093 DEBUG:teuthology.orchestra.run.trial038:> rm -fr /tmp/tmp.aHhOcww9uW 2026-02-24T18:38:09.110 DEBUG:teuthology.misc:Transferring archived files from trial038:/var/lib/ceph/mon/ceph-b to /home/teuthworker/mnt/teuthology/yuriw-2026-02-24_16:22:43-rados-wip-yuri6-testing-2026-02-22-1629-squid-distro-default-trial/68639/data/mon.b.tgz 2026-02-24T18:38:09.110 DEBUG:teuthology.orchestra.run.trial038:> mktemp 2026-02-24T18:38:09.166 INFO:teuthology.orchestra.run.trial038.stdout:/tmp/tmp.swwNJsAOBJ 2026-02-24T18:38:09.167 DEBUG:teuthology.orchestra.run.trial038:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.swwNJsAOBJ 2026-02-24T18:38:09.276 DEBUG:teuthology.orchestra.run.trial038:> sudo chmod 0666 /tmp/tmp.swwNJsAOBJ 2026-02-24T18:38:09.358 DEBUG:teuthology.orchestra.remote:trial038:/tmp/tmp.swwNJsAOBJ is 377KB 2026-02-24T18:38:09.453 DEBUG:teuthology.orchestra.run.trial038:> rm -fr /tmp/tmp.swwNJsAOBJ 2026-02-24T18:38:09.469 DEBUG:teuthology.misc:Transferring archived files from trial038:/var/lib/ceph/mon/ceph-c to /home/teuthworker/mnt/teuthology/yuriw-2026-02-24_16:22:43-rados-wip-yuri6-testing-2026-02-22-1629-squid-distro-default-trial/68639/data/mon.c.tgz 2026-02-24T18:38:09.470 DEBUG:teuthology.orchestra.run.trial038:> mktemp 2026-02-24T18:38:09.525 INFO:teuthology.orchestra.run.trial038.stdout:/tmp/tmp.CCoZQCaUc9 2026-02-24T18:38:09.525 DEBUG:teuthology.orchestra.run.trial038:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.CCoZQCaUc9 2026-02-24T18:38:09.633 DEBUG:teuthology.orchestra.run.trial038:> sudo chmod 0666 /tmp/tmp.CCoZQCaUc9 2026-02-24T18:38:09.713 DEBUG:teuthology.orchestra.remote:trial038:/tmp/tmp.CCoZQCaUc9 is 377KB 2026-02-24T18:38:09.806 DEBUG:teuthology.orchestra.run.trial038:> rm -fr /tmp/tmp.CCoZQCaUc9 2026-02-24T18:38:09.817 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c has been restored 2026-02-24T18:38:09.823 INFO:tasks.ceph:Cleaning ceph cluster... 2026-02-24T18:38:09.823 DEBUG:teuthology.orchestra.run.trial038:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2026-02-24T18:38:10.264 DEBUG:tasks.ceph:valgrind exception message: valgrind error: Leak_StillReachable malloc malloc _dl_new_object 2026-02-24T18:38:10.264 INFO:tasks.ceph:Archiving crash dumps... 2026-02-24T18:38:10.267 DEBUG:teuthology.misc:Transferring archived files from trial038:/var/lib/ceph/crash to /home/teuthworker/mnt/teuthology/yuriw-2026-02-24_16:22:43-rados-wip-yuri6-testing-2026-02-22-1629-squid-distro-default-trial/68639/remote/trial038/crash 2026-02-24T18:38:10.267 DEBUG:teuthology.orchestra.run.trial038:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2026-02-24T18:38:10.292 INFO:tasks.ceph:Compressing logs... 2026-02-24T18:38:10.292 DEBUG:teuthology.orchestra.run.trial038:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2026-02-24T18:38:10.379 INFO:teuthology.orchestra.run.trial038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50490.log 2026-02-24T18:38:10.379 INFO:teuthology.orchestra.run.trial038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2026-02-24T18:38:10.379 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.50490.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2026-02-24T18:38:10.379 INFO:teuthology.orchestra.run.trial038.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50490.log.gz 2026-02-24T18:38:10.379 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47094.log 2026-02-24T18:38:10.379 INFO:teuthology.orchestra.run.trial038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46069.log 2026-02-24T18:38:10.379 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-mon.a.log: /var/log/ceph/ceph-client.admin.47094.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47094.log.gz 2026-02-24T18:38:10.379 INFO:teuthology.orchestra.run.trial038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47682.log 2026-02-24T18:38:10.379 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.46069.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.46069.log.gz -5 2026-02-24T18:38:10.379 INFO:teuthology.orchestra.run.trial038.stderr: --verbose -- /var/log/ceph/ceph-client.admin.46189.log 2026-02-24T18:38:10.379 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.47682.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47682.log.gz 2026-02-24T18:38:10.379 INFO:teuthology.orchestra.run.trial038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46750.log 2026-02-24T18:38:10.379 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.46189.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46189.log.gz 2026-02-24T18:38:10.380 INFO:teuthology.orchestra.run.trial038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50834.log 2026-02-24T18:38:10.380 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.46750.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46750.log.gz 2026-02-24T18:38:10.380 INFO:teuthology.orchestra.run.trial038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2026-02-24T18:38:10.380 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.50834.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50834.log.gz 2026-02-24T18:38:10.380 INFO:teuthology.orchestra.run.trial038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48172.log 2026-02-24T18:38:10.380 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50098.log 2026-02-24T18:38:10.380 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.48172.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48172.log.gz 2026-02-24T18:38:10.380 INFO:teuthology.orchestra.run.trial038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50049.log 2026-02-24T18:38:10.380 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.50098.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48074.log 2026-02-24T18:38:10.380 INFO:teuthology.orchestra.run.trial038.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50098.log.gz 2026-02-24T18:38:10.380 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.50049.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50049.log.gz 2026-02-24T18:38:10.380 INFO:teuthology.orchestra.run.trial038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50196.log 2026-02-24T18:38:10.380 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.48074.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2026-02-24T18:38:10.380 INFO:teuthology.orchestra.run.trial038.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48074.log.gz 2026-02-24T18:38:10.380 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.50196.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50196.log.gz 2026-02-24T18:38:10.380 INFO:teuthology.orchestra.run.trial038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45966.log 2026-02-24T18:38:10.380 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49705.log 2026-02-24T18:38:10.380 INFO:teuthology.orchestra.run.trial038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50539.log 2026-02-24T18:38:10.380 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.45966.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45966.log.gz 2026-02-24T18:38:10.380 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.49705.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49705.log.gz 2026-02-24T18:38:10.380 INFO:teuthology.orchestra.run.trial038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48564.log 2026-02-24T18:38:10.380 INFO:teuthology.orchestra.run.trial038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50392.log 2026-02-24T18:38:10.380 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.50539.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50539.log.gz 2026-02-24T18:38:10.380 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.48564.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48564.log.gz 2026-02-24T18:38:10.380 INFO:teuthology.orchestra.run.trial038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46701.log 2026-02-24T18:38:10.380 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.50392.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50441.log 2026-02-24T18:38:10.380 INFO:teuthology.orchestra.run.trial038.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50392.log.gz 2026-02-24T18:38:10.380 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.46701.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45374.log 2026-02-24T18:38:10.380 INFO:teuthology.orchestra.run.trial038.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46701.log.gz 2026-02-24T18:38:10.381 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.50441.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46848.log 2026-02-24T18:38:10.381 INFO:teuthology.orchestra.run.trial038.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50441.log.gz 2026-02-24T18:38:10.381 INFO:teuthology.orchestra.run.trial038.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.45374.log: -- /var/log/ceph/ceph-client.admin.48711.log 2026-02-24T18:38:10.381 INFO:teuthology.orchestra.run.trial038.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45374.log.gz 2026-02-24T18:38:10.381 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.46848.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46848.log.gz 2026-02-24T18:38:10.381 INFO:teuthology.orchestra.run.trial038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44631.log 2026-02-24T18:38:10.381 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.48711.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48711.log.gz 2026-02-24T18:38:10.381 INFO:teuthology.orchestra.run.trial038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49902.log 2026-02-24T18:38:10.381 INFO:teuthology.orchestra.run.trial038.stderr: 94.0% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2026-02-24T18:38:10.381 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.44631.log: gzip -5 0.0% --verbose -- replaced with /var/log/ceph/ceph-client.admin.44631.log.gz -- 2026-02-24T18:38:10.381 INFO:teuthology.orchestra.run.trial038.stderr: /var/log/ceph/ceph-client.admin.48221.log 2026-02-24T18:38:10.381 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.49902.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49902.log.gz 2026-02-24T18:38:10.381 INFO:teuthology.orchestra.run.trial038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51373.log 2026-02-24T18:38:10.381 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.48221.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51177.log 2026-02-24T18:38:10.381 INFO:teuthology.orchestra.run.trial038.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48221.log.gz 2026-02-24T18:38:10.381 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.51373.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51373.log.gz 2026-02-24T18:38:10.381 INFO:teuthology.orchestra.run.trial038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2026-02-24T18:38:10.381 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.51177.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51177.log.gz 2026-02-24T18:38:10.381 INFO:teuthology.orchestra.run.trial038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50981.log 2026-02-24T18:38:10.381 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46068.log 2026-02-24T18:38:10.381 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.50981.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50981.log.gz 2026-02-24T18:38:10.381 INFO:teuthology.orchestra.run.trial038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47584.log 2026-02-24T18:38:10.381 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.46068.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51422.log 2026-02-24T18:38:10.381 INFO:teuthology.orchestra.run.trial038.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46068.log.gz 2026-02-24T18:38:10.381 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.47584.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47584.log.gz 2026-02-24T18:38:10.381 INFO:teuthology.orchestra.run.trial038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46995.log 2026-02-24T18:38:10.381 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.51422.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47290.log 2026-02-24T18:38:10.381 INFO:teuthology.orchestra.run.trial038.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51422.log.gz 2026-02-24T18:38:10.381 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.46995.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46995.log.gz 2026-02-24T18:38:10.381 INFO:teuthology.orchestra.run.trial038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43561.log 2026-02-24T18:38:10.381 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.47290.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46946.log 2026-02-24T18:38:10.381 INFO:teuthology.orchestra.run.trial038.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47290.log.gz 2026-02-24T18:38:10.381 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.43561.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43561.log.gz 2026-02-24T18:38:10.382 INFO:teuthology.orchestra.run.trial038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45420.log 2026-02-24T18:38:10.382 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.46946.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51226.log 2026-02-24T18:38:10.382 INFO:teuthology.orchestra.run.trial038.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46946.log.gz 2026-02-24T18:38:10.382 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.45420.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43736.log 2026-02-24T18:38:10.382 INFO:teuthology.orchestra.run.trial038.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45420.log.gz 2026-02-24T18:38:10.382 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.51226.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51226.log.gz 2026-02-24T18:38:10.382 INFO:teuthology.orchestra.run.trial038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48270.log 2026-02-24T18:38:10.382 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.43736.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43736.log.gz 2026-02-24T18:38:10.382 INFO:teuthology.orchestra.run.trial038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50686.log 2026-02-24T18:38:10.382 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.48270.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48270.log.gz 2026-02-24T18:38:10.382 INFO:teuthology.orchestra.run.trial038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48858.log 2026-02-24T18:38:10.382 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.50686.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50686.log.gz 2026-02-24T18:38:10.382 INFO:teuthology.orchestra.run.trial038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49067.log 2026-02-24T18:38:10.382 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.48858.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48858.log.gz 2026-02-24T18:38:10.382 INFO:teuthology.orchestra.run.trial038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43340.log 2026-02-24T18:38:10.382 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.49067.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49067.log.gz 2026-02-24T18:38:10.382 INFO:teuthology.orchestra.run.trial038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51716.log 2026-02-24T18:38:10.382 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.43340.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43340.log.gz 2026-02-24T18:38:10.382 INFO:teuthology.orchestra.run.trial038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48662.log 2026-02-24T18:38:10.382 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.51716.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49460.log 2026-02-24T18:38:10.382 INFO:teuthology.orchestra.run.trial038.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51716.log.gz 2026-02-24T18:38:10.382 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.48662.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48662.log.gz 2026-02-24T18:38:10.382 INFO:teuthology.orchestra.run.trial038.stderr:gzip -5 --verbose -- /var/log/ceph/valgrind/osd.0.log 2026-02-24T18:38:10.382 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.49460.log: gzip -5 --verbose -- /var/log/ceph/valgrind/osd.1.log 2026-02-24T18:38:10.382 INFO:teuthology.orchestra.run.trial038.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49460.log.gz 2026-02-24T18:38:10.382 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/valgrind/osd.0.log: gzip -5 --verbose -- /var/log/ceph/valgrind/osd.2.log 2026-02-24T18:38:10.382 INFO:teuthology.orchestra.run.trial038.stderr: 68.2% -- replaced with /var/log/ceph/valgrind/osd.0.log.gz 2026-02-24T18:38:10.382 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/valgrind/osd.1.log: gzip -5 --verbose -- /var/log/ceph/valgrind/mon.a.log 2026-02-24T18:38:10.382 INFO:teuthology.orchestra.run.trial038.stderr: 67.9% -- replaced with /var/log/ceph/valgrind/osd.1.log.gz 2026-02-24T18:38:10.382 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/valgrind/osd.2.log: gzip -5 --verbose -- /var/log/ceph/valgrind/mon.b.log 2026-02-24T18:38:10.382 INFO:teuthology.orchestra.run.trial038.stderr: 68.2% -- replaced with /var/log/ceph/valgrind/osd.2.log.gz 2026-02-24T18:38:10.382 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/valgrind/mon.a.log: gzip -5 --verbose -- /var/log/ceph/valgrind/mon.c.log 2026-02-24T18:38:10.383 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/valgrind/mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49411.log 2026-02-24T18:38:10.383 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/valgrind/mon.c.log: 94.1% -- replaced with /var/log/ceph/valgrind/mon.a.log.gz 2026-02-24T18:38:10.383 INFO:teuthology.orchestra.run.trial038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49754.log 2026-02-24T18:38:10.383 INFO:teuthology.orchestra.run.trial038.stderr: 94.1% -- replaced with /var/log/ceph/valgrind/mon.b.log.gz 2026-02-24T18:38:10.383 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.49411.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47388.log 0.0% 2026-02-24T18:38:10.383 INFO:teuthology.orchestra.run.trial038.stderr: -- replaced with /var/log/ceph/ceph-client.admin.49411.log.gz 2026-02-24T18:38:10.383 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.49754.log: 94.1% -- replaced with /var/log/ceph/valgrind/mon.c.log.gz 2026-02-24T18:38:10.383 INFO:teuthology.orchestra.run.trial038.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49754.log.gz 2026-02-24T18:38:10.383 INFO:teuthology.orchestra.run.trial038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46603.log 2026-02-24T18:38:10.383 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.47388.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47388.log.gz 2026-02-24T18:38:10.383 INFO:teuthology.orchestra.run.trial038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49853.log 2026-02-24T18:38:10.383 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.46603.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.46603.log.gz --verbose 2026-02-24T18:38:10.383 INFO:teuthology.orchestra.run.trial038.stderr: -- /var/log/ceph/ceph-client.admin.49018.log 2026-02-24T18:38:10.383 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.49853.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46799.log 2026-02-24T18:38:10.383 INFO:teuthology.orchestra.run.trial038.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49853.log.gz 2026-02-24T18:38:10.383 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.49018.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51275.log 2026-02-24T18:38:10.383 INFO:teuthology.orchestra.run.trial038.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49018.log.gz 2026-02-24T18:38:10.383 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.46799.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46799.log.gz 2026-02-24T18:38:10.383 INFO:teuthology.orchestra.run.trial038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46456.log 2026-02-24T18:38:10.383 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.51275.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51275.log.gz 2026-02-24T18:38:10.383 INFO:teuthology.orchestra.run.trial038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50784.log 2026-02-24T18:38:10.383 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.46456.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46456.log.gz 2026-02-24T18:38:10.383 INFO:teuthology.orchestra.run.trial038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50000.log 2026-02-24T18:38:10.383 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.50784.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50147.log 2026-02-24T18:38:10.383 INFO:teuthology.orchestra.run.trial038.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50784.log.gz 2026-02-24T18:38:10.383 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.50000.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50000.log.gz 2026-02-24T18:38:10.383 INFO:teuthology.orchestra.run.trial038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51079.log 2026-02-24T18:38:10.383 INFO:teuthology.orchestra.run.trial038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47241.log 2026-02-24T18:38:10.383 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.50147.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50147.log.gz 2026-02-24T18:38:10.383 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.51079.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51079.log.gz 2026-02-24T18:38:10.383 INFO:teuthology.orchestra.run.trial038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45466.log 2026-02-24T18:38:10.383 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.47241.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46185.log 2026-02-24T18:38:10.383 INFO:teuthology.orchestra.run.trial038.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47241.log.gz 2026-02-24T18:38:10.383 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.45466.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45466.log.gz 2026-02-24T18:38:10.383 INFO:teuthology.orchestra.run.trial038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47044.log 2026-02-24T18:38:10.384 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.46185.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46185.log.gz 2026-02-24T18:38:10.384 INFO:teuthology.orchestra.run.trial038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48515.log 2026-02-24T18:38:10.384 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.47044.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49264.log 2026-02-24T18:38:10.384 INFO:teuthology.orchestra.run.trial038.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47044.log.gz 2026-02-24T18:38:10.384 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.48515.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48515.log.gz 2026-02-24T18:38:10.384 INFO:teuthology.orchestra.run.trial038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43434.log 2026-02-24T18:38:10.384 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.49264.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48466.log 2026-02-24T18:38:10.384 INFO:teuthology.orchestra.run.trial038.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49264.log.gz 2026-02-24T18:38:10.384 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.43434.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43434.log.gz 2026-02-24T18:38:10.384 INFO:teuthology.orchestra.run.trial038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51618.log 2026-02-24T18:38:10.384 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.48466.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43812.log 2026-02-24T18:38:10.384 INFO:teuthology.orchestra.run.trial038.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48466.log.gz 2026-02-24T18:38:10.384 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.51618.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51618.log.gz 2026-02-24T18:38:10.384 INFO:teuthology.orchestra.run.trial038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46067.log 2026-02-24T18:38:10.384 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.43812.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46897.log 2026-02-24T18:38:10.384 INFO:teuthology.orchestra.run.trial038.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43812.log.gz 2026-02-24T18:38:10.384 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.46067.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46067.log.gz 2026-02-24T18:38:10.384 INFO:teuthology.orchestra.run.trial038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47878.log 2026-02-24T18:38:10.384 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.46897.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45235.log 2026-02-24T18:38:10.384 INFO:teuthology.orchestra.run.trial038.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46897.log.gz 2026-02-24T18:38:10.388 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.47878.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47878.log.gz 2026-02-24T18:38:10.388 INFO:teuthology.orchestra.run.trial038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45327.log 2026-02-24T18:38:10.388 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.45235.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47633.log 2026-02-24T18:38:10.388 INFO:teuthology.orchestra.run.trial038.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45235.log.gz 2026-02-24T18:38:10.388 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.45327.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45327.log.gz 2026-02-24T18:38:10.388 INFO:teuthology.orchestra.run.trial038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45768.log 2026-02-24T18:38:10.388 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.47633.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47731.log 2026-02-24T18:38:10.388 INFO:teuthology.orchestra.run.trial038.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47633.log.gz 2026-02-24T18:38:10.388 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.45768.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45768.log.gz 2026-02-24T18:38:10.388 INFO:teuthology.orchestra.run.trial038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47535.log 2026-02-24T18:38:10.388 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.47731.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2026-02-24T18:38:10.388 INFO:teuthology.orchestra.run.trial038.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47731.log.gz 2026-02-24T18:38:10.388 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.47535.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48123.log 2026-02-24T18:38:10.388 INFO:teuthology.orchestra.run.trial038.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47535.log.gz 2026-02-24T18:38:10.388 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2026-02-24T18:38:10.388 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.48123.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50245.log 2026-02-24T18:38:10.388 INFO:teuthology.orchestra.run.trial038.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48123.log.gz 2026-02-24T18:38:10.388 INFO:teuthology.orchestra.run.trial038.stderr: 92.5% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2026-02-24T18:38:10.388 INFO:teuthology.orchestra.run.trial038.stderr: 95.3% -- replaced with /var/log/ceph/ceph.audit.log.gz 2026-02-24T18:38:10.388 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47486.log 2026-02-24T18:38:10.388 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.50245.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45512.log 2026-02-24T18:38:10.388 INFO:teuthology.orchestra.run.trial038.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50245.log.gz 2026-02-24T18:38:10.388 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.47486.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47486.log.gz 2026-02-24T18:38:10.388 INFO:teuthology.orchestra.run.trial038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49509.log 2026-02-24T18:38:10.388 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.45512.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2026-02-24T18:38:10.388 INFO:teuthology.orchestra.run.trial038.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45512.log.gz 2026-02-24T18:38:10.388 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.49509.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49951.log 2026-02-24T18:38:10.388 INFO:teuthology.orchestra.run.trial038.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49509.log.gz 2026-02-24T18:38:10.388 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46244.log 2026-02-24T18:38:10.388 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.49951.log: 92.5%gzip -- replaced with /var/log/ceph/ceph-osd.0.log.gz -5 2026-02-24T18:38:10.388 INFO:teuthology.orchestra.run.trial038.stderr: --verbose -- /var/log/ceph/ceph-client.admin.49607.log 2026-02-24T18:38:10.389 INFO:teuthology.orchestra.run.trial038.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49951.log.gz 2026-02-24T18:38:10.389 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.46244.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51569.log 2026-02-24T18:38:10.389 INFO:teuthology.orchestra.run.trial038.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46244.log.gz 2026-02-24T18:38:10.389 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.49607.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43897.log 2026-02-24T18:38:10.389 INFO:teuthology.orchestra.run.trial038.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49607.log.gz 2026-02-24T18:38:10.389 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.51569.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51569.log.gz 2026-02-24T18:38:10.389 INFO:teuthology.orchestra.run.trial038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50588.log 2026-02-24T18:38:10.389 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.43897.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43897.log.gz 2026-02-24T18:38:10.389 INFO:teuthology.orchestra.run.trial038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47143.log 2026-02-24T18:38:10.389 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.50588.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50735.log 2026-02-24T18:38:10.389 INFO:teuthology.orchestra.run.trial038.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50588.log.gz 2026-02-24T18:38:10.389 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.47143.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47143.log.gz 2026-02-24T18:38:10.389 INFO:teuthology.orchestra.run.trial038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49656.log 2026-02-24T18:38:10.389 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.50735.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49804.log 2026-02-24T18:38:10.389 INFO:teuthology.orchestra.run.trial038.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50735.log.gz 2026-02-24T18:38:10.389 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.49656.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51324.log 2026-02-24T18:38:10.389 INFO:teuthology.orchestra.run.trial038.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49656.log.gz 2026-02-24T18:38:10.389 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.49804.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49804.log.gz 2026-02-24T18:38:10.389 INFO:teuthology.orchestra.run.trial038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51520.log 2026-02-24T18:38:10.389 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.51324.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50932.log 2026-02-24T18:38:10.389 INFO:teuthology.orchestra.run.trial038.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51324.log.gz 2026-02-24T18:38:10.389 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.51520.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44585.log 2026-02-24T18:38:10.389 INFO:teuthology.orchestra.run.trial038.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51520.log.gz 2026-02-24T18:38:10.389 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.50932.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46532.log 0.0% 2026-02-24T18:38:10.389 INFO:teuthology.orchestra.run.trial038.stderr: -- replaced with /var/log/ceph/ceph-client.admin.50932.log.gz 2026-02-24T18:38:10.389 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.44585.log: gzip -5 --verbose 0.0% -- -- replaced with /var/log/ceph/ceph-client.admin.44585.log.gz /var/log/ceph/ceph-client.admin.50343.log 2026-02-24T18:38:10.389 INFO:teuthology.orchestra.run.trial038.stderr: 2026-02-24T18:38:10.389 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.46532.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48907.log 2026-02-24T18:38:10.389 INFO:teuthology.orchestra.run.trial038.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46532.log.gz 2026-02-24T18:38:10.389 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.50343.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45675.log 2026-02-24T18:38:10.389 INFO:teuthology.orchestra.run.trial038.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50343.log.gz 2026-02-24T18:38:10.389 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.48907.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48907.log.gz 2026-02-24T18:38:10.389 INFO:teuthology.orchestra.run.trial038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48613.log 2026-02-24T18:38:10.389 INFO:teuthology.orchestra.run.trial038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43387.log 2026-02-24T18:38:10.389 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.45675.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45675.log.gz 2026-02-24T18:38:10.391 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.48613.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48613.log.gz 2026-02-24T18:38:10.391 INFO:teuthology.orchestra.run.trial038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46485.log 2026-02-24T18:38:10.391 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.43387.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50883.log 2026-02-24T18:38:10.391 INFO:teuthology.orchestra.run.trial038.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43387.log.gz 2026-02-24T18:38:10.391 INFO:teuthology.orchestra.run.trial038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46338.log 2026-02-24T18:38:10.391 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.46485.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46485.log.gz 2026-02-24T18:38:10.391 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.50883.log: gzip -5 --verbose 0.0% -- -- replaced with /var/log/ceph/ceph-client.admin.50883.log.gz /var/log/ceph/ceph-client.admin.47437.log 2026-02-24T18:38:10.391 INFO:teuthology.orchestra.run.trial038.stderr: 2026-02-24T18:38:10.391 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.46338.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49362.log 2026-02-24T18:38:10.391 INFO:teuthology.orchestra.run.trial038.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46338.log.gz 2026-02-24T18:38:10.391 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.47437.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48760.log 2026-02-24T18:38:10.391 INFO:teuthology.orchestra.run.trial038.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47437.log.gz 2026-02-24T18:38:10.391 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.49362.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51030.log 2026-02-24T18:38:10.391 INFO:teuthology.orchestra.run.trial038.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49362.log.gz 2026-02-24T18:38:10.391 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.48760.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48969.log 2026-02-24T18:38:10.391 INFO:teuthology.orchestra.run.trial038.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48760.log.gz 2026-02-24T18:38:10.391 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.51030.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51030.log.gz 2026-02-24T18:38:10.391 INFO:teuthology.orchestra.run.trial038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51667.log 2026-02-24T18:38:10.391 INFO:teuthology.orchestra.run.trial038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47829.log 2026-02-24T18:38:10.391 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.48969.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48969.log.gz 2026-02-24T18:38:10.391 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.51667.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51667.log.gz 2026-02-24T18:38:10.391 INFO:teuthology.orchestra.run.trial038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2026-02-24T18:38:10.392 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.47829.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48025.log 2026-02-24T18:38:10.392 INFO:teuthology.orchestra.run.trial038.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47829.log.gz 2026-02-24T18:38:10.392 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48417.log 2026-02-24T18:38:10.392 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.48025.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48025.log.gz 2026-02-24T18:38:10.392 INFO:teuthology.orchestra.run.trial038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51765.log 2026-02-24T18:38:10.392 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.48417.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46385.log 2026-02-24T18:38:10.392 INFO:teuthology.orchestra.run.trial038.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48417.log.gz 2026-02-24T18:38:10.392 INFO:teuthology.orchestra.run.trial038.stderr:gzip/var/log/ceph/ceph-client.admin.51765.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.45868.log 2026-02-24T18:38:10.392 INFO:teuthology.orchestra.run.trial038.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51765.log.gz 2026-02-24T18:38:10.392 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.46385.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46385.log.gz 2026-02-24T18:38:10.392 INFO:teuthology.orchestra.run.trial038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51471.log 2026-02-24T18:38:10.393 INFO:teuthology.orchestra.run.trial038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50637.log 2026-02-24T18:38:10.393 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.45868.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45868.log.gz 2026-02-24T18:38:10.393 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.51471.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45917.log 2026-02-24T18:38:10.393 INFO:teuthology.orchestra.run.trial038.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51471.log.gz 2026-02-24T18:38:10.393 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.50637.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50637.log.gz 2026-02-24T18:38:10.393 INFO:teuthology.orchestra.run.trial038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48368.log 2026-02-24T18:38:10.393 INFO:teuthology.orchestra.run.trial038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49117.log 2026-02-24T18:38:10.393 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.45917.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45917.log.gz 2026-02-24T18:38:10.394 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.48368.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2026-02-24T18:38:10.394 INFO:teuthology.orchestra.run.trial038.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48368.log.gz 2026-02-24T18:38:10.394 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.49117.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49117.log.gz 2026-02-24T18:38:10.394 INFO:teuthology.orchestra.run.trial038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50294.log 2026-02-24T18:38:10.394 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.39157.log 2026-02-24T18:38:10.394 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.50294.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50294.log.gz 2026-02-24T18:38:10.394 INFO:teuthology.orchestra.run.trial038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51128.log 2026-02-24T18:38:10.395 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph.tmp-client.admin.39157.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45281.log 2026-02-24T18:38:10.395 INFO:teuthology.orchestra.run.trial038.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.39157.log.gz 2026-02-24T18:38:10.395 INFO:teuthology.orchestra.run.trial038.stderr: 95.1% -- replaced with /var/log/ceph/ceph.log.gz 2026-02-24T18:38:10.395 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.51128.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48319.log 2026-02-24T18:38:10.395 INFO:teuthology.orchestra.run.trial038.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51128.log.gz 2026-02-24T18:38:10.395 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.45281.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.45281.log.gz --verbose 2026-02-24T18:38:10.395 INFO:teuthology.orchestra.run.trial038.stderr: -- /var/log/ceph/ceph-client.admin.46309.log 2026-02-24T18:38:10.395 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.48319.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48319.log.gz 2026-02-24T18:38:10.395 INFO:teuthology.orchestra.run.trial038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49313.log 2026-02-24T18:38:10.396 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.46309.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49215.log 2026-02-24T18:38:10.396 INFO:teuthology.orchestra.run.trial038.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46309.log.gz 2026-02-24T18:38:10.396 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.49313.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47339.log 0.0% 2026-02-24T18:38:10.396 INFO:teuthology.orchestra.run.trial038.stderr: -- replaced with /var/log/ceph/ceph-client.admin.49313.log.gz 2026-02-24T18:38:10.396 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.49215.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47780.log 0.0% 2026-02-24T18:38:10.396 INFO:teuthology.orchestra.run.trial038.stderr: -- replaced with /var/log/ceph/ceph-client.admin.49215.log.gz 2026-02-24T18:38:10.396 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.47339.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47339.log.gz 2026-02-24T18:38:10.396 INFO:teuthology.orchestra.run.trial038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45745.log 2026-02-24T18:38:10.397 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.47780.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47780.log.gz 2026-02-24T18:38:10.397 INFO:teuthology.orchestra.run.trial038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48809.log 2026-02-24T18:38:10.397 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.45745.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49166.log 2026-02-24T18:38:10.397 INFO:teuthology.orchestra.run.trial038.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45745.log.gz 2026-02-24T18:38:10.397 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.48809.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46652.log 2026-02-24T18:38:10.397 INFO:teuthology.orchestra.run.trial038.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48809.log.gz 2026-02-24T18:38:10.397 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.49166.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49166.log.gz 2026-02-24T18:38:10.398 INFO:teuthology.orchestra.run.trial038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51812.log 2026-02-24T18:38:10.398 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.46652.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46652.log.gz 2026-02-24T18:38:10.398 INFO:teuthology.orchestra.run.trial038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47927.log 2026-02-24T18:38:10.398 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.51812.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51812.log.gz 2026-02-24T18:38:10.398 INFO:teuthology.orchestra.run.trial038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43052.log 2026-02-24T18:38:10.398 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.47927.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47927.log.gz 2026-02-24T18:38:10.398 INFO:teuthology.orchestra.run.trial038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49558.log 2026-02-24T18:38:10.399 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.43052.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43052.log.gz 2026-02-24T18:38:10.399 INFO:teuthology.orchestra.run.trial038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47192.log 2026-02-24T18:38:10.399 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.49558.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49558.log.gz 2026-02-24T18:38:10.399 INFO:teuthology.orchestra.run.trial038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43609.log 2026-02-24T18:38:10.399 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.47192.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47192.log.gz 2026-02-24T18:38:10.399 INFO:teuthology.orchestra.run.trial038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47976.log 2026-02-24T18:38:10.399 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.43609.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43609.log.gz 2026-02-24T18:38:10.400 INFO:teuthology.orchestra.run.trial038.stderr:/var/log/ceph/ceph-client.admin.47976.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47976.log.gz 2026-02-24T18:38:10.417 INFO:teuthology.orchestra.run.trial038.stderr: 91.7% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2026-02-24T18:38:10.421 INFO:teuthology.orchestra.run.trial038.stderr: 92.8% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2026-02-24T18:38:10.439 INFO:teuthology.orchestra.run.trial038.stderr: 90.8% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2026-02-24T18:38:10.445 INFO:teuthology.orchestra.run.trial038.stderr: 90.6% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2026-02-24T18:38:10.481 INFO:teuthology.orchestra.run.trial038.stderr: 89.4% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2026-02-24T18:38:10.483 INFO:teuthology.orchestra.run.trial038.stderr: 2026-02-24T18:38:10.483 INFO:teuthology.orchestra.run.trial038.stderr:real 0m0.134s 2026-02-24T18:38:10.483 INFO:teuthology.orchestra.run.trial038.stderr:user 0m0.412s 2026-02-24T18:38:10.483 INFO:teuthology.orchestra.run.trial038.stderr:sys 0m0.080s 2026-02-24T18:38:10.483 INFO:tasks.ceph:Archiving logs... 2026-02-24T18:38:10.483 DEBUG:teuthology.misc:Transferring archived files from trial038:/var/log/ceph to /home/teuthworker/mnt/teuthology/yuriw-2026-02-24_16:22:43-rados-wip-yuri6-testing-2026-02-22-1629-squid-distro-default-trial/68639/remote/trial038/log 2026-02-24T18:38:10.486 DEBUG:teuthology.orchestra.run.trial038:> sudo tar c -f - -C /var/log/ceph -- . 2026-02-24T18:38:10.718 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/github.com_ceph_ceph-c_f90f4eaebb28b619cf76e80cbbe997656da69579/qa/tasks/ceph.py", line 329, in valgrind_post yield File "/home/teuthworker/src/git.ceph.com_teuthology_e6d9099ce03dc21193fda653159ec82465c3a94a/teuthology/contextutil.py", line 34, in nested yield vars File "/home/teuthworker/src/github.com_ceph_ceph-c_f90f4eaebb28b619cf76e80cbbe997656da69579/qa/tasks/ceph.py", line 1922, in task healthy(ctx=ctx, config=dict(cluster=config['cluster'])) File "/home/teuthworker/src/github.com_ceph_ceph-c_f90f4eaebb28b619cf76e80cbbe997656da69579/qa/tasks/ceph.py", line 1484, in healthy manager.wait_until_healthy(timeout=300) File "/home/teuthworker/src/github.com_ceph_ceph-c_f90f4eaebb28b619cf76e80cbbe997656da69579/qa/tasks/ceph_manager.py", line 3303, in wait_until_healthy assert time.time() - start < timeout, \ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ AssertionError: timeout expired in wait_until_healthy During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e6d9099ce03dc21193fda653159ec82465c3a94a/teuthology/run_tasks.py", line 112, in run_tasks manager.__enter__() File "/usr/lib/python3.12/contextlib.py", line 137, in __enter__ return next(self.gen) ^^^^^^^^^^^^^^ File "/home/teuthworker/src/github.com_ceph_ceph-c_f90f4eaebb28b619cf76e80cbbe997656da69579/qa/tasks/ceph.py", line 1919, in task with contextutil.nested(*subtasks): File "/usr/lib/python3.12/contextlib.py", line 158, in __exit__ self.gen.throw(value) File "/home/teuthworker/src/git.ceph.com_teuthology_e6d9099ce03dc21193fda653159ec82465c3a94a/teuthology/contextutil.py", line 56, in nested raise exc[1] File "/home/teuthworker/src/git.ceph.com_teuthology_e6d9099ce03dc21193fda653159ec82465c3a94a/teuthology/contextutil.py", line 48, in nested if exit(*exc): ^^^^^^^^^^ File "/usr/lib/python3.12/contextlib.py", line 158, in __exit__ self.gen.throw(value) File "/home/teuthworker/src/github.com_ceph_ceph-c_f90f4eaebb28b619cf76e80cbbe997656da69579/qa/tasks/ceph.py", line 346, in valgrind_post raise valgrind_exception Exception: valgrind error: Leak_StillReachable malloc malloc _dl_new_object 2026-02-24T18:38:10.720 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2026-02-24T18:38:10.744 DEBUG:teuthology.run_tasks:Unwinding manager install 2026-02-24T18:38:10.758 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2026-02-24T18:38:10.759 DEBUG:teuthology.orchestra.run.trial038:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2026-02-24T18:38:10.868 DEBUG:teuthology.run_tasks:Unwinding manager clock 2026-02-24T18:38:10.884 INFO:teuthology.task.clock:Checking final clock skew... 2026-02-24T18:38:10.884 DEBUG:teuthology.orchestra.run.trial038:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-24T18:38:10.901 INFO:teuthology.orchestra.run.trial038.stderr:bash: line 1: ntpq: command not found 2026-02-24T18:38:10.942 INFO:teuthology.orchestra.run.trial038.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-24T18:38:10.942 INFO:teuthology.orchestra.run.trial038.stdout:=============================================================================== 2026-02-24T18:38:10.942 INFO:teuthology.orchestra.run.trial038.stdout:^- time2.lshiy.com 2 7 377 101 +392us[+2437ns] +/- 132ms 2026-02-24T18:38:10.942 INFO:teuthology.orchestra.run.trial038.stdout:^* time.cloudflare.com 3 7 377 43 -66us[ -458us] +/- 6703us 2026-02-24T18:38:10.942 INFO:teuthology.orchestra.run.trial038.stdout:^- ec.sedley.net 2 7 377 42 +321us[ +321us] +/- 36ms 2026-02-24T18:38:10.942 INFO:teuthology.orchestra.run.trial038.stdout:^- 144.202.41.38.vultruserc> 2 7 377 39 +1547us[+1547us] +/- 37ms 2026-02-24T18:38:10.942 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2026-02-24T18:38:10.954 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2026-02-24T18:38:10.955 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2026-02-24T18:38:10.968 DEBUG:teuthology.orchestra.run.trial038:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2026-02-24T18:38:11.038 DEBUG:teuthology.orchestra.run.trial038:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2026-02-24T18:38:11.064 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T18:38:11.065 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2026-02-24T18:38:11.080 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2026-02-24T18:38:11.094 INFO:teuthology.task.internal:Duration was 1454.313124 seconds 2026-02-24T18:38:11.095 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2026-02-24T18:38:11.110 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2026-02-24T18:38:11.110 DEBUG:teuthology.orchestra.run.trial038:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2026-02-24T18:38:11.146 INFO:teuthology.orchestra.run.trial038.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-24T18:38:11.396 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2026-02-24T18:38:11.396 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@trial038.front.sepia.ceph.com 2026-02-24T18:38:11.396 DEBUG:teuthology.orchestra.run.trial038:> grep -E --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | grep -E -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | grep -E -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2026-02-24T18:38:11.454 INFO:teuthology.task.internal.syslog:Gathering journactl... 2026-02-24T18:38:11.454 DEBUG:teuthology.orchestra.run.trial038:> sudo journalctl > /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-24T18:38:11.557 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2026-02-24T18:38:11.558 DEBUG:teuthology.orchestra.run.trial038:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --max-args=1 --max-procs=0 --verbose --no-run-if-empty -- gzip -5 --verbose -- 2026-02-24T18:38:11.581 INFO:teuthology.orchestra.run.trial038.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-24T18:38:11.582 INFO:teuthology.orchestra.run.trial038.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-24T18:38:11.582 INFO:teuthology.orchestra.run.trial038.stderr:/home/ubuntu/cephtest/archive/syslog/misc.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/misc.log.gz 2026-02-24T18:38:11.582 INFO:teuthology.orchestra.run.trial038.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-24T18:38:11.583 INFO:teuthology.orchestra.run.trial038.stderr:/home/ubuntu/cephtest/archive/syslog/kern.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2026-02-24T18:38:11.593 INFO:teuthology.orchestra.run.trial038.stderr:/home/ubuntu/cephtest/archive/syslog/journalctl.log: 83.2% -- replaced with /home/ubuntu/cephtest/archive/syslog/journalctl.log.gz 2026-02-24T18:38:11.595 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2026-02-24T18:38:11.614 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2026-02-24T18:38:11.614 DEBUG:teuthology.orchestra.run.trial038:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2026-02-24T18:38:11.659 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2026-02-24T18:38:11.675 DEBUG:teuthology.orchestra.run.trial038:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2026-02-24T18:38:11.723 INFO:teuthology.orchestra.run.trial038.stdout:kernel.core_pattern = core 2026-02-24T18:38:11.737 DEBUG:teuthology.orchestra.run.trial038:> test -e /home/ubuntu/cephtest/archive/coredump 2026-02-24T18:38:11.791 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-24T18:38:11.791 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2026-02-24T18:38:11.811 INFO:teuthology.task.internal:Transferring archived files... 2026-02-24T18:38:11.811 DEBUG:teuthology.misc:Transferring archived files from trial038:/home/ubuntu/cephtest/archive to /home/teuthworker/mnt/teuthology/yuriw-2026-02-24_16:22:43-rados-wip-yuri6-testing-2026-02-22-1629-squid-distro-default-trial/68639/remote/trial038 2026-02-24T18:38:11.811 DEBUG:teuthology.orchestra.run.trial038:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2026-02-24T18:38:11.863 INFO:teuthology.task.internal:Removing archive directory... 2026-02-24T18:38:11.863 DEBUG:teuthology.orchestra.run.trial038:> rm -rf -- /home/ubuntu/cephtest/archive 2026-02-24T18:38:11.912 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2026-02-24T18:38:11.933 INFO:teuthology.task.internal:Not uploading archives. 2026-02-24T18:38:11.933 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2026-02-24T18:38:11.950 INFO:teuthology.task.internal:Tidying up after the test... 2026-02-24T18:38:11.950 DEBUG:teuthology.orchestra.run.trial038:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2026-02-24T18:38:11.969 INFO:teuthology.orchestra.run.trial038.stdout: 920834 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 24 18:38 /home/ubuntu/cephtest 2026-02-24T18:38:11.970 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2026-02-24T18:38:11.987 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2026-02-24T18:38:12.016 INFO:teuthology.run:Summary data: description: rados/valgrind-leaks/{1-start 2-inject-leak/none centos_latest} duration: 1454.3131239414215 failure_reason: '"2026-02-24T18:31:56.256593+0000 mon.a (mon.0) 62 : cluster [WRN] Health check failed: 2 OSD(s) experiencing slow operations in BlueStore (BLUESTORE_SLOW_OP_ALERT)" in cluster log' flavor: default owner: scheduled_yuriw@soko04.front.sepia.ceph.com sentry_event: null status: fail success: false 2026-02-24T18:38:12.016 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com 2026-02-24T18:38:12.097 INFO:teuthology.run:FAIL